Learn Artificial Intelligence and Machine Learning from basics to advanced. Build ML models and understand AI concepts.

Python for Data Science

Introduction to AI, neural networks, and AI concepts

AI Fundamentals

Supervised learning, unsupervised learning, and model training

Machine Learning Basics

Neural networks, CNNs, RNNs, and TensorFlow

Deep Learning

Build real-world ML projects and deploy models

ML Projects

AI & Machine Learning Path

Learn AI and Artificial Intelligence concepts, applications, and modern AI technologies.

Learn AI basics.

## What is AI?

AI means Artificial Intelligence. It makes computers smart like humans.

## Simple Example

When you talk to Siri or Alexa, that's AI!

```
You: "What's the weather in Boston?"
AI: "It's 72°F and sunny in Boston!"
```

## Where Do We Use AI?

- **Phones**: Face unlock, voice assistants
- **Apps**: Netflix recommendations, Spotify playlists
- **Cars**: Self-driving features
- **Shopping**: Amazon product suggestions

## Types of AI

**Narrow AI** - Does one specific task (like playing chess)
**General AI** - Does many tasks like humans (not yet created!)

## How AI Learns

AI learns from examples, like a kid learning to recognize dogs by seeing many dog pictures.

## Remember

- AI is everywhere today
- It makes our lives easier
- AI learns from data

Understand what Artificial Intelligence is and how it works.

What is AI?

Know the difference.

## AI vs Machine Learning

**AI** = Making computers smart
**Machine Learning (ML)** = How AI learns from data

ML is a part of AI!

## Simple Analogy

- **AI** = Entire car
- **ML** = The engine that powers it

## Example

**AI Application**: Email spam filter

**ML Part**: Learning which emails are spam by looking at thousands of examples

## Real-World Example

Netflix using AI:
- **AI Goal**: Recommend movies you'll like
- **ML Method**: Learns from what you watched before

## Deep Learning

Deep Learning is a special type of ML that uses neural networks (inspired by human brain).

**AI > Machine Learning > Deep Learning**

## Remember

- ML is how AI learns
- Deep Learning is advanced ML
- All are related but different

Understand the difference between AI and Machine Learning.

AI vs Machine Learning

Understand AI process.

## The Learning Process

1. **Get Data**: Collect examples
2. **Train**: AI learns patterns
3. **Test**: Check if AI learned correctly
4. **Use**: AI makes predictions

## Example: Teaching AI to Recognize Cats

**Step 1**: Show 10,000 cat pictures
**Step 2**: AI finds patterns (fur, ears, whiskers)
**Step 3**: Test with new cat pictures
**Step 4**: AI can now identify cats!

## Training vs Inference

**Training**: Teaching AI (takes time)
**Inference**: AI using what it learned (fast)

Like studying for exam (training) vs taking exam (inference).

## Data is Key

More data = Better AI

If you show AI only 10 pictures, it won't learn well.
Show it 1 million pictures, it becomes very accurate!

## AI Mistakes

AI can make mistakes, especially:
- With new types of data
- When training data was limited
- When data had errors

## Remember

- AI learns from examples
- More data = better results
- Training takes time, using is fast

How AI Works

Three main types of learning.

## 1. Supervised Learning

AI learns from labeled examples.

**Example**: Email spam detection
- Show AI 1000 emails labeled "spam" or "not spam"
- AI learns the difference
- Now AI can classify new emails

**Real Use**: House price prediction in Miami

## 2. Unsupervised Learning

AI finds patterns without labels.

**Example**: Customer grouping
- Give AI customer data
- AI groups similar customers automatically
- No labels needed!

**Real Use**: Netflix grouping similar movies

## 3. Reinforcement Learning

AI learns by trial and error.

**Example**: Teaching AI to play chess
- AI tries moves
- Gets reward for good moves
- Gets penalty for bad moves
- Learns which moves are best

**Real Use**: Self-driving cars, game AI

## Which to Use?

- **Supervised**: When you have labeled data
- **Unsupervised**: When you want to find patterns
- **Reinforcement**: When AI needs to make decisions

## Remember

- Supervised needs labeled data
- Unsupervised finds patterns
- Reinforcement learns by trying

Learn about supervised, unsupervised, and reinforcement learning.

Types of Machine Learning

AI everywhere around you.

## Your Phone

**Face ID**: AI recognizes your face
**Siri/Google Assistant**: AI understands your voice
**Autocorrect**: AI predicts what you meant to type

## Social Media

**Instagram**: AI suggests posts you might like
**TikTok**: AI picks videos for your feed
**Facebook**: AI shows relevant ads

## Shopping

**Amazon**: AI recommends products
**Price comparison**: AI finds best deals
**Virtual try-on**: AI shows how clothes look on you

## Entertainment

**Netflix**: AI suggests shows
**Spotify**: AI creates playlists
**YouTube**: AI recommends videos

## Transportation

**Google Maps**: AI finds fastest route
**Uber**: AI calculates price and matches drivers
**Tesla**: AI assists with driving

## Home

**Smart thermostats**: AI learns your preferences
**Alexa**: AI controls smart home devices
**Roomba**: AI vacuums your house

## Remember

- AI is in most apps you use
- It makes things personalized
- AI learns your preferences

See how AI is used in everyday applications.

AI in Daily Life

Start learning AI today.

## What You Need to Know

**Math**: Basic algebra (nothing scary!)
**Programming**: Python basics
**Logic**: Problem-solving mindset

## Start with Python

Python is the #1 language for AI.

```python
# Your first AI-related code
print("Hello AI from San Francisco!")
```

## Key Libraries

**NumPy**: Math operations
**Pandas**: Data handling
**Scikit-learn**: Machine learning
**TensorFlow/PyTorch**: Deep learning

## Learning Path

1. Learn Python basics (2-4 weeks)
2. Learn data manipulation (1-2 weeks)
3. Start with simple ML models (2-3 weeks)
4. Build small projects

## First Project Ideas

- Predict house prices in Austin
- Classify spam emails
- Recommend movies
- Recognize handwritten digits

## Free Resources

- Google Colab (run code in browser)
- Kaggle (datasets and tutorials)
- YouTube tutorials
- Online courses

## Remember

- Start small and simple
- Practice with real projects
- Don't get overwhelmed

Getting Started with AI

Data is fuel for AI.

## Why Data Matters

AI = Smart only when trained with good data

Bad data = Bad AI
Good data = Good AI

## Types of Data

**Numbers**: Age, price, temperature
**Text**: Reviews, emails, articles
**Images**: Photos, X-rays, satellite images
**Audio**: Voice recordings, music
**Video**: Movies, surveillance footage

## Data Quality

**Good Data**:
- Accurate
- Complete
- Relevant
- Enough quantity

**Bad Data**:
- Missing values
- Errors
- Outdated
- Too little

## Example

Training AI to predict apartment rent in Chicago:

**Good Data**: 10,000 apartments with size, location, amenities
**Bad Data**: 50 apartments with missing information

## Data Preparation

1. **Collect**: Gather data
2. **Clean**: Remove errors
3. **Organize**: Structure properly
4. **Split**: Train data + Test data

## Real Example

Netflix has data on:
- What you watched
- When you watched
- What you rated
- What you skipped

This data helps AI recommend shows!

## Remember

- More quality data = better AI
- Clean data before using
- Data privacy is important

Understanding Data for AI

What is an AI model?

## AI Model Definition

A model is like a recipe that AI uses to make predictions.

**Recipe (Model)**: Steps to bake a cake
**Ingredients (Data)**: Flour, eggs, sugar
**Result (Prediction)**: Delicious cake!

## How Models Work

1. **Input**: Give data to model
2. **Process**: Model analyzes
3. **Output**: Model gives answer

## Example: Email Spam Filter

**Input**: Email text
**Model**: Checks for spam patterns
**Output**: "Spam" or "Not Spam"

## Training a Model

Training = Teaching the model

Like teaching a dog tricks:
- Show examples
- Reward correct behavior
- Model learns patterns

## Model Types

**Simple Models**: Linear Regression (straight line predictions)
**Complex Models**: Neural Networks (brain-like learning)

## Choosing a Model

**Small Data**: Use simple models
**Large Data**: Can use complex models

Start simple, then get fancy!

## Saving Models

Once trained, save your model:
- Use it later
- Share with others
- Don't retrain every time

## Remember

- Models are AI's brain
- Training = learning process
- Start with simple models

Understand what AI models are and how they work.

AI Models Explained

Understanding brain-like AI.

## What are Neural Networks?

AI systems inspired by human brain.

Just like brain has neurons, neural networks have artificial neurons!

## How They Work

**Layers**:
- Input layer (receives data)
- Hidden layers (process data)
- Output layer (gives answer)

## Simple Example

Recognizing if image is a cat:

**Input**: Picture pixels
**Hidden layers**: Find patterns (ears, whiskers, fur)
**Output**: "Cat" or "Not Cat"

## Neurons and Connections

Each neuron:
1. Receives input
2. Processes it
3. Sends output

Like passing a message through a chain of people!

## Weights

Connections have weights (importance levels).

During training, weights adjust to improve accuracy.

## Real Example

Face recognition in your phone:
- Takes your face photo
- Neural network processes
- Learns your unique features
- Unlocks phone when it sees you!

## Deep Learning

Neural networks with many layers = Deep Learning

More layers = Can learn more complex patterns

## Remember

- Neural networks mimic brain
- Made of layers and neurons
- Learn by adjusting weights

Neural Networks Basics

Train AI the right way.

## Split Your Data

**Training Data (80%)**: Teach AI
**Testing Data (20%)**: Check if AI learned

Like studying with practice questions, then taking the real exam!

## Why Split?

If AI sees test data during training, it's cheating!

We want AI to work on NEW data it hasn't seen.

## Training Process

1. Show training data
2. AI makes predictions
3. Check if predictions are correct
4. AI adjusts itself
5. Repeat until accurate

## Example

Teaching AI to predict house prices in Denver:

**Training**: 800 houses
**Testing**: 200 houses

Train on 800, test on 200 new houses.

## Overfitting Problem

**Overfitting**: AI memorizes training data but fails on new data

Like memorizing answers instead of understanding concepts!

## Solution to Overfitting

- Use more data
- Simpler model
- Validation techniques

## Validation Data

Sometimes split into 3 parts:
- Training (60%)
- Validation (20%)
- Testing (20%)

## Measuring Accuracy

**Good**: 95% accuracy on test data
**Bad**: 99% on training, 60% on test (overfitting!)

## Remember

- Always split data
- Test on unseen data
- Watch for overfitting

Learn how to train and test AI models properly.

Training and Testing AI

Build responsible AI.

## Why Ethics Matter

AI affects real people's lives:
- Loan approvals
- Job applications
- Medical diagnoses
- Criminal justice

## AI Bias

AI learns from data. If data has bias, AI will too!

## Example of Bias

Hiring AI trained on past hires:
- If company mostly hired men
- AI learns: men = better candidates
- AI discriminates against women

**This is wrong!**

## Sources of Bias

1. **Historical data**: Past discrimination
2. **Incomplete data**: Missing groups
3. **Biased labels**: Human prejudice

## Real Case

Amazon's hiring AI showed bias against women. They stopped using it!

## Fairness in AI

AI should treat everyone fairly:
- All races equally
- All genders equally
- All ages equally

## Privacy Concerns

AI often uses personal data:
- Face recognition
- Location tracking
- Browsing history

**Balance**: Useful AI vs Privacy protection

## Building Fair AI

- Use diverse data
- Test for bias
- Include diverse teams
- Be transparent

## Remember

- AI can be biased
- Check for fairness
- Protect user privacy
- Build responsible AI

AI Ethics and Bias

Tools to build AI.

## Cloud Platforms

**Google Colab**: Free Python notebooks
**Kaggle**: Datasets and competitions
**AWS SageMaker**: Amazon's AI platform
**Google Cloud AI**: Google's AI tools

## Programming Libraries

**Scikit-learn**: Easy machine learning
**TensorFlow**: Google's deep learning
**PyTorch**: Facebook's deep learning
**Keras**: Simple neural networks

## No-Code AI Tools

**Teachable Machine**: Train models without code
**AutoML**: Automated machine learning
**Azure ML Studio**: Visual AI building

## Data Tools

**Pandas**: Data manipulation
**NumPy**: Math operations
**Matplotlib**: Data visualization

## Pre-trained Models

**ChatGPT API**: Use OpenAI's models
**Hugging Face**: Pre-trained NLP models
**Vision APIs**: Image recognition services

## Example: Using Scikit-learn

```python
from sklearn.linear_model import LinearRegression

# Simple prediction model
model = LinearRegression()
model.fit(X_train, y_train)
predictions = model.predict(X_test)
```

## Getting Started

1. **Start**: Google Colab (free, browser-based)
2. **Learn**: Scikit-learn (easy to use)
3. **Advanced**: TensorFlow/PyTorch

## Remember

- Start with simple tools
- Google Colab is great for beginners
- Many tools are free

AI Tools and Platforms

Teaching AI to see.

## What is Computer Vision?

Making computers understand images and videos.

Like teaching a computer to "see" like humans!

## How Computers See Images

Images are made of pixels (tiny dots).

Each pixel has a color value (0-255 for each: Red, Green, Blue).

AI reads these numbers!

## Common Tasks

**Image Classification**: What's in the image?
- Cat or dog?
- Apple or orange?

**Object Detection**: Where is it in the image?
- Find all cars in photo
- Locate faces

**Image Segmentation**: Outline exact shape
- Separate person from background
- Medical image analysis

## Real-World Uses

**Face Unlock**: iPhone Face ID
**Self-driving Cars**: Detect pedestrians, signs
**Medical**: Detect diseases in X-rays
**Retail**: Checkout without cashier
**Social Media**: Auto-tag friends in photos

## Example in Action

Instagram filters:
1. Detect your face
2. Find eyes, nose, mouth
3. Add filter effects

## Simple Concept

AI learns by seeing thousands of examples:
- Show 10,000 cat pictures
- AI learns cat features
- AI can identify new cat pictures

## Getting Started

Try with: Teachable Machine by Google
- Upload images
- Train model in browser
- Test immediately!

## Remember

- Computer vision = AI seeing images
- Used everywhere today
- Start with simple projects

Learn how AI sees and understands images.

Computer Vision Basics

Teaching AI to understand language.

## What is NLP?

Natural Language Processing = AI understanding human language.

Like teaching computers to read and understand English!

## Common NLP Tasks

**Sentiment Analysis**: Is text positive or negative?
- "Great product!" = Positive
- "Terrible service" = Negative

**Translation**: Convert between languages
- English to Spanish
- Japanese to English

**Chatbots**: Answer questions
- Customer service bots
- Siri, Alexa

**Text Summarization**: Make text shorter
- Summary of long article
- TLDR generator

## Real-World Examples

**Gmail**: Smart Reply suggestions
**Grammarly**: Grammar correction
**Siri/Alexa**: Voice commands
**Google Translate**: Language translation
**Spam Filters**: Detect spam emails

## How NLP Works

1. **Clean text**: Remove extra spaces, punctuation
2. **Tokenize**: Split into words
3. **Understand meaning**: AI analyzes
4. **Generate output**: Response or action

## Example

Analyzing customer review:

**Input**: "The hotel in Miami was amazing! Great service."

**NLP Process**:
- Detects positive words: "amazing", "great"
- Sentiment: Positive
- Topic: Hotel service

**Output**: Positive review about service

## Challenges

- Sarcasm: "Oh great, another delay" (negative, not positive!)
- Context: "Bank" (river bank or money bank?)
- Slang: "That's lit!" (good)

## Getting Started

Try ChatGPT or similar tools to see NLP in action!

## Remember

- NLP = AI understanding text
- Used in chatbots, translation
- Powers voice assistants

Natural Language Processing Basics

Build your first AI projects.

## 1. House Price Predictor

Predict house prices in Austin based on size and location.

**What you'll learn**: Linear Regression
**Tools**: Scikit-learn, Pandas
**Time**: 1-2 days

## 2. Spam Email Classifier

Detect if email is spam or not.

**What you'll learn**: Text classification
**Tools**: Scikit-learn, Python
**Time**: 2-3 days

## 3. Movie Recommender

Suggest movies based on user ratings.

**What you'll learn**: Recommendation systems
**Tools**: Pandas, Scikit-learn
**Time**: 3-4 days

## 4. Handwritten Digit Recognition

Recognize numbers 0-9 from images.

**What you'll learn**: Image classification
**Tools**: TensorFlow/PyTorch
**Dataset**: MNIST (famous beginner dataset)
**Time**: 4-5 days

## 5. Sentiment Analyzer

Analyze if movie reviews are positive or negative.

**What you'll learn**: NLP, sentiment analysis
**Tools**: Scikit-learn, NLTK
**Time**: 2-3 days

## 6. Iris Flower Classifier

Classify iris flowers into species.

**What you'll learn**: Classification basics
**Tools**: Scikit-learn
**Dataset**: Iris dataset (classic beginner dataset)
**Time**: 1 day

## 7. Customer Segmentation

Group customers by shopping behavior.

**What you'll learn**: Clustering
**Tools**: Scikit-learn, K-means
**Time**: 2-3 days

## Getting Datasets

- **Kaggle**: Thousands of free datasets
- **UCI ML Repository**: Classic datasets
- **Google Dataset Search**: Find any dataset

## Tips for Success

1. Start with simplest project
2. Follow tutorials first
3. Modify and experiment
4. Share on GitHub
5. Move to next project

## Remember

- Start small and simple
- Complete one project at a time
- Practice makes perfect
- Build portfolio of projects

Simple AI projects to start your learning journey.

AI Project Ideas for Beginners

Predict with straight lines.

## What is Linear Regression?

Finding the best straight line to predict values.

Like predicting house prices based on size!

## Simple Example

**Data**: Houses in Seattle
- 1000 sq ft = $300,000
- 1500 sq ft = $450,000
- 2000 sq ft = $600,000

**Pattern**: Bigger house = Higher price
**Line**: Best fit through these points

## The Formula

```
y = mx + b

y = price (what we predict)
x = size (what we know)
m = slope
b = y-intercept
```

## Python Code

```python
from sklearn.linear_model import LinearRegression
import numpy as np

# Data
X = np.array([[1000], [1500], [2000]])  # Size
y = np.array([300000, 450000, 600000])   # Price

# Train model
model = LinearRegression()
model.fit(X, y)

# Predict
new_house = [[1800]]
price = model.predict(new_house)
print(f"Predicted price: ${price[0]:,.0f}")
```

## When to Use

- Predicting numbers
- Clear relationship between variables
- Simple and fast

## Limitations

- Only works for linear relationships
- Can't handle complex patterns
- Sensitive to outliers

## Remember

- Best for simple predictions
- Fast and easy to understand
- Start here for regression problems

Learn linear regression for prediction tasks.

Linear Regression

Categorize data with AI.

## What is Classification?

Putting things into categories.

**Examples**:
- Email: Spam or Not Spam
- Fruit: Apple or Orange
- Tumor: Benign or Malignant

## Popular Algorithms

**Logistic Regression**: Simple, fast
**Decision Trees**: Easy to visualize
**Random Forest**: More accurate
**SVM**: Good for complex data
**Neural Networks**: Most powerful

## Simple Example

Classify fruits based on weight and color:

```python
from sklearn.tree import DecisionTreeClassifier

# Data: [weight_grams, color_score]
X = [[150, 1], [170, 1], [140, 0], [130, 0]]
y = ['apple', 'apple', 'orange', 'orange']

# Train
model = DecisionTreeClassifier()
model.fit(X, y)

# Predict
new_fruit = [[160, 1]]
result = model.predict(new_fruit)
print(f"This is an: {result[0]}")
```

## Binary vs Multi-class

**Binary**: 2 categories (Yes/No)
**Multi-class**: 3+ categories (Apple/Orange/Banana)

## Evaluation Metrics

**Accuracy**: How many correct predictions
**Precision**: True positives / All positives
**Recall**: True positives / All actual positives

## Real Applications

- Medical diagnosis
- Credit card fraud detection
- Face recognition
- Customer churn prediction

## Remember

- Choose algorithm based on data
- Start with simple models
- Evaluate properly

Classification Algorithms

Tree-based decision making.

## What are Decision Trees?

Like a flowchart that makes decisions!

Each branch asks a question, leading to final decision.

## Example Decision Process

**Goal**: Should I go to the beach in Miami?

```
Is it sunny?
├─ Yes → Is temperature > 70°F?
│         ├─ Yes → Go to beach!
│         └─ No → Stay home
└─ No → Stay home
```

## How It Works

1. Find best question to split data
2. Create branches
3. Repeat for each branch
4. Stop when data is pure or max depth reached

## Python Example

```python
from sklearn.tree import DecisionTreeClassifier

# Weather data: [temperature, sunny]
X = [[85, 1], [70, 1], [60, 0], [50, 0]]
y = ['beach', 'beach', 'home', 'home']

model = DecisionTreeClassifier(max_depth=2)
model.fit(X, y)

# Predict
prediction = model.predict([[75, 1]])
print(prediction)  # 'beach'
```

## Advantages

- Easy to understand
- No data scaling needed
- Works with numbers and categories
- Visual representation

## Disadvantages

- Can overfit easily
- Unstable (small data change = different tree)
- Not great for very complex data

## Hyperparameters

**max_depth**: Maximum tree depth
**min_samples_split**: Minimum samples to split
**min_samples_leaf**: Minimum samples in leaf

## Remember

- Very intuitive algorithm
- Prone to overfitting
- Use Random Forest for better results

Decision Trees

Many trees = Better predictions.

## What is Random Forest?

Collection of many decision trees working together!

Like asking 100 experts instead of 1.

## How It Works

1. Create many decision trees
2. Each tree votes on answer
3. Majority vote wins

## Example

Predicting if customer will buy product in Austin:

- Tree 1: Yes
- Tree 2: No
- Tree 3: Yes
- Tree 4: Yes
- Tree 5: Yes

**Final answer**: Yes (3/5 voted yes)

## Python Code

```python
from sklearn.ensemble import RandomForestClassifier

# Customer data: [age, income_k]
X = [[25, 40], [30, 60], [35, 80], [40, 100]]
y = [0, 0, 1, 1]  # 0=No buy, 1=Buy

# Train with 100 trees
model = RandomForestClassifier(n_estimators=100)
model.fit(X, y)

# Predict
customer = [[32, 70]]
prediction = model.predict(customer)
print("Will buy!" if prediction[0] == 1 else "Won't buy")
```

## Why Better Than Single Tree?

- Less overfitting
- More stable
- More accurate
- Handles noise better

## Disadvantages

- Slower to train
- Harder to interpret
- Uses more memory

## Feature Importance

Random Forest shows which features matter most!

```python
importances = model.feature_importances_
print(f"Age importance: {importances[0]}")
print(f"Income importance: {importances[1]}")
```

## Remember

- Very powerful algorithm
- Great for most problems
- Trade-off: accuracy vs speed

Improve predictions using multiple decision trees.

Random Forests

Learn from your neighbors.

## What is KNN?

"Tell me who your friends are, I'll tell you who you are"

Classify based on K nearest similar examples.

## Simple Example

Classifying new house in Denver:

Your house: 1500 sq ft, $400k

**3 Nearest Neighbors**:
- House A: 1450 sq ft, $390k → Affordable
- House B: 1520 sq ft, $410k → Affordable  
- House C: 1480 sq ft, $395k → Affordable

**Result**: Your house is "Affordable" (3/3 agree)

## How It Works

1. Calculate distance to all points
2. Find K nearest neighbors
3. Take majority vote
4. Assign category

## Python Code

```python
from sklearn.neighbors import KNeighborsClassifier

# Data: [size_sqft, price_k]
X = [[1000, 300], [1500, 400], [2000, 600], [2500, 800]]
y = ['affordable', 'affordable', 'expensive', 'expensive']

# K=3 means check 3 nearest neighbors
model = KNeighborsClassifier(n_neighbors=3)
model.fit(X, y)

# Predict
new_house = [[1800, 550]]
result = model.predict(new_house)
print(result)  # 'expensive'
```

## Choosing K

**K=1**: Too sensitive to noise
**K=100**: Too general
**K=3 to 7**: Usually good starting point

## Distance Metrics

**Euclidean**: Straight line distance (most common)
**Manhattan**: Grid-like distance
**Cosine**: Angle-based

## Advantages

- Simple to understand
- No training needed
- Works for classification and regression

## Disadvantages

- Slow for large datasets
- Sensitive to feature scaling
- Doesn't work well in high dimensions

## Remember

- Scale your features!
- Start with K=5
- Fast to implement, slow to predict

Classify based on nearest similar data points.

K-Nearest Neighbors

Draw the best dividing line.

## What is SVM?

Finds the best line/plane to separate categories.

Like drawing a fence between cats and dogs!

## Simple Concept

Imagine separating red and blue dots on paper:

**Goal**: Draw line with maximum space on both sides

SVM finds the line with biggest "margin" (gap).

## How It Works

1. Find possible separating lines
2. Choose line with maximum margin
3. Use support vectors (closest points)

## Python Example

```python
from sklearn.svm import SVC

# Customer data: [age, purchases]
X = [[22, 2], [25, 3], [45, 10], [50, 12]]
y = [0, 0, 1, 1]  # 0=Regular, 1=Premium

model = SVC(kernel='linear')
model.fit(X, y)

# Predict
customer = [[35, 7]]
result = model.predict(customer)
print("Premium" if result[0] == 1 else "Regular")
```

## Kernels

**Linear**: Straight line separation
**RBF**: Curved boundaries (most popular)
**Polynomial**: Complex curves

## When to Use

- Clear separation between classes
- Medium-sized datasets
- High-dimensional data

## Advantages

- Effective in high dimensions
- Memory efficient
- Versatile (different kernels)

## Disadvantages

- Slow for large datasets
- Hard to interpret
- Sensitive to feature scaling

## Real Applications

- Text classification
- Image recognition
- Bioinformatics

## Remember

- Scale features first!
- Start with RBF kernel
- Great for clear separations

Support Vector Machines

Group similar things together.

## What is K-Means?

Automatically groups similar data points.

No labels needed! (Unsupervised learning)

## Real Example

Group customers in San Francisco by shopping behavior:

**Cluster 1**: Young, low spending
**Cluster 2**: Middle-age, medium spending
**Cluster 3**: Older, high spending

## How It Works

1. Choose K (number of groups)
2. Place K random center points
3. Assign each point to nearest center
4. Move centers to group average
5. Repeat until stable

## Python Code

```python
from sklearn.cluster import KMeans
import numpy as np

# Customer data: [age, annual_purchases]
X = np.array([
    [25, 10], [22, 8], [50, 50], 
    [48, 55], [70, 80], [72, 85]
])

# Find 3 clusters
model = KMeans(n_clusters=3)
model.fit(X)

# Get cluster assignments
labels = model.labels_
print(labels)  # [0, 0, 1, 1, 2, 2]

# Predict cluster for new customer
new_customer = [[30, 15]]
cluster = model.predict(new_customer)
print(f"Customer belongs to cluster {cluster[0]}")
```

## Choosing K

Use "Elbow Method":
- Try different K values
- Plot total distance
- Choose "elbow" point

## Applications

- Customer segmentation
- Image compression
- Document clustering
- Anomaly detection

## Advantages

- Simple and fast
- Scales to large data
- Easy to implement

## Disadvantages

- Must choose K manually
- Sensitive to initial centers
- Assumes spherical clusters

## Remember

- Start with 3-5 clusters
- Scale features first
- Try multiple runs

Clustering with K-Means

Simplify complex data.

## What is PCA?

Reduces number of features while keeping important info.

Like summarizing a long book into key points!

## Why Use PCA?

- Too many features slow down training
- Visualization (can't plot 100 dimensions!)
- Remove noise
- Reduce storage

## Real Example

Customer data with 50 features → Reduce to 5 key features

Maybe those 5 capture 95% of important information!

## How It Works

1. Find directions with most variation
2. Project data onto those directions
3. Keep top N components

## Python Code

```python
from sklearn.decomposition import PCA
import numpy as np

# Data with many features
X = np.array([
    [2.5, 2.4, 1.5, 3.2],
    [0.5, 0.7, 0.9, 1.1],
    [2.2, 2.9, 1.8, 3.5],
    [1.9, 2.2, 1.6, 2.8]
])

# Reduce to 2 components
pca = PCA(n_components=2)
X_reduced = pca.fit_transform(X)

print(f"Original shape: {X.shape}")
print(f"Reduced shape: {X_reduced.shape}")

# Check variance explained
print(f"Variance explained: {pca.explained_variance_ratio_}")
```

## Choosing Components

Keep components that explain 95% of variance:

```python
pca = PCA(n_components=0.95)  # Keep 95% variance
X_reduced = pca.fit_transform(X)
```

## Applications

- Image compression
- Noise reduction
- Feature extraction
- Data visualization

## Advantages

- Fast
- No parameters to tune
- Interpretable

## Disadvantages

- Loses some information
- Assumes linear relationships
- Hard to interpret components

## Remember

- Use before training models
- Great for visualization
- Try keeping 95% variance

Reduce dimensions while keeping important information.

Principal Component Analysis

Make better features = Better AI.

## What is Feature Engineering?

Creating new features from existing data to improve model performance.

Like giving AI better hints!

## Why Important?

Good features > Complex models

Simple model + Great features > Complex model + Bad features

## Common Techniques

**1. Creating New Features**

```python
# From date, create multiple features
df['year'] = df['date'].dt.year
df['month'] = df['date'].dt.month
df['day_of_week'] = df['date'].dt.dayofweek
df['is_weekend'] = df['day_of_week'].isin([5, 6])
```

**2. Combining Features**

```python
# House data
df['price_per_sqft'] = df['price'] / df['sqft']
df['total_rooms'] = df['bedrooms'] + df['bathrooms']
```

**3. Binning**

```python
# Age groups
df['age_group'] = pd.cut(df['age'], 
                         bins=[0, 18, 35, 50, 100],
                         labels=['child', 'young', 'middle', 'senior'])
```

**4. Encoding Categories**

```python
# One-hot encoding
df_encoded = pd.get_dummies(df, columns=['city'])
# Miami → [1, 0, 0]
# Austin → [0, 1, 0]
# Denver → [0, 0, 1]
```

## Real Example - House Prices

Original features: size, bedrooms

New features:
- price_per_sqft = price / size
- has_garage = bedrooms > 2
- age_of_house = current_year - built_year

## Feature Scaling

```python
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
```

## Feature Selection

Remove features that don't help:

```python
from sklearn.feature_selection import SelectKBest

selector = SelectKBest(k=10)
X_selected = selector.fit_transform(X, y)
```

## Remember

- Domain knowledge helps
- Try different combinations
- Remove correlated features
- Scale numeric features

Feature Engineering

Test models properly.

## What is Cross-Validation?

Better way to test model performance than single train/test split.

Like taking multiple practice exams instead of just one!

## Why Needed?

Single split might be lucky or unlucky:
- Lucky: Easy test data → Overestimate accuracy
- Unlucky: Hard test data → Underestimate accuracy

## K-Fold Cross-Validation

Split data into K parts, test K times:

**5-Fold Example**:
1. Use fold 1 as test, others as train
2. Use fold 2 as test, others as train
3. Use fold 3 as test, others as train
4. Use fold 4 as test, others as train
5. Use fold 5 as test, others as train

**Final score**: Average of all 5 tests

## Python Code

```python
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier()

# 5-fold cross-validation
scores = cross_val_score(model, X, y, cv=5)

print(f"Scores: {scores}")
print(f"Average: {scores.mean():.2f}")
print(f"Std Dev: {scores.std():.2f}")
```

## Stratified K-Fold

Keeps class proportions in each fold:

```python
from sklearn.model_selection import StratifiedKFold

skf = StratifiedKFold(n_splits=5)
scores = cross_val_score(model, X, y, cv=skf)
```

## Leave-One-Out

Use each sample as test once:

Good for small datasets, but slow!

## Time Series Split

For time-based data:

```python
from sklearn.model_selection import TimeSeriesSplit

tscv = TimeSeriesSplit(n_splits=5)
scores = cross_val_score(model, X, y, cv=tscv)
```

## Best Practices

- Use 5 or 10 folds
- Stratified for imbalanced data
- Time series split for temporal data
- More folds = more reliable, but slower

## Remember

- More reliable than single split
- Standard is 5-fold
- Reports average performance

Cross-Validation

Fine-tune your models.

## What are Hyperparameters?

Settings you choose BEFORE training.

**Model parameters**: Learned during training
**Hyperparameters**: Set by you

## Examples

**Random Forest**:
- n_estimators (number of trees)
- max_depth (tree depth)
- min_samples_split

**Neural Network**:
- learning_rate
- batch_size
- number of layers

## Grid Search

Try all combinations:

```python
from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForestClassifier

# Define parameter grid
param_grid = {
    'n_estimators': [50, 100, 200],
    'max_depth': [5, 10, 15],
    'min_samples_split': [2, 5, 10]
}

model = RandomForestClassifier()

# Try all combinations
grid_search = GridSearchCV(model, param_grid, cv=5)
grid_search.fit(X, y)

print(f"Best params: {grid_search.best_params_}")
print(f"Best score: {grid_search.best_score_:.2f}")
```

## Random Search

Try random combinations (faster!):

```python
from sklearn.model_selection import RandomizedSearchCV

random_search = RandomizedSearchCV(
    model, 
    param_grid, 
    n_iter=20,  # Try 20 random combinations
    cv=5
)
random_search.fit(X, y)
```

## Bayesian Optimization

Smart search that learns from previous tries:

```python
# Using optuna library
import optuna

def objective(trial):
    n_estimators = trial.suggest_int('n_estimators', 50, 200)
    max_depth = trial.suggest_int('max_depth', 5, 15)
    
    model = RandomForestClassifier(
        n_estimators=n_estimators,
        max_depth=max_depth
    )
    return cross_val_score(model, X, y, cv=5).mean()

study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=50)
```

## Best Practices

1. Start with default parameters
2. Use cross-validation
3. Random search first (fast)
4. Grid search for fine-tuning
5. Don't overtune on test data!

## Common Parameters to Tune

**Tree Models**: max_depth, n_estimators
**SVM**: C, gamma, kernel
**Neural Networks**: learning_rate, batch_size

## Remember

- Good features > Perfect hyperparameters
- Start simple
- Use random search for many parameters

Optimize model settings for best performance.

Hyperparameter Tuning

Fix unbalanced datasets.

## What is Imbalanced Data?

When one class has way more examples than others.

**Example**: Fraud detection
- 99,000 normal transactions
- 1,000 fraud transactions

## Why It's a Problem

Model learns to always predict majority class:

Predict "Not Fraud" for everything → 99% accuracy!
But catches 0% of actual fraud!

## Solution 1: Oversampling

Add more minority class examples:

```python
from imblearn.over_sampling import SMOTE

smote = SMOTE()
X_resampled, y_resampled = smote.fit_resample(X, y)

print(f"Original: {len(y)}")
print(f"After SMOTE: {len(y_resampled)}")
```

## Solution 2: Undersampling

Remove majority class examples:

```python
from imblearn.under_sampling import RandomUnderSampler

rus = RandomUnderSampler()
X_resampled, y_resampled = rus.fit_resample(X, y)
```

## Solution 3: Class Weights

Tell model to care more about minority class:

```python
from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(class_weight='balanced')
model.fit(X, y)
```

## Solution 4: Different Metrics

Don't use accuracy! Use:

**Precision**: Of predicted frauds, how many were real?
**Recall**: Of real frauds, how many did we catch?
**F1-Score**: Balance of precision and recall

```python
from sklearn.metrics import classification_report

predictions = model.predict(X_test)
print(classification_report(y_test, predictions))
```

## Real Example - Credit Card Fraud

```python
from imblearn.over_sampling import SMOTE
from sklearn.ensemble import RandomForestClassifier

# Apply SMOTE
smote = SMOTE()
X_train_balanced, y_train_balanced = smote.fit_resample(X_train, y_train)

# Train with class weights
model = RandomForestClassifier(class_weight='balanced')
model.fit(X_train_balanced, y_train_balanced)
```

## When to Use What

- **Lots of data**: Undersample
- **Little data**: Oversample (SMOTE)
- **Can't resample**: Use class weights

## Remember

- Never use accuracy alone
- Use F1-score or AUC
- Combine multiple techniques

Handling Imbalanced Data

Teamwork makes AI better.

## What is Ensemble Learning?

Combining multiple models to get better results.

**Idea**: Ask 10 doctors instead of 1!

## Types of Ensembles

**1. Bagging**: Train same model on different data
**2. Boosting**: Train models sequentially, fix mistakes
**3. Stacking**: Combine different model types

## Bagging Example - Random Forest

```python
from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier

base_model = DecisionTreeClassifier()
bagging = BaggingClassifier(
    base_model,
    n_estimators=10,  # 10 trees
    max_samples=0.8   # 80% of data each
)

bagging.fit(X_train, y_train)
```

## Boosting Example - AdaBoost

Focus on mistakes:

```python
from sklearn.ensemble import AdaBoostClassifier

model = AdaBoostClassifier(n_estimators=50)
model.fit(X_train, y_train)

# Each new model focuses on previous mistakes
```

## Gradient Boosting

Most powerful boosting method:

```python
from sklearn.ensemble import GradientBoostingClassifier

model = GradientBoostingClassifier(
    n_estimators=100,
    learning_rate=0.1,
    max_depth=3
)
model.fit(X_train, y_train)
```

## XGBoost - Industry Standard

```python
import xgboost as xgb

model = xgb.XGBClassifier(
    n_estimators=100,
    learning_rate=0.1,
    max_depth=5
)
model.fit(X_train, y_train)
```

## Voting Classifier

Combine different models:

```python
from sklearn.ensemble import VotingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import SVC

voting = VotingClassifier(
    estimators=[
        ('lr', LogisticRegression()),
        ('dt', DecisionTreeClassifier()),
        ('svc', SVC())
    ],
    voting='hard'  # Majority vote
)

voting.fit(X_train, y_train)
```

## When to Use

- **Bagging**: Reduce overfitting
- **Boosting**: Improve accuracy
- **Stacking**: Maximum performance

## Remember

- Ensemble usually better than single model
- XGBoost often wins competitions
- Trade-off: accuracy vs training time

Combine multiple models for better predictions.

Ensemble Learning

AI that sees images.

## What are CNNs?

Neural networks designed for images.

Like how your brain processes visual information!

## How CNNs Work

**Layers**:
1. **Convolution**: Detect features (edges, shapes)
2. **Pooling**: Reduce size, keep important info
3. **Dense**: Make final decision

## Real Example - Cat vs Dog

1. **Convolution**: Finds edges, fur patterns
2. **Pooling**: Keeps key features
3. **Dense**: Decides "Cat" or "Dog"

## Basic CNN in Python

```python
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense, Flatten

model = Sequential([
    # Convolution layer
    Conv2D(32, (3, 3), activation='relu', input_shape=(64, 64, 3)),
    
    # Pooling layer
    MaxPooling2D(pool_size=(2, 2)),
    
    # Another conv + pool
    Conv2D(64, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2, 2)),
    
    # Flatten for dense layers
    Flatten(),
    Dense(128, activation='relu'),
    Dense(1, activation='sigmoid')  # Output: cat or dog
])

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
```

## Convolution Explained

Slide a small filter over image:

**Filter**: 3x3 grid that detects patterns
**Output**: Feature map showing where pattern exists

## Pooling Explained

Shrink image while keeping important features:

**Max Pooling**: Take maximum value in each region
**Result**: Smaller image, same key info

## Famous CNN Architectures

- **LeNet**: Early CNN (1998)
- **AlexNet**: Won ImageNet (2012)
- **VGG**: Very deep (2014)
- **ResNet**: 152 layers! (2015)

## Applications

- Face recognition
- Self-driving cars
- Medical image analysis
- Quality control in manufacturing

## Transfer Learning

Use pre-trained models:

```python
from tensorflow.keras.applications import VGG16

# Load pre-trained model
base_model = VGG16(weights='imagenet', include_top=False)

# Add your own layers
# Train only your layers
```

## Remember

- CNNs are for images
- Convolution finds patterns
- Pooling reduces size
- Use transfer learning!

Convolutional Neural Networks

AI with memory.

## What are RNNs?

Neural networks that remember previous inputs.

Perfect for sequences: text, time series, speech!

## Why Special?

Regular neural networks forget previous inputs.

RNNs have memory - like reading a story!

## How RNNs Work

Each step:
1. Read current input
2. Remember previous state
3. Combine both
4. Output prediction

Like predicting next word:
"The weather in Miami is..."
→ RNN remembers context → predicts "sunny"

## Simple RNN in Python

```python
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense

model = Sequential([
    LSTM(50, return_sequences=True, input_shape=(timesteps, features)),
    LSTM(50),
    Dense(1)
])

model.compile(optimizer='adam', loss='mse')
```

## Types of RNNs

**1. Simple RNN**: Basic memory (rarely used now)

**2. LSTM** (Long Short-Term Memory):
- Better memory
- Avoids vanishing gradient
- Most popular

**3. GRU** (Gated Recurrent Unit):
- Simpler than LSTM
- Faster
- Similar performance

## Example - Stock Price Prediction

```python
# Prepare data: last 60 days → predict next day
X = []  # Past 60 days
y = []  # Next day price

for i in range(60, len(prices)):
    X.append(prices[i-60:i])
    y.append(prices[i])

# Build LSTM
model = Sequential([
    LSTM(50, return_sequences=True, input_shape=(60, 1)),
    LSTM(50),
    Dense(1)
])

model.fit(X, y, epochs=50)
```

## Applications

- Stock price prediction
- Language translation
- Text generation
- Speech recognition
- Music generation

## Challenges

- Slow to train
- Can't parallelize well
- Vanishing gradient problem

## Modern Alternative - Transformers

Newer architecture (used in ChatGPT):
- No recurrence
- Parallel training
- Better performance

## Remember

- RNNs for sequences
- LSTM most common
- Transformers are replacing RNNs

Recurrent Neural Networks

Process text with AI.

## Text Preprocessing

Clean text before using:

```python
import re
from nltk.corpus import stopwords
from nltk.stem import PorterStemmer

text = "The cats are running in Boston!"

# Lowercase
text = text.lower()

# Remove punctuation
text = re.sub(r'[^ws]', '', text)

# Remove stopwords
stop_words = set(stopwords.words('english'))
words = [w for w in text.split() if w not in stop_words]

# Stemming
stemmer = PorterStemmer()
words = [stemmer.stem(w) for w in words]

print(words)  # ['cat', 'run', 'boston']
```

## Tokenization

Split text into pieces:

```python
from nltk.tokenize import word_tokenize

text = "Hello from San Francisco!"
tokens = word_tokenize(text)
print(tokens)  # ['Hello', 'from', 'San', 'Francisco', '!']
```

## Bag of Words

Convert text to numbers:

```python
from sklearn.feature_extraction.text import CountVectorizer

texts = [
    "I love AI",
    "AI is amazing",
    "I love machine learning"
]

vectorizer = CountVectorizer()
X = vectorizer.fit_transform(texts)

print(vectorizer.get_feature_names_out())
print(X.toarray())
```

## TF-IDF

Better than bag of words:

```python
from sklearn.feature_extraction.text import TfidfVectorizer

vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(texts)

# Rare words get higher scores
# Common words get lower scores
```

## Sentiment Analysis

```python
from textblob import TextBlob

text = "The restaurant in Austin was amazing!"
blob = TextBlob(text)

sentiment = blob.sentiment.polarity
if sentiment > 0:
    print("Positive!")
elif sentiment < 0:
    print("Negative!")
else:
    print("Neutral")
```

## Named Entity Recognition

Find names, places, organizations:

```python
import spacy

nlp = spacy.load("en_core_web_sm")
text = "Apple Inc. is located in Cupertino, California."

doc = nlp(text)
for ent in doc.ents:
    print(f"{ent.text}: {ent.label_}")
# Apple Inc.: ORG
# Cupertino: GPE
# California: GPE
```

## Text Classification

```python
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline

pipeline = Pipeline([
    ('tfidf', TfidfVectorizer()),
    ('clf', MultinomialNB())
])

pipeline.fit(X_train, y_train)
predictions = pipeline.predict(X_test)
```

## Remember

- Always preprocess text
- TF-IDF better than bag of words
- Use pre-trained models when possible

Natural Language Processing Tasks

Words as numbers.

## What are Word Embeddings?

Representing words as vectors (lists of numbers).

Similar words have similar vectors!

## Why Better than One-Hot?

**One-Hot**: Each word is unique, no relationships
- cat = [1, 0, 0, 0]
- dog = [0, 1, 0, 0]
- king = [0, 0, 1, 0]

**Embeddings**: Capture relationships
- cat and dog are similar (both animals)
- king and queen are similar (both royalty)

## Word2Vec

Popular embedding method:

```python
from gensim.models import Word2Vec

sentences = [
    ['cat', 'sits', 'on', 'mat'],
    ['dog', 'plays', 'in', 'park'],
    ['cat', 'and', 'dog', 'play']
]

# Train Word2Vec
model = Word2Vec(sentences, vector_size=100, window=5, min_count=1)

# Get vector for word
vector = model.wv['cat']
print(vector)

# Find similar words
similar = model.wv.most_similar('cat', topn=3)
print(similar)  # [('dog', 0.87), ...]
```

## Cool Math with Embeddings

```python
# king - man + woman ≈ queen
result = model.wv.most_similar(
    positive=['king', 'woman'],
    negative=['man']
)
print(result[0])  # 'queen'
```

## Pre-trained Embeddings

Use embeddings trained on millions of documents:

**GloVe**: Stanford's embeddings
**FastText**: Facebook's embeddings
**BERT**: Google's contextual embeddings

```python
import gensim.downloader as api

# Load pre-trained GloVe
glove = api.load("glove-wiki-gigaword-100")

# Use immediately
similar = glove.most_similar('python')
print(similar)
```

## Using in Neural Networks

```python
from tensorflow.keras.layers import Embedding

model = Sequential([
    Embedding(input_dim=10000, output_dim=128),
    LSTM(64),
    Dense(1, activation='sigmoid')
])
```

## BERT - Contextual Embeddings

Unlike Word2Vec, BERT understands context:

"Bank" in "river bank" vs "money bank" gets different vectors!

```python
from transformers import BertTokenizer, BertModel

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased')

text = "The bank by the river"
inputs = tokenizer(text, return_tensors='pt')
outputs = model(**inputs)
```

## Applications

- Text classification
- Machine translation
- Sentiment analysis
- Question answering

## Remember

- Embeddings capture word meanings
- Pre-trained embeddings save time
- BERT understands context

Word Embeddings

Predict the future with data.

## What is Time Series?

Data collected over time: stock prices, temperature, sales.

**Goal**: Predict future values

## Key Concepts

**Trend**: Long-term direction (up/down)
**Seasonality**: Repeating patterns (summer/winter)
**Noise**: Random fluctuations

## Simple Moving Average

```python
import pandas as pd

# Calculate 7-day moving average
df['MA_7'] = df['sales'].rolling(window=7).mean()

# Predict: next value = average of last 7
```

## ARIMA Model

Classic time series model:

```python
from statsmodels.tsa.arima.model import ARIMA

# Fit ARIMA
model = ARIMA(data, order=(1, 1, 1))
model_fit = model.fit()

# Forecast next 10 days
forecast = model_fit.forecast(steps=10)
print(forecast)
```

## LSTM for Time Series

```python
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense
import numpy as np

# Prepare data: use last 30 days to predict next day
def create_sequences(data, seq_length=30):
    X, y = [], []
    for i in range(len(data) - seq_length):
        X.append(data[i:i+seq_length])
        y.append(data[i+seq_length])
    return np.array(X), np.array(y)

X, y = create_sequences(prices)

# Build model
model = Sequential([
    LSTM(50, return_sequences=True, input_shape=(30, 1)),
    LSTM(50),
    Dense(1)
])

model.compile(optimizer='adam', loss='mse')
model.fit(X, y, epochs=50, batch_size=32)

# Predict
next_day = model.predict(last_30_days)
```

## Prophet (by Facebook)

Easy and powerful:

```python
from fbprophet import Prophet

# Data needs 'ds' (date) and 'y' (value)
df = pd.DataFrame({
    'ds': dates,
    'y': values
})

model = Prophet()
model.fit(df)

# Forecast 365 days
future = model.make_future_dataframe(periods=365)
forecast = model.predict(future)

# Plot
model.plot(forecast)
```

## Evaluation Metrics

```python
from sklearn.metrics import mean_absolute_error, mean_squared_error

mae = mean_absolute_error(y_true, y_pred)
rmse = np.sqrt(mean_squared_error(y_true, y_pred))

print(f"MAE: {mae}")
print(f"RMSE: {rmse}")
```

## Real Applications

- Stock price prediction
- Sales forecasting
- Weather prediction
- Traffic forecasting
- Energy consumption

## Best Practices

1. Check for stationarity
2. Handle seasonality
3. Use walk-forward validation
4. Don't predict too far ahead

## Remember

- Time order matters!
- Don't shuffle data
- Use Prophet for easy start
- LSTM for complex patterns

Predict future values from time-based data.

Time Series Forecasting

Spot the unusual.

## What is Anomaly Detection?

Finding data points that don't fit the pattern.

**Examples**:
- Credit card fraud
- Manufacturing defects
- Network intrusion
- Equipment failure

## Statistical Method - Z-Score

```python
import numpy as np

def detect_anomalies(data, threshold=3):
    mean = np.mean(data)
    std = np.std(data)
    
    anomalies = []
    for i, value in enumerate(data):
        z_score = (value - mean) / std
        if abs(z_score) > threshold:
            anomalies.append(i)
    
    return anomalies

# Find anomalies
sales = [100, 105, 98, 102, 500, 97]  # 500 is anomaly
anomalies = detect_anomalies(sales)
print(f"Anomalies at: {anomalies}")  # [4]
```

## Isolation Forest

Popular ML method:

```python
from sklearn.ensemble import IsolationForest

# Train on normal data
model = IsolationForest(contamination=0.1)  # Expect 10% anomalies
model.fit(X_train)

# Predict: -1 = anomaly, 1 = normal
predictions = model.predict(X_test)

anomalies = X_test[predictions == -1]
print(f"Found {len(anomalies)} anomalies")
```

## One-Class SVM

```python
from sklearn.svm import OneClassSVM

# Train only on normal data
model = OneClassSVM(nu=0.1)  # nu = expected anomaly rate
model.fit(X_normal)

# Detect anomalies in new data
predictions = model.predict(X_test)
# -1 = anomaly, 1 = normal
```

## Autoencoder Approach

Neural network that reconstructs input:

```python
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, Dense

# Build autoencoder
input_layer = Input(shape=(features,))
encoded = Dense(32, activation='relu')(input_layer)
encoded = Dense(16, activation='relu')(encoded)
decoded = Dense(32, activation='relu')(encoded)
decoded = Dense(features, activation='linear')(decoded)

autoencoder = Model(input_layer, decoded)
autoencoder.compile(optimizer='adam', loss='mse')

# Train on normal data
autoencoder.fit(X_normal, X_normal, epochs=50)

# Anomalies = high reconstruction error
reconstructed = autoencoder.predict(X_test)
errors = np.mean(np.abs(X_test - reconstructed), axis=1)

threshold = np.percentile(errors, 95)
anomalies = errors > threshold
```

## Local Outlier Factor

Finds points far from neighbors:

```python
from sklearn.neighbors import LocalOutlierFactor

lof = LocalOutlierFactor(n_neighbors=20)
predictions = lof.fit_predict(X)

# -1 = anomaly, 1 = normal
```

## Real Example - Credit Card Fraud

```python
from sklearn.ensemble import IsolationForest

# Transaction features: amount, location, time, etc.
model = IsolationForest(contamination=0.01)  # 1% fraud
model.fit(transactions)

# Check new transaction
new_transaction = [[250, 2, 1545]]  # [amount, location_id, hour]
is_fraud = model.predict(new_transaction)

if is_fraud == -1:
    print("⚠️ Potential fraud detected!")
```

## Evaluation

Hard because anomalies are rare!

Use precision, recall, F1-score

## Remember

- Need mostly normal data for training
- Isolation Forest often best choice
- High false positives common
- Combine with business rules

Anomaly Detection

Suggest items users will love.

## What are Recommendation Systems?

Suggest products, movies, or content users might like.

**Examples**: Netflix shows, Amazon products, Spotify songs

## Types of Recommendations

**1. Content-Based**: Based on item features
"You liked Action movies, here are more Action movies"

**2. Collaborative Filtering**: Based on user behavior
"Users like you also liked these movies"

**3. Hybrid**: Combine both approaches

## Content-Based Filtering

```python
from sklearn.metrics.pairwise import cosine_similarity
import pandas as pd

# Movie features: [action, comedy, drama, romance]
movies = pd.DataFrame({
    'title': ['Movie A', 'Movie B', 'Movie C'],
    'action': [1, 0, 1],
    'comedy': [0, 1, 0],
    'drama': [0, 1, 1]
})

# User watched Movie A
user_liked = movies[movies['title'] == 'Movie A'].iloc[0, 1:]

# Find similar movies
similarities = cosine_similarity(
    [user_liked], 
    movies.iloc[:, 1:]
)[0]

# Recommend top 2 (excluding Movie A itself)
recommendations = movies.iloc[similarities.argsort()[-3:-1]]
print(recommendations['title'])
```

## Collaborative Filtering - User-Based

```python
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np

# User-item ratings matrix
ratings = np.array([
    [5, 3, 0, 1],  # User 1
    [4, 0, 0, 1],  # User 2
    [1, 1, 0, 5],  # User 3
    [1, 0, 0, 4],  # User 4
])

# Find similar users to User 1
user_similarity = cosine_similarity(ratings)

# Get User 1's similar users
similar_users = user_similarity[0]

# Predict ratings for unwatched items
# (Weighted average of similar users' ratings)
```

## Matrix Factorization

Advanced collaborative filtering:

```python
from scipy.sparse.linalg import svds

# Decompose ratings matrix
U, sigma, Vt = svds(ratings, k=2)

# Reconstruct with predictions
predicted_ratings = np.dot(np.dot(U, np.diag(sigma)), Vt)

# Recommend highest predicted ratings
```

## Using Surprise Library

```python
from surprise import SVD, Dataset, Reader
from surprise.model_selection import cross_validate

# Load data
reader = Reader(rating_scale=(1, 5))
data = Dataset.load_from_df(df[['user_id', 'item_id', 'rating']], reader)

# Train SVD
model = SVD()
cross_validate(model, data, measures=['RMSE', 'MAE'], cv=5)

# Get prediction
prediction = model.predict(user_id='Tom', item_id='Movie123')
print(f"Predicted rating: {prediction.est}")
```

## Deep Learning Approach

```python
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, Embedding, Flatten, Dense, Concatenate

# User and item inputs
user_input = Input(shape=(1,))
item_input = Input(shape=(1,))

# Embeddings
user_embedding = Embedding(n_users, 50)(user_input)
item_embedding = Embedding(n_items, 50)(item_input)

user_vec = Flatten()(user_embedding)
item_vec = Flatten()(item_embedding)

# Combine
concat = Concatenate()([user_vec, item_vec])
dense = Dense(128, activation='relu')(concat)
output = Dense(1)(dense)

model = Model([user_input, item_input], output)
model.compile(optimizer='adam', loss='mse')
```

## Cold Start Problem

**New User**: No history to base recommendations on
**Solution**: Ask preferences, use popular items

**New Item**: No ratings yet
**Solution**: Use content features, show to diverse users

## Evaluation

```python
from sklearn.metrics import mean_absolute_error

# Predict ratings
predictions = model.predict(X_test)

# Calculate error
mae = mean_absolute_error(y_test, predictions)
print(f"MAE: {mae}")
```

## Remember

- Collaborative filtering often best
- Handle cold start problem
- A/B test recommendations
- Balance accuracy with diversity

Build systems that recommend items to users.

Recommendation Systems

Make models available to users.

## Why Deploy?

Model in notebook ≠ Value
Model in production = Real impact!

## Save Your Model

```python
import joblib
from sklearn.ensemble import RandomForestClassifier

# Train model
model = RandomForestClassifier()
model.fit(X_train, y_train)

# Save model
joblib.dump(model, 'model.pkl')

# Load model
loaded_model = joblib.load('model.pkl')
prediction = loaded_model.predict(X_new)
```

## Flask API

Create a web API for your model:

```python
from flask import Flask, request, jsonify
import joblib

app = Flask(__name__)

# Load model at startup
model = joblib.load('model.pkl')

@app.route('/predict', methods=['POST'])
def predict():
    # Get data from request
    data = request.json
    features = [[data['age'], data['income']]]
    
    # Make prediction
    prediction = model.predict(features)[0]
    
    return jsonify({'prediction': int(prediction)})

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)
```

## Test API

```python
import requests

# Send request
response = requests.post('http://localhost:5000/predict', 
                        json={'age': 30, 'income': 50000})

result = response.json()
print(f"Prediction: {result['prediction']}")
```

## FastAPI (Modern Alternative)

```python
from fastapi import FastAPI
from pydantic import BaseModel
import joblib

app = FastAPI()
model = joblib.load('model.pkl')

class PredictionInput(BaseModel):
    age: int
    income: float

@app.post('/predict')
def predict(input_data: PredictionInput):
    features = [[input_data.age, input_data.income]]
    prediction = model.predict(features)[0]
    return {'prediction': int(prediction)}

# Run with: uvicorn main:app --reload
```

## Docker Container

```dockerfile
# Dockerfile
FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY model.pkl .
COPY app.py .

CMD ["python", "app.py"]
```

```bash
# Build and run
docker build -t ml-model .
docker run -p 5000:5000 ml-model
```

## Cloud Deployment

**AWS SageMaker**:
```python
import sagemaker
from sagemaker.sklearn import SKLearnModel

model = SKLearnModel(
    model_data='s3://bucket/model.tar.gz',
    role=role,
    entry_point='inference.py'
)

predictor = model.deploy(instance_type='ml.m5.large')
prediction = predictor.predict(data)
```

**Google Cloud AI Platform**:
```bash
gcloud ai-platform models create my_model
gcloud ai-platform versions create v1 \
    --model my_model \
    --origin gs://bucket/model \
    --runtime-version 2.3
```

## Monitoring

Track model performance:

```python
import logging

logging.basicConfig(level=logging.INFO)

@app.route('/predict', methods=['POST'])
def predict():
    data = request.json
    prediction = model.predict([data['features']])
    
    # Log for monitoring
    logging.info(f"Input: {data['features']}, Prediction: {prediction}")
    
    return jsonify({'prediction': int(prediction[0])})
```

## Best Practices

1. Version your models
2. Monitor predictions
3. Handle errors gracefully
4. Add authentication
5. Use caching for frequent queries
6. Set up CI/CD pipeline

## Remember

- Start with simple Flask API
- Use Docker for consistency
- Monitor model performance
- Plan for model updates

Model Deployment Basics

Explain AI decisions.

## Why Interpretability Matters

**Trust**: Users need to trust AI
**Debugging**: Find model errors
**Compliance**: Regulations require explanations
**Improvement**: Understand to improve

## Feature Importance

See which features matter most:

```python
from sklearn.ensemble import RandomForestClassifier
import pandas as pd

model = RandomForestClassifier()
model.fit(X_train, y_train)

# Get feature importance
importances = model.feature_importances_
features = pd.DataFrame({
    'feature': X_train.columns,
    'importance': importances
}).sort_values('importance', ascending=False)

print(features)
# age          0.45
# income       0.30
# location     0.15
# ...
```

## SHAP Values

Explain individual predictions:

```python
import shap

# Create explainer
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)

# Explain single prediction
shap.force_plot(
    explainer.expected_value[1],
    shap_values[1][0],
    X_test.iloc[0]
)

# Summary of all predictions
shap.summary_plot(shap_values[1], X_test)
```

## LIME

Local explanations:

```python
from lime.lime_tabular import LimeTabularExplainer

# Create explainer
explainer = LimeTabularExplainer(
    X_train.values,
    feature_names=X_train.columns,
    class_names=['Not Approved', 'Approved']
)

# Explain one prediction
i = 0
exp = explainer.explain_instance(
    X_test.iloc[i].values,
    model.predict_proba
)

exp.show_in_notebook()

# Feature contributions
exp.as_list()
# [('age > 30', 0.35), ('income > 50k', 0.28), ...]
```

## Partial Dependence Plots

Show feature effect:

```python
from sklearn.inspection import plot_partial_dependence

plot_partial_dependence(
    model,
    X_train,
    features=['age', 'income'],
    grid_resolution=50
)

# Shows: How does prediction change with age/income?
```

## Permutation Importance

```python
from sklearn.inspection import permutation_importance

result = permutation_importance(
    model, X_test, y_test,
    n_repeats=10,
    random_state=42
)

importance_df = pd.DataFrame({
    'feature': X_train.columns,
    'importance': result.importances_mean
}).sort_values('importance', ascending=False)

print(importance_df)
```

## For Neural Networks

```python
import tensorflow as tf

# Gradient-based explanations
with tf.GradientTape() as tape:
    inputs = tf.constant(X_test[:1])
    tape.watch(inputs)
    predictions = model(inputs)

# Get gradients
gradients = tape.gradient(predictions, inputs)

# Features with high gradient = important
```

## Model-Agnostic Methods

Work with any model:

**Anchors**: "If age > 30 AND income > 50k, then Approved"

**Counterfactuals**: "Change income from $40k to $55k → Approved"

## Real Example - Loan Approval

```python
# Train model
model.fit(X_train, y_train)

# Customer denied
customer = X_test.iloc[5]
prediction = model.predict([customer])[0]  # 0 = Denied

# Explain why
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values([customer])

# Output: 
# Age (-0.2): Too young
# Income (-0.3): Below threshold
# Credit score (+0.1): Good, but not enough
```

## Remember

- Always explain important decisions
- SHAP for tree models
- LIME for any model
- Combine multiple methods

Understand what your AI models are doing.

Model Interpretability

Test models properly.

## What is A/B Testing?

Compare two versions to see which performs better.

**Version A**: Current model (baseline)
**Version B**: New model (challenger)

## Why A/B Test AI?

Offline accuracy ≠ Real-world performance

**Reasons**:
- User behavior differs
- Data drift
- Business metrics matter more than accuracy

## Basic Setup

```python
import random

def get_model_for_user(user_id):
    # Randomly assign 50% to each model
    if random.random() < 0.5:
        return 'model_a'  # Current model
    else:
        return 'model_b'  # New model

# Track results
results = {
    'model_a': {'shown': 0, 'clicked': 0},
    'model_b': {'shown': 0, 'clicked': 0}
}
```

## Real Example - Recommendation System

```python
from flask import Flask, request

app = Flask(__name__)

@app.route('/recommend')
def recommend():
    user_id = request.args.get('user_id')
    
    # Assign model
    model_version = assign_model(user_id)
    
    if model_version == 'A':
        recommendations = model_a.predict(user_id)
    else:
        recommendations = model_b.predict(user_id)
    
    # Log for analysis
    log_experiment(user_id, model_version, recommendations)
    
    return {'recommendations': recommendations}
```

## Track Metrics

```python
import pandas as pd

# Collect results
results_df = pd.DataFrame({
    'user_id': [1, 2, 3, 4, 5, 6],
    'model': ['A', 'B', 'A', 'B', 'A', 'B'],
    'clicked': [1, 1, 0, 1, 1, 1]
})

# Calculate metrics
by_model = results_df.groupby('model')['clicked'].agg(['sum', 'count', 'mean'])

print(by_model)
# model  sum  count  mean
# A      2    3      0.67
# B      3    3      1.00
```

## Statistical Significance

```python
from scipy import stats

# Get data for both groups
a_clicks = results_df[results_df['model'] == 'A']['clicked']
b_clicks = results_df[results_df['model'] == 'B']['clicked']

# T-test
t_stat, p_value = stats.ttest_ind(a_clicks, b_clicks)

print(f"P-value: {p_value}")

if p_value < 0.05:
    print("✅ Result is statistically significant!")
else:
    print("❌ Not enough evidence, need more data")
```

## Sample Size Calculation

How many users needed?

```python
from statsmodels.stats.power import zt_ind_solve_power

# Calculate required sample size
n = zt_ind_solve_power(
    effect_size=0.2,  # Expected improvement
    alpha=0.05,        # Significance level
    power=0.8          # Statistical power
)

print(f"Need {int(n)} users per group")
```

## Multi-Armed Bandit

Smart A/B test that learns:

```python
import numpy as np

class EpsilonGreedy:
    def __init__(self, epsilon=0.1):
        self.epsilon = epsilon
        self.counts = {'A': 0, 'B': 0}
        self.values = {'A': 0, 'B': 0}
    
    def select(self):
        # Sometimes explore (random)
        if random.random() < self.epsilon:
            return random.choice(['A', 'B'])
        
        # Usually exploit (best so far)
        return 'A' if self.values['A'] > self.values['B'] else 'B'
    
    def update(self, model, reward):
        self.counts[model] += 1
        n = self.counts[model]
        
        # Update average reward
        self.values[model] = ((n - 1) / n) * self.values[model] + reward / n

# Use it
bandit = EpsilonGreedy()

for user in users:
    model = bandit.select()
    reward = get_reward(user, model)
    bandit.update(model, reward)
```

## Monitoring Dashboard

```python
import matplotlib.pyplot as plt

def plot_results(results_df):
    # Click-through rate over time
    results_df['date'] = pd.to_datetime(results_df['timestamp'])
    
    for model in ['A', 'B']:
        data = results_df[results_df['model'] == model]
        ctr = data.groupby('date')['clicked'].mean()
        plt.plot(ctr, label=f'Model {model}')
    
    plt.legend()
    plt.title('Click-Through Rate Over Time')
    plt.xlabel('Date')
    plt.ylabel('CTR')
    plt.show()
```

## Best Practices

1. Run for at least 1-2 weeks
2. Need 1000+ users minimum
3. Check multiple metrics
4. Watch for novelty effect
5. Document everything

## Remember

- Offline metrics ≠ online performance
- Need statistical significance
- Monitor continuously
- Consider multi-armed bandits

Test AI models in production scientifically.

A/B Testing for AI

Create more training data.

## What is Data Augmentation?

Creating new training examples from existing ones.

**Goal**: More data → Better model

## Image Augmentation

```python
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Define augmentations
datagen = ImageDataGenerator(
    rotation_range=20,      # Rotate up to 20 degrees
    width_shift_range=0.2,  # Shift left/right
    height_shift_range=0.2, # Shift up/down
    horizontal_flip=True,   # Mirror image
    zoom_range=0.2,         # Zoom in/out
    fill_mode='nearest'
)

# Apply to images
augmented_images = datagen.flow(images, batch_size=32)

# Use in training
model.fit(augmented_images, epochs=50)
```

## Albumentations (Better for Images)

```python
import albumentations as A
import cv2

# Define pipeline
transform = A.Compose([
    A.RandomRotate90(),
    A.Flip(),
    A.Transpose(),
    A.GaussNoise(),
    A.OneOf([
        A.MotionBlur(p=0.2),
        A.MedianBlur(blur_limit=3, p=0.1),
        A.Blur(blur_limit=3, p=0.1),
    ], p=0.2),
    A.RandomBrightnessContrast(p=0.2),
])

# Apply
image = cv2.imread('photo.jpg')
augmented = transform(image=image)['image']
```

## Text Augmentation

```python
import nlpaug.augmenter.word as naw

# Synonym replacement
aug = naw.SynonymAug(aug_src='wordnet')
text = "The food in Miami is amazing"
augmented = aug.augment(text)
print(augmented)
# "The food in Miami is fantastic"

# Back translation (more realistic)
aug = naw.BackTranslationAug(
    from_model_name='facebook/wmt19-en-de',
    to_model_name='facebook/wmt19-de-en'
)
augmented = aug.augment(text)
```

## Audio Augmentation

```python
import librosa
import numpy as np

def augment_audio(audio, sr):
    # Time stretch
    audio_stretched = librosa.effects.time_stretch(audio, rate=1.1)
    
    # Pitch shift
    audio_pitched = librosa.effects.pitch_shift(audio, sr=sr, n_steps=2)
    
    # Add noise
    noise = np.random.randn(len(audio))
    audio_noisy = audio + 0.005 * noise
    
    return [audio_stretched, audio_pitched, audio_noisy]
```

## Tabular Data Augmentation

```python
from sklearn.utils import resample
import pandas as pd

# Oversample minority class
df_minority = df[df['target'] == 1]
df_majority = df[df['target'] == 0]

# Resample with replacement
df_minority_upsampled = resample(
    df_minority,
    replace=True,
    n_samples=len(df_majority),
    random_state=42
)

# Combine
df_balanced = pd.concat([df_majority, df_minority_upsampled])
```

## SMOTE for Tabular Data

```python
from imblearn.over_sampling import SMOTE

# Create synthetic samples
smote = SMOTE()
X_resampled, y_resampled = smote.fit_resample(X, y)

print(f"Original: {len(X)}")
print(f"After SMOTE: {len(X_resampled)}")
```

## MixUp (Advanced)

Mix two images:

```python
def mixup(x1, y1, x2, y2, alpha=0.2):
    # Random mixing ratio
    lam = np.random.beta(alpha, alpha)
    
    # Mix images
    x = lam * x1 + (1 - lam) * x2
    
    # Mix labels
    y = lam * y1 + (1 - lam) * y2
    
    return x, y

# Use in training
x_mix, y_mix = mixup(image1, label1, image2, label2)
```

## CutMix

Cut part of image and paste another:

```python
def cutmix(x1, y1, x2, y2, alpha=1.0):
    lam = np.random.beta(alpha, alpha)
    
    # Random box
    h, w = x1.shape[:2]
    cut_h = int(h * np.sqrt(1 - lam))
    cut_w = int(w * np.sqrt(1 - lam))
    
    cx = np.random.randint(w)
    cy = np.random.randint(h)
    
    x1y1 = np.clip([cx - cut_w // 2, cy - cut_h // 2], 0, [w, h])
    x2y2 = np.clip([cx + cut_w // 2, cy + cut_h // 2], 0, [w, h])
    
    # Replace region
    x1[x1y1[1]:x2y2[1], x1y1[0]:x2y2[0]] = x2[x1y1[1]:x2y2[1], x1y1[0]:x2y2[0]]
    
    return x1, y1
```

## Best Practices

1. **Images**: Always augment (flip, rotate, crop)
2. **Text**: Be careful (meaning shouldn't change)
3. **Validation**: Never augment validation data!
4. **Realistic**: Augmentations should be realistic

## Remember

- More data = better model
- Augment during training
- Don't overdo it
- Keep validation data clean

Data Augmentation

Production ML best practices.

## What is MLOps?

DevOps for Machine Learning.

**Goal**: Reliably deploy and maintain ML systems

## Key Components

1. **Version Control**: Code, data, models
2. **CI/CD**: Automated testing and deployment
3. **Monitoring**: Track model performance
4. **Reproducibility**: Same code = same results

## Version Control

```bash
# Git for code
git add model.py data_processing.py
git commit -m "Update model architecture"

# DVC for data and models
dvc add data/training_data.csv
dvc add models/model.pkl
git add data/training_data.csv.dvc models/model.pkl.dvc
git commit -m "Update data and model"
```

## Experiment Tracking

```python
import mlflow

# Start experiment
mlflow.set_experiment("house_price_prediction")

with mlflow.start_run():
    # Log parameters
    mlflow.log_param("n_estimators", 100)
    mlflow.log_param("max_depth", 10)
    
    # Train model
    model.fit(X_train, y_train)
    
    # Log metrics
    accuracy = model.score(X_test, y_test)
    mlflow.log_metric("accuracy", accuracy)
    
    # Log model
    mlflow.sklearn.log_model(model, "model")
```

## Automated Training Pipeline

```python
# training_pipeline.py
def training_pipeline():
    # 1. Load data
    data = load_data('s3://bucket/data.csv')
    
    # 2. Preprocess
    X, y = preprocess(data)
    X_train, X_test, y_train, y_test = train_test_split(X, y)
    
    # 3. Train
    model = train_model(X_train, y_train)
    
    # 4. Evaluate
    metrics = evaluate_model(model, X_test, y_test)
    
    # 5. Save if better
    if metrics['accuracy'] > current_best:
        save_model(model, 'production_model.pkl')
        
    return metrics

# Run automatically
if __name__ == "__main__":
    results = training_pipeline()
    print(f"New model accuracy: {results['accuracy']}")
```

## CI/CD for ML

```.github/workflows/ml_pipeline.yml
name: ML Pipeline

on:
  push:
    branches: [main]

jobs:
  train:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: 3.9
      
      - name: Install dependencies
        run: pip install -r requirements.txt
      
      - name: Run tests
        run: pytest tests/
      
      - name: Train model
        run: python train.py
      
      - name: Validate model
        run: python validate_model.py
      
      - name: Deploy if tests pass
        run: python deploy.py
```

## Monitoring

```python
import prometheus_client
from flask import Flask

app = Flask(__name__)

# Metrics
predictions_counter = prometheus_client.Counter(
    'predictions_total',
    'Total predictions made'
)

prediction_time = prometheus_client.Histogram(
    'prediction_duration_seconds',
    'Time spent making prediction'
)

@app.route('/predict', methods=['POST'])
@prediction_time.time()
def predict():
    data = request.json
    
    # Make prediction
    prediction = model.predict([data['features']])
    
    # Track metrics
    predictions_counter.inc()
    
    return jsonify({'prediction': int(prediction[0])})
```

## Data Drift Detection

```python
from evidently.dashboard import Dashboard
from evidently.tabs import DataDriftTab

# Compare training vs production data
dashboard = Dashboard(tabs=[DataDriftTab()])
dashboard.calculate(reference_data, production_data)

# Alert if drift detected
if dashboard.metrics['data_drift']['share_of_drifted_features'] > 0.3:
    send_alert("Data drift detected!")
```

## Model Registry

```python
import mlflow

# Register model
model_uri = "runs:/abc123/model"
mlflow.register_model(model_uri, "house_price_model")

# Transition to production
client = mlflow.tracking.MlflowClient()
client.transition_model_version_stage(
    name="house_price_model",
    version=3,
    stage="Production"
)

# Load production model
model = mlflow.pyfunc.load_model("models:/house_price_model/Production")
```

## Feature Store

```python
from feast import FeatureStore

store = FeatureStore(repo_path=".")

# Get features
features = store.get_online_features(
    features=[
        'user_features:age',
        'user_features:income',
        'transaction_features:avg_amount'
    ],
    entity_rows=[{"user_id": 123}]
).to_dict()

# Use in prediction
prediction = model.predict([list(features.values())])
```

## Best Practices

1. **Version everything**: code, data, models
2. **Automate training**: CI/CD pipelines
3. **Monitor constantly**: accuracy, latency, drift
4. **Test thoroughly**: unit tests, integration tests
5. **Document**: model cards, data sheets

## Tools Ecosystem

- **Experiment tracking**: MLflow, Weights & Biases
- **Pipelines**: Kubeflow, Airflow
- **Monitoring**: Prometheus, Grafana
- **Serving**: TensorFlow Serving, Seldon

## Remember

- MLOps = DevOps + ML
- Automate everything
- Monitor in production
- Version all artifacts

MLOps Basics

Reuse powerful models.

## What is Transfer Learning?

Using models trained on huge datasets for your specific task.

Like learning piano after learning keyboard - transfer skills!

## Why Transfer Learning?

**Problems it solves**:
- Don't have millions of images
- Can't afford weeks of training
- Limited GPU resources

## Pre-trained Models

**Vision**:
- VGG16, ResNet50, InceptionV3
- Trained on ImageNet (1.4M images, 1000 classes)

**Language**:
- BERT, GPT, RoBERTa
- Trained on billions of words

## Using Pre-trained Model

```python
from tensorflow.keras.applications import VGG16
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D

# Load pre-trained model (without top layer)
base_model = VGG16(weights='imagenet', include_top=False)

# Freeze base layers
for layer in base_model.layers:
    layer.trainable = False

# Add custom layers
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(256, activation='relu')(x)
predictions = Dense(10, activation='softmax')(x)  # 10 classes

model = Model(inputs=base_model.input, outputs=predictions)
```

## Fine-Tuning Strategy

**Step 1**: Train only top layers
```python
# Freeze all base layers
for layer in base_model.layers:
    layer.trainable = False

model.compile(optimizer='adam', loss='categorical_crossentropy')
model.fit(X_train, y_train, epochs=5)
```

**Step 2**: Unfreeze some layers
```python
# Unfreeze last 10 layers
for layer in base_model.layers[-10:]:
    layer.trainable = True

# Use lower learning rate
model.compile(optimizer=Adam(lr=0.0001), loss='categorical_crossentropy')
model.fit(X_train, y_train, epochs=10)
```

## Real Example - Medical Images

```python
# Classify X-rays with only 1000 images
from tensorflow.keras.applications import ResNet50

base = ResNet50(weights='imagenet', include_top=False, input_shape=(224, 224, 3))

# Freeze most layers
for layer in base.layers[:-20]:
    layer.trainable = False

# Custom classifier
x = GlobalAveragePooling2D()(base.output)
x = Dense(128, activation='relu')(x)
x = Dropout(0.5)(x)
output = Dense(2, activation='softmax')(x)  # Normal vs Pneumonia

model = Model(base.input, output)
model.compile(optimizer=Adam(1e-4), loss='categorical_crossentropy', metrics=['accuracy'])

# Train on small dataset
model.fit(X_train, y_train, epochs=20, validation_data=(X_val, y_val))
```

## Transfer Learning for NLP

```python
from transformers import BertTokenizer, TFBertForSequenceClassification
import tensorflow as tf

# Load pre-trained BERT
model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

# Tokenize text
texts = ["Great product!", "Terrible service"]
encodings = tokenizer(texts, truncation=True, padding=True, return_tensors='tf')

# Fine-tune on your data
model.compile(
    optimizer=tf.keras.optimizers.Adam(1e-5),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics=['accuracy']
)

model.fit(encodings['input_ids'], labels, epochs=3)
```

## Feature Extraction vs Fine-Tuning

**Feature Extraction**:
- Freeze all layers
- Use as feature extractor
- Fast, less data needed

**Fine-Tuning**:
- Unfreeze some layers
- Adapt to your data
- Better performance, needs more data

## Domain Adaptation

When source and target differ:

```python
# Trained on photos, using on sketches
# Use domain adaptation techniques

from tensorflow.keras.layers import Lambda

# Gradient reversal for domain adaptation
def gradient_reversal(x):
    return x * -1

# Add domain classifier
domain_output = Lambda(gradient_reversal)(shared_features)
domain_output = Dense(1, activation='sigmoid')(domain_output)
```

## Best Practices

1. Start with frozen layers
2. Use small learning rate when fine-tuning
3. Augment data heavily
4. Monitor validation loss closely
5. Try different pre-trained models

## Remember

- Transfer learning saves time and resources
- Always start with pre-trained models
- Fine-tune carefully with low learning rate
- Works amazingly well with small datasets

Transfer Learning

AI that creates.

## What are GANs?

Two neural networks competing:

**Generator**: Creates fake data
**Discriminator**: Detects fake vs real

Like counterfeiter vs detective!

## How GANs Work

1. Generator creates fake image
2. Discriminator tries to identify it
3. Both improve through competition
4. Eventually, fakes look real!

## Simple GAN

```python
from tensorflow.keras.layers import Dense, Reshape, Flatten
from tensorflow.keras.models import Sequential
import numpy as np

# Generator: Noise → Image
generator = Sequential([
    Dense(128, activation='relu', input_shape=(100,)),
    Dense(256, activation='relu'),
    Dense(784, activation='tanh'),
    Reshape((28, 28, 1))
])

# Discriminator: Image → Real/Fake
discriminator = Sequential([
    Flatten(input_shape=(28, 28, 1)),
    Dense(256, activation='relu'),
    Dense(128, activation='relu'),
    Dense(1, activation='sigmoid')  # 0=Fake, 1=Real
])

discriminator.compile(optimizer='adam', loss='binary_crossentropy')
```

## Training Loop

```python
def train_gan(generator, discriminator, epochs=10000):
    for epoch in range(epochs):
        # 1. Train Discriminator
        
        # Real images
        real_images = X_train[np.random.randint(0, X_train.shape[0], 32)]
        real_labels = np.ones((32, 1))
        
        # Fake images
        noise = np.random.normal(0, 1, (32, 100))
        fake_images = generator.predict(noise)
        fake_labels = np.zeros((32, 1))
        
        # Train discriminator
        d_loss_real = discriminator.train_on_batch(real_images, real_labels)
        d_loss_fake = discriminator.train_on_batch(fake_images, fake_labels)
        
        # 2. Train Generator
        
        noise = np.random.normal(0, 1, (32, 100))
        misleading_labels = np.ones((32, 1))  # Want discriminator to think they're real
        
        # Freeze discriminator
        discriminator.trainable = False
        g_loss = gan.train_on_batch(noise, misleading_labels)
        discriminator.trainable = True
        
        if epoch % 1000 == 0:
            print(f"Epoch {epoch}, D Loss: {d_loss_real}, G Loss: {g_loss}")
```

## DCGAN - Deep Convolutional GAN

Better for images:

```python
from tensorflow.keras.layers import Conv2D, Conv2DTranspose, BatchNormalization, LeakyReLU

# Generator with Conv layers
generator = Sequential([
    Dense(7*7*256, input_shape=(100,)),
    Reshape((7, 7, 256)),
    
    Conv2DTranspose(128, 5, strides=1, padding='same'),
    BatchNormalization(),
    LeakyReLU(),
    
    Conv2DTranspose(64, 5, strides=2, padding='same'),
    BatchNormalization(),
    LeakyReLU(),
    
    Conv2DTranspose(1, 5, strides=2, padding='same', activation='tanh')
])

# Discriminator
discriminator = Sequential([
    Conv2D(64, 5, strides=2, padding='same', input_shape=(28, 28, 1)),
    LeakyReLU(),
    
    Conv2D(128, 5, strides=2, padding='same'),
    LeakyReLU(),
    
    Flatten(),
    Dense(1, activation='sigmoid')
])
```

## Conditional GAN

Generate specific types:

```python
# Input: noise + label (e.g., which digit to generate)
noise_input = Input(shape=(100,))
label_input = Input(shape=(10,))  # One-hot encoded

# Combine inputs
combined = Concatenate()([noise_input, label_input])

# Generator
x = Dense(128, activation='relu')(combined)
output = Dense(784, activation='tanh')(x)

cgan_generator = Model([noise_input, label_input], output)

# Now can generate specific digits!
noise = np.random.normal(0, 1, (1, 100))
label = np.array([[0, 0, 1, 0, 0, 0, 0, 0, 0, 0]])  # Generate "2"
generated_digit = cgan_generator.predict([noise, label])
```

## Applications

- **Image Generation**: Create realistic faces
- **Image-to-Image**: Photos → Sketches, Day → Night
- **Super Resolution**: Low res → High res
- **Data Augmentation**: Create training data
- **Art Generation**: Novel artwork

## Famous GANs

**StyleGAN**: Photorealistic faces
**Pix2Pix**: Image-to-image translation
**CycleGAN**: Unpaired image translation
**BigGAN**: High-resolution images

## Challenges

- **Mode Collapse**: Generates same thing repeatedly
- **Training Instability**: Hard to balance G and D
- **Evaluation**: Hard to measure quality

## Tips for Stable Training

1. Use LeakyReLU instead of ReLU
2. Batch normalization helps
3. Use Adam optimizer
4. Train discriminator more frequently
5. Add noise to real images

## Remember

- GANs are powerful but tricky to train
- Start with DCGAN architecture
- Monitor both losses
- Expect instability

Generative Adversarial Networks

Architecture that changed AI.

## What are Transformers?

Revolutionary architecture that replaced RNNs.

**Key Innovation**: Self-attention mechanism

**Powers**: ChatGPT, BERT, GPT-4, Gemini

## Why Better Than RNNs?

**RNNs**:
- Process sequentially (slow)
- Forget long-range dependencies
- Can't parallelize

**Transformers**:
- Process entire sequence at once (fast)
- Capture long-range dependencies
- Fully parallelizable

## Self-Attention Explained

**Goal**: Each word looks at all other words

Example: "The cat sat on the mat"
- "sat" pays attention to "cat" (who sat?)
- "sat" pays attention to "mat" (where?)

## Attention Mechanism

```python
import numpy as np

def scaled_dot_product_attention(Q, K, V):
    """
    Q: Query matrix
    K: Key matrix
    V: Value matrix
    """
    # Calculate attention scores
    d_k = Q.shape[-1]
    scores = np.matmul(Q, K.transpose()) / np.sqrt(d_k)
    
    # Softmax to get weights
    weights = softmax(scores)
    
    # Weighted sum of values
    output = np.matmul(weights, V)
    return output

# Example
sentence = "The cat sat"
# Each word becomes Q, K, V vectors
attention_output = scaled_dot_product_attention(Q, K, V)
```

## Multi-Head Attention

Multiple attention mechanisms in parallel:

```python
from tensorflow.keras.layers import Layer

class MultiHeadAttention(Layer):
    def __init__(self, d_model, num_heads):
        super().__init__()
        self.num_heads = num_heads
        self.d_model = d_model
        
        self.depth = d_model // num_heads
        
        self.wq = Dense(d_model)
        self.wk = Dense(d_model)
        self.wv = Dense(d_model)
        
        self.dense = Dense(d_model)
    
    def split_heads(self, x, batch_size):
        x = tf.reshape(x, (batch_size, -1, self.num_heads, self.depth))
        return tf.transpose(x, perm=[0, 2, 1, 3])
    
    def call(self, q, k, v):
        batch_size = tf.shape(q)[0]
        
        # Linear projections
        q = self.wq(q)
        k = self.wk(k)
        v = self.wv(v)
        
        # Split into multiple heads
        q = self.split_heads(q, batch_size)
        k = self.split_heads(k, batch_size)
        v = self.split_heads(v, batch_size)
        
        # Scaled dot-product attention
        attention = scaled_dot_product_attention(q, k, v)
        
        # Concatenate heads
        attention = tf.transpose(attention, perm=[0, 2, 1, 3])
        concat_attention = tf.reshape(attention, (batch_size, -1, self.d_model))
        
        # Final linear projection
        output = self.dense(concat_attention)
        return output
```

## Positional Encoding

Since Transformers process all words at once, need to encode position:

```python
def positional_encoding(max_len, d_model):
    pos = np.arange(max_len)[:, np.newaxis]
    i = np.arange(d_model)[np.newaxis, :]
    
    angles = pos / np.power(10000, (2 * (i // 2)) / d_model)
    
    # Even indices: sin
    angles[:, 0::2] = np.sin(angles[:, 0::2])
    # Odd indices: cos
    angles[:, 1::2] = np.cos(angles[:, 1::2])
    
    return angles

# Add to input embeddings
embeddings = word_embeddings + positional_encoding
```

## Full Transformer Block

```python
class TransformerBlock(Layer):
    def __init__(self, d_model, num_heads, ff_dim):
        super().__init__()
        
        self.attention = MultiHeadAttention(d_model, num_heads)
        self.ffn = Sequential([
            Dense(ff_dim, activation='relu'),
            Dense(d_model)
        ])
        
        self.layernorm1 = LayerNormalization()
        self.layernorm2 = LayerNormalization()
        self.dropout1 = Dropout(0.1)
        self.dropout2 = Dropout(0.1)
    
    def call(self, x, training):
        # Multi-head attention
        attn_output = self.attention(x, x, x)
        attn_output = self.dropout1(attn_output, training=training)
        out1 = self.layernorm1(x + attn_output)  # Residual connection
        
        # Feed-forward network
        ffn_output = self.ffn(out1)
        ffn_output = self.dropout2(ffn_output, training=training)
        out2 = self.layernorm2(out1 + ffn_output)  # Residual connection
        
        return out2
```

## Encoder-Decoder Architecture

**Encoder**: Processes input sequence
**Decoder**: Generates output sequence

Used for translation, summarization, etc.

## Using Pre-trained Transformers

```python
from transformers import pipeline

# Text classification
classifier = pipeline('sentiment-analysis')
result = classifier("The service in Austin was amazing!")
print(result)  # [{'label': 'POSITIVE', 'score': 0.9998}]

# Text generation
generator = pipeline('text-generation', model='gpt2')
text = generator("In San Francisco", max_length=50)
print(text)

# Question answering
qa = pipeline('question-answering')
context = "The Golden Gate Bridge is in San Francisco."
question = "Where is the Golden Gate Bridge?"
answer = qa(question=question, context=context)
print(answer)
```

## Key Models

**BERT**: Bidirectional Encoder (understanding)
**GPT**: Decoder only (generation)
**T5**: Encoder-Decoder (versatile)
**BART**: Encoder-Decoder (generation + understanding)

## Applications

- Machine translation
- Text summarization
- Question answering
- Text generation
- Sentiment analysis
- Named entity recognition

## Remember

- Self-attention is the key innovation
- Positional encoding captures order
- Pre-trained models are very powerful
- Transformers revolutionized NLP

Understand the architecture behind ChatGPT.

Transformer Architecture

AI that learns by doing.

## What is Reinforcement Learning?

Agent learns by interacting with environment.

**Key Idea**: Actions → Rewards/Penalties → Learning

Like training a dog with treats!

## Key Concepts

**Agent**: The learner (AI)
**Environment**: The world
**State**: Current situation
**Action**: What agent can do
**Reward**: Feedback from environment

## Example - Robot Navigation

**State**: Robot's position in room
**Actions**: Move forward, turn left, turn right
**Reward**: +10 for reaching goal, -1 for hitting wall

## Q-Learning

Simple RL algorithm:

```python
import numpy as np

# Q-table: State x Action → Expected reward
Q = np.zeros((num_states, num_actions))

# Hyperparameters
learning_rate = 0.1
discount = 0.95
epsilon = 0.1  # Exploration rate

for episode in range(1000):
    state = env.reset()
    done = False
    
    while not done:
        # Choose action (epsilon-greedy)
        if np.random.random() < epsilon:
            action = env.action_space.sample()  # Explore
        else:
            action = np.argmax(Q[state])  # Exploit
        
        # Take action
        next_state, reward, done, _ = env.step(action)
        
        # Update Q-value
        old_q = Q[state, action]
        next_max = np.max(Q[next_state])
        new_q = old_q + learning_rate * (reward + discount * next_max - old_q)
        Q[state, action] = new_q
        
        state = next_state

# Use learned policy
state = env.reset()
while not done:
    action = np.argmax(Q[state])
    state, reward, done, _ = env.step(action)
```

## OpenAI Gym

Standard RL environment:

```python
import gym

# Create environment
env = gym.make('CartPole-v1')

# Reset environment
state = env.reset()

for _ in range(1000):
    env.render()
    
    # Take random action
    action = env.action_space.sample()
    
    # Get result
    next_state, reward, done, info = env.step(action)
    
    if done:
        break

env.close()
```

## Deep Q-Network (DQN)

Q-Learning with neural network:

```python
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from collections import deque
import random

class DQNAgent:
    def __init__(self, state_size, action_size):
        self.state_size = state_size
        self.action_size = action_size
        
        self.memory = deque(maxlen=2000)
        self.gamma = 0.95  # Discount rate
        self.epsilon = 1.0  # Exploration rate
        self.epsilon_decay = 0.995
        self.epsilon_min = 0.01
        
        self.model = self._build_model()
    
    def _build_model(self):
        model = Sequential()
        model.add(Dense(24, input_dim=self.state_size, activation='relu'))
        model.add(Dense(24, activation='relu'))
        model.add(Dense(self.action_size, activation='linear'))
        model.compile(loss='mse', optimizer=Adam(lr=0.001))
        return model
    
    def remember(self, state, action, reward, next_state, done):
        self.memory.append((state, action, reward, next_state, done))
    
    def act(self, state):
        if np.random.random() <= self.epsilon:
            return random.randrange(self.action_size)
        q_values = self.model.predict(state)
        return np.argmax(q_values[0])
    
    def replay(self, batch_size):
        minibatch = random.sample(self.memory, batch_size)
        
        for state, action, reward, next_state, done in minibatch:
            target = reward
            if not done:
                target += self.gamma * np.amax(self.model.predict(next_state)[0])
            
            target_f = self.model.predict(state)
            target_f[0][action] = target
            
            self.model.fit(state, target_f, epochs=1, verbose=0)
        
        if self.epsilon > self.epsilon_min:
            self.epsilon *= self.epsilon_decay

# Train DQN
agent = DQNAgent(state_size=4, action_size=2)

for episode in range(1000):
    state = env.reset()
    state = np.reshape(state, [1, 4])
    
    for time in range(500):
        action = agent.act(state)
        next_state, reward, done, _ = env.step(action)
        next_state = np.reshape(next_state, [1, 4])
        
        agent.remember(state, action, reward, next_state, done)
        state = next_state
        
        if done:
            print(f"Episode: {episode}, Score: {time}")
            break
    
    if len(agent.memory) > 32:
        agent.replay(32)
```

## Policy Gradient

Learn policy directly:

```python
# Instead of Q-values, output action probabilities
model = Sequential([
    Dense(24, input_dim=state_size, activation='relu'),
    Dense(24, activation='relu'),
    Dense(action_size, activation='softmax')  # Probabilities
])

# Sample action from probability distribution
probs = model.predict(state)[0]
action = np.random.choice(action_size, p=probs)
```

## Applications

- Game playing (AlphaGo, Atari)
- Robotics
- Self-driving cars
- Resource management
- Trading algorithms
- Recommendation systems

## Challenges

- Sparse rewards
- Exploration vs exploitation
- Sample inefficiency
- Stability

## Remember

- RL learns through trial and error
- Q-Learning for discrete actions
- DQN for complex environments
- Requires lots of episodes

Reinforcement Learning Basics

AI that builds AI.

## What is AutoML?

Automatically find best model and hyperparameters.

**Goal**: Make ML accessible to everyone

## Why AutoML?

- Save time on experimentation
- Find models you wouldn't try manually
- Optimize better than humans
- Democratize ML

## Auto-Sklearn

Automated sklearn pipeline:

```python
from autosklearn.classification import AutoSklearnClassifier

# Create AutoML model
automl = AutoSklearnClassifier(
    time_left_for_this_task=3600,  # 1 hour
    per_run_time_limit=300,         # 5 min per model
)

# Fit - tries many models automatically
automl.fit(X_train, y_train)

# Get best model
print(automl.show_models())

# Predict
predictions = automl.predict(X_test)

# See what worked best
print(automl.leaderboard())
```

## TPOT - Genetic Programming

Evolves ML pipelines:

```python
from tpot import TPOTClassifier

# Genetic algorithm to find best pipeline
tpot = TPOTClassifier(
    generations=5,
    population_size=50,
    verbosity=2,
    random_state=42
)

tpot.fit(X_train, y_train)

# Get accuracy
print(tpot.score(X_test, y_test))

# Export best pipeline as Python code
tpot.export('best_pipeline.py')
```

Generated pipeline might look like:

```python
# Auto-generated by TPOT
from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('classifier', RandomForestClassifier(n_estimators=100, max_depth=10))
])
```

## H2O AutoML

Enterprise-grade AutoML:

```python
import h2o
from h2o.automl import H2OAutoML

h2o.init()

# Load data
train = h2o.import_file("train.csv")

# Specify target and features
y = "target"
X = train.columns
X.remove(y)

# Run AutoML
aml = H2OAutoML(max_models=20, max_runtime_secs=3600)
aml.train(x=X, y=y, training_frame=train)

# View leaderboard
lb = aml.leaderboard
print(lb.head())

# Best model
best_model = aml.leader
predictions = best_model.predict(test)
```

## Neural Architecture Search (NAS)

Find best neural network architecture:

```python
import keras_tuner as kt

def build_model(hp):
    model = Sequential()
    
    # Tune number of layers
    for i in range(hp.Int('num_layers', 1, 5)):
        model.add(Dense(
            units=hp.Int(f'units_{i}', 32, 512, step=32),
            activation=hp.Choice('activation', ['relu', 'tanh'])
        ))
    
    model.add(Dense(10, activation='softmax'))
    
    # Tune learning rate
    model.compile(
        optimizer=Adam(hp.Float('learning_rate', 1e-4, 1e-2, sampling='log')),
        loss='categorical_crossentropy',
        metrics=['accuracy']
    )
    
    return model

# Search for best architecture
tuner = kt.RandomSearch(
    build_model,
    objective='val_accuracy',
    max_trials=50,
    directory='nas_results'
)

tuner.search(X_train, y_train, epochs=10, validation_data=(X_val, y_val))

# Get best model
best_model = tuner.get_best_models(num_models=1)[0]
```

## Optuna - Hyperparameter Optimization

```python
import optuna

def objective(trial):
    # Suggest hyperparameters
    n_estimators = trial.suggest_int('n_estimators', 50, 500)
    max_depth = trial.suggest_int('max_depth', 2, 32)
    learning_rate = trial.suggest_float('learning_rate', 1e-4, 1e-1, log=True)
    
    # Train model
    model = XGBClassifier(
        n_estimators=n_estimators,
        max_depth=max_depth,
        learning_rate=learning_rate
    )
    model.fit(X_train, y_train)
    
    # Return metric to optimize
    return model.score(X_val, y_val)

# Optimize
study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=100)

# Best hyperparameters
print(f"Best params: {study.best_params}")
print(f"Best score: {study.best_value}")

# Visualize optimization
optuna.visualization.plot_optimization_history(study)
optuna.visualization.plot_param_importances(study)
```

## Google Cloud AutoML

```python
from google.cloud import automl

# Create client
client = automl.AutoMlClient()

# Create dataset
dataset = client.create_dataset(
    parent=f"projects/{project_id}/locations/us-central1",
    dataset={
        "display_name": "my_dataset",
        "image_classification_dataset_metadata": {}
    }
)

# Import images
# Train model (automatically)
# Deploy model
# All handled by Google Cloud
```

## Benefits

- Save development time
- Try many models automatically
- Good starting point
- Reproducible

## Limitations

- Expensive (compute)
- Black box (less control)
- May not beat expert tuning
- Limited custom features

## Best Practices

1. Start with AutoML for baseline
2. Use found architecture as starting point
3. Combine with domain knowledge
4. Set reasonable time budget
5. Always validate manually

## Remember

- AutoML is a powerful starting point
- Not a replacement for ML knowledge
- Great for prototyping
- Can discover surprising solutions

AutoML and Neural Architecture Search

AI that focuses.

## What is Attention?

Mechanism that lets model focus on relevant parts of input.

Like reading a long document and highlighting important parts!

## Why Attention?

**Problem with RNNs**:
- Long sequences → Information gets lost
- Can't remember beginning by the end

**Solution**:
- Look back at all previous inputs
- Focus on relevant parts

## Attention in Machine Translation

Translating: "The cat sat on the mat" → "Le chat..."

When generating "chat", attention focuses on "cat"!

## Basic Attention Mechanism

```python
import numpy as np

def attention(query, keys, values):
    """
    query: What we're looking for
    keys: What we're looking in
    values: What we return
    """
    # Calculate attention scores
    scores = np.dot(query, keys.T)  # Similarity
    
    # Normalize to probabilities
    weights = softmax(scores)
    
    # Weighted sum of values
    output = np.dot(weights, values)
    
    return output, weights

# Example
query = np.array([1, 0, 0])  # Current word encoding
keys = np.array([
    [1, 0, 0],  # Previous word 1
    [0, 1, 0],  # Previous word 2
    [0, 0, 1]   # Previous word 3
])
values = keys  # Usually same as keys

output, weights = attention(query, keys, values)
print(f"Attention weights: {weights}")  # [0.7, 0.2, 0.1]
# Focuses most on word 1!
```

## Scaled Dot-Product Attention

```python
def scaled_dot_product_attention(Q, K, V, mask=None):
    """
    Q: Query matrix
    K: Key matrix
    V: Value matrix
    """
    d_k = Q.shape[-1]
    
    # Calculate attention scores
    scores = np.matmul(Q, K.transpose(-2, -1)) / np.sqrt(d_k)
    
    # Apply mask (for decoder to not look ahead)
    if mask is not None:
        scores = scores + (mask * -1e9)
    
    # Softmax
    attention_weights = softmax(scores, axis=-1)
    
    # Weighted sum
    output = np.matmul(attention_weights, V)
    
    return output, attention_weights
```

## Attention in Seq2Seq

```python
from tensorflow.keras.layers import Layer
import tensorflow as tf

class BahdanauAttention(Layer):
    def __init__(self, units):
        super().__init__()
        self.W1 = Dense(units)
        self.W2 = Dense(units)
        self.V = Dense(1)
    
    def call(self, query, values):
        # query: decoder hidden state [batch, hidden]
        # values: encoder outputs [batch, seq_len, hidden]
        
        # Expand query to match values shape
        query_with_time = tf.expand_dims(query, 1)
        
        # Score calculation
        score = self.V(tf.nn.tanh(
            self.W1(query_with_time) + self.W2(values)
        ))
        
        # Attention weights
        attention_weights = tf.nn.softmax(score, axis=1)
        
        # Context vector
        context = attention_weights * values
        context = tf.reduce_sum(context, axis=1)
        
        return context, attention_weights

# Use in decoder
class Decoder(Layer):
    def __init__(self, vocab_size, embedding_dim, hidden_units):
        super().__init__()
        self.embedding = Embedding(vocab_size, embedding_dim)
        self.gru = GRU(hidden_units, return_sequences=True, return_state=True)
        self.fc = Dense(vocab_size)
        self.attention = BahdanauAttention(hidden_units)
    
    def call(self, x, hidden, encoder_outputs):
        # Apply attention
        context, attention_weights = self.attention(hidden, encoder_outputs)
        
        # Embed input
        x = self.embedding(x)
        
        # Concatenate context and embedded input
        x = tf.concat([tf.expand_dims(context, 1), x], axis=-1)
        
        # Pass through GRU
        output, state = self.gru(x)
        
        # Final prediction
        x = self.fc(output)
        
        return x, state, attention_weights
```

## Self-Attention

Each element attends to all elements (including itself):

```python
def self_attention(X):
    """
    X: Input sequence [batch, seq_len, d_model]
    """
    # Create Q, K, V from same input
    Q = X @ Wq
    K = X @ Wk
    V = X @ Wv
    
    # Scaled dot-product attention
    output = scaled_dot_product_attention(Q, K, V)
    
    return output

# Example: "The cat sat"
# "cat" can attend to "The", "cat", "sat"
# Learns relationships between all words
```

## Multi-Head Attention

Multiple attention mechanisms in parallel:

```python
class MultiHeadAttention(Layer):
    def __init__(self, d_model, num_heads):
        super().__init__()
        self.num_heads = num_heads
        self.d_model = d_model
        
        assert d_model % num_heads == 0
        
        self.depth = d_model // num_heads
        
        self.wq = Dense(d_model)
        self.wk = Dense(d_model)
        self.wv = Dense(d_model)
        self.dense = Dense(d_model)
    
    def split_heads(self, x, batch_size):
        x = tf.reshape(x, (batch_size, -1, self.num_heads, self.depth))
        return tf.transpose(x, perm=[0, 2, 1, 3])
    
    def call(self, query, key, value):
        batch_size = tf.shape(query)[0]
        
        # Linear projections
        query = self.wq(query)
        key = self.wk(key)
        value = self.wv(value)
        
        # Split into multiple heads
        query = self.split_heads(query, batch_size)
        key = self.split_heads(key, batch_size)
        value = self.split_heads(value, batch_size)
        
        # Scaled dot-product attention for each head
        scaled_attention = scaled_dot_product_attention(query, key, value)
        
        # Concatenate heads
        concat_attention = tf.reshape(scaled_attention, (batch_size, -1, self.d_model))
        
        # Final linear projection
        output = self.dense(concat_attention)
        
        return output
```

## Visualizing Attention

```python
import matplotlib.pyplot as plt
import seaborn as sns

def plot_attention(attention, sentence, translation):
    fig, ax = plt.subplots(figsize=(10, 10))
    
    sns.heatmap(attention, xticklabels=sentence, yticklabels=translation, ax=ax)
    
    ax.set_xlabel('Input')
    ax.set_ylabel('Output')
    plt.show()

# Shows which input words the model focused on for each output word
```

## Applications

- Machine translation
- Text summarization
- Image captioning
- Question answering
- Speech recognition

## Remember

- Attention allows focusing on relevant parts
- Self-attention enables Transformers
- Multi-head attention learns different relationships
- Visualization helps understand what model learned

Attention Mechanisms

AI for networks and relationships.

## What are Graph Neural Networks?

Neural networks that work on graph data.

**Graphs**: Networks of nodes and edges
**Examples**: Social networks, molecules, knowledge graphs

## Why GNNs?

Traditional NNs need fixed-size inputs.

**Graphs are different**:
- Variable number of nodes
- Complex relationships
- Irregular structure

## Graph Structure

```python
import networkx as nx

# Create graph
G = nx.Graph()

# Add nodes (people)
G.add_nodes_from(['Alice', 'Bob', 'Charlie', 'David'])

# Add edges (friendships)
G.add_edges_from([
    ('Alice', 'Bob'),
    ('Bob', 'Charlie'),
    ('Charlie', 'David')
])

# Visualize
nx.draw(G, with_labels=True)
```

## Message Passing

**Core idea**: Each node aggregates information from neighbors

**Steps**:
1. Each node has features
2. Nodes send messages to neighbors
3. Each node aggregates received messages
4. Update node features

```python
# Simplified message passing
def message_passing(node_features, adjacency_matrix):
    # For each node
    for i in range(num_nodes):
        # Aggregate neighbor features
        neighbor_sum = 0
        for j in range(num_nodes):
            if adjacency_matrix[i][j] == 1:  # Is neighbor
                neighbor_sum += node_features[j]
        
        # Update node feature
        node_features[i] = activation(W @ neighbor_sum + b)
    
    return node_features
```

## Graph Convolutional Network (GCN)

```python
import torch
import torch.nn as nn
import torch.nn.functional as F

class GCNLayer(nn.Module):
    def __init__(self, in_features, out_features):
        super().__init__()
        self.weight = nn.Parameter(torch.FloatTensor(in_features, out_features))
        self.bias = nn.Parameter(torch.FloatTensor(out_features))
        
    def forward(self, x, adj):
        # x: node features [num_nodes, in_features]
        # adj: adjacency matrix [num_nodes, num_nodes]
        
        # Aggregate neighbors
        support = torch.mm(x, self.weight)
        output = torch.spmm(adj, support) + self.bias
        
        return output

class GCN(nn.Module):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super().__init__()
        self.gc1 = GCNLayer(input_dim, hidden_dim)
        self.gc2 = GCNLayer(hidden_dim, output_dim)
    
    def forward(self, x, adj):
        x = F.relu(self.gc1(x, adj))
        x = F.dropout(x, training=self.training)
        x = self.gc2(x, adj)
        return F.log_softmax(x, dim=1)

# Use GCN
model = GCN(input_dim=feature_dim, hidden_dim=16, output_dim=num_classes)
output = model(node_features, adjacency_matrix)
```

## Using PyTorch Geometric

```python
import torch
from torch_geometric.nn import GCNConv
from torch_geometric.data import Data

# Create graph data
edge_index = torch.tensor([
    [0, 1, 1, 2],  # Source nodes
    [1, 0, 2, 1]   # Target nodes
], dtype=torch.long)

x = torch.tensor([  # Node features
    [-1], [0], [1]
], dtype=torch.float)

data = Data(x=x, edge_index=edge_index)

# Build GNN
class Net(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = GCNConv(1, 16)
        self.conv2 = GCNConv(16, 2)
    
    def forward(self, data):
        x, edge_index = data.x, data.edge_index
        
        x = self.conv1(x, edge_index)
        x = F.relu(x)
        x = F.dropout(x, training=self.training)
        x = self.conv2(x, edge_index)
        
        return F.log_softmax(x, dim=1)

model = Net()
```

## Graph Attention Networks (GAT)

Learns importance of neighbors:

```python
from torch_geometric.nn import GATConv

class GAT(torch.nn.Module):
    def __init__(self, in_channels, out_channels):
        super().__init__()
        self.conv1 = GATConv(in_channels, 8, heads=8, dropout=0.6)
        self.conv2 = GATConv(8 * 8, out_channels, heads=1, concat=False, dropout=0.6)
    
    def forward(self, x, edge_index):
        x = F.dropout(x, p=0.6, training=self.training)
        x = F.elu(self.conv1(x, edge_index))
        x = F.dropout(x, p=0.6, training=self.training)
        x = self.conv2(x, edge_index)
        return F.log_softmax(x, dim=1)
```

## Node Classification

Predict properties of nodes:

```python
# Example: Classify users in social network

# Forward pass
output = model(data)

# Calculate loss (only on training nodes)
loss = F.nll_loss(output[data.train_mask], data.y[data.train_mask])

# Backprop
loss.backward()
optimizer.step()

# Evaluate
pred = output.argmax(dim=1)
correct = (pred[data.test_mask] == data.y[data.test_mask]).sum()
accuracy = int(correct) / int(data.test_mask.sum())
```

## Graph Classification

Classify entire graphs:

```python
from torch_geometric.nn import global_mean_pool

class GraphClassifier(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = GCNConv(num_features, 64)
        self.conv2 = GCNConv(64, 64)
        self.fc = torch.nn.Linear(64, num_classes)
    
    def forward(self, data):
        x, edge_index, batch = data.x, data.edge_index, data.batch
        
        # Node-level operations
        x = F.relu(self.conv1(x, edge_index))
        x = F.relu(self.conv2(x, edge_index))
        
        # Graph-level pooling
        x = global_mean_pool(x, batch)
        
        # Classification
        x = self.fc(x)
        return F.log_softmax(x, dim=1)

# Use for molecule classification, etc.
```

## Applications

- **Social networks**: Friend recommendations, influence prediction
- **Molecules**: Drug discovery, property prediction
- **Knowledge graphs**: Question answering, reasoning
- **Traffic**: Traffic prediction, route optimization
- **Recommendation**: User-item graphs

## Real Example - Molecular Property Prediction

```python
from torch_geometric.datasets import MoleculeNet

# Load molecule dataset
dataset = MoleculeNet(root='data', name='ESOL')

# Each molecule is a graph
# Nodes = atoms, Edges = bonds

# Train GNN to predict solubility
model = GCN(input_dim=dataset.num_features, hidden_dim=64, output_dim=1)

for data in loader:
    output = model(data.x, data.edge_index, data.batch)
    loss = F.mse_loss(output, data.y)
    loss.backward()
```

## Remember

- GNNs work on graph-structured data
- Message passing is the core operation
- PyTorch Geometric makes implementation easy
- Many applications beyond images and text

Neural networks for graph-structured data.

Graph Neural Networks

AI that learns how to learn.

## What is Meta-Learning?

Training models to adapt quickly to new tasks.

**Goal**: Learn from just a few examples

Like learning your 5th language is easier than your 1st!

## Few-Shot Learning

Learn from very few examples:

**1-shot**: 1 example per class
**5-shot**: 5 examples per class
**Zero-shot**: 0 examples (just description)

## Why Meta-Learning?

**Problem**: Deep learning needs lots of data

**Solution**: Train on many tasks, adapt quickly to new task

## Example Scenario

**Training**: Learn 1000 different tasks
**Testing**: New task with only 5 examples
**Result**: Model adapts quickly!

## Model-Agnostic Meta-Learning (MAML)

Most popular meta-learning algorithm:

```python
import torch
import torch.nn as nn

class MAML:
    def __init__(self, model, inner_lr=0.01, meta_lr=0.001):
        self.model = model
        self.inner_lr = inner_lr
        self.meta_optimizer = torch.optim.Adam(model.parameters(), lr=meta_lr)
    
    def inner_loop(self, support_x, support_y):
        """Adapt to a single task"""
        # Clone model
        adapted_model = copy.deepcopy(self.model)
        inner_optimizer = torch.optim.SGD(adapted_model.parameters(), lr=self.inner_lr)
        
        # Few gradient steps on support set
        for _ in range(5):
            predictions = adapted_model(support_x)
            loss = F.cross_entropy(predictions, support_y)
            
            inner_optimizer.zero_grad()
            loss.backward()
            inner_optimizer.step()
        
        return adapted_model
    
    def outer_loop(self, tasks):
        """Update meta-parameters"""
        meta_loss = 0
        
        for task in tasks:
            support_x, support_y, query_x, query_y = task
            
            # Inner loop: adapt to task
            adapted_model = self.inner_loop(support_x, support_y)
            
            # Evaluate on query set
            predictions = adapted_model(query_x)
            loss = F.cross_entropy(predictions, query_y)
            meta_loss += loss
        
        # Update original model
        self.meta_optimizer.zero_grad()
        meta_loss.backward()
        self.meta_optimizer.step()
        
        return meta_loss

# Usage
model = SimpleCNN()
maml = MAML(model)

# Training
for epoch in range(100):
    # Sample batch of tasks
    tasks = sample_tasks(batch_size=32)
    
    # Meta-update
    loss = maml.outer_loop(tasks)
    print(f"Epoch {epoch}, Loss: {loss}")

# Testing: Adapt to new task with few examples
adapted = maml.inner_loop(new_task_support_x, new_task_support_y)
accuracy = test(adapted, new_task_query_x, new_task_query_y)
```

## Prototypical Networks

Learn prototypes (representations) for each class:

```python
class PrototypicalNetwork(nn.Module):
    def __init__(self, encoder):
        super().__init__()
        self.encoder = encoder
    
    def forward(self, support_x, support_y, query_x):
        # Encode all examples
        support_features = self.encoder(support_x)
        query_features = self.encoder(query_x)
        
        # Calculate prototype for each class
        unique_labels = torch.unique(support_y)
        prototypes = []
        
        for label in unique_labels:
            # Average of all examples in class
            class_examples = support_features[support_y == label]
            prototype = class_examples.mean(dim=0)
            prototypes.append(prototype)
        
        prototypes = torch.stack(prototypes)
        
        # Classify query based on nearest prototype
        distances = torch.cdist(query_features, prototypes)
        predictions = (-distances).softmax(dim=1)
        
        return predictions

# Usage
encoder = SimpleCNN()
model = PrototypicalNetwork(encoder)

# 5-way 1-shot learning
support_x = ...  # [5 classes × 1 example × features]
support_y = ...  # [5]
query_x = ...    # [test examples]

predictions = model(support_x, support_y, query_x)
```

## Matching Networks

Attention-based few-shot learning:

```python
class MatchingNetwork(nn.Module):
    def __init__(self, encoder):
        super().__init__()
        self.encoder = encoder
        self.attention = nn.MultiheadAttention(embed_dim=128, num_heads=1)
    
    def forward(self, support_x, support_y, query_x):
        # Encode
        support_emb = self.encoder(support_x)
        query_emb = self.encoder(query_x)
        
        # Attention from query to support set
        attended, weights = self.attention(
            query_emb.unsqueeze(0),
            support_emb.unsqueeze(0),
            support_emb.unsqueeze(0)
        )
        
        # Weighted combination of support labels
        predictions = weights @ support_y
        
        return predictions
```

## Omniglot Dataset

Standard meta-learning benchmark:

```python
from torchvision.datasets import Omniglot

# 1623 characters from 50 alphabets
dataset = Omniglot(root='./data', download=True)

# Create N-way K-shot tasks
def create_task(N=5, K=1):
    # Sample N classes
    classes = np.random.choice(len(dataset.classes), N, replace=False)
    
    support_x, support_y = [], []
    query_x, query_y = [], []
    
    for i, cls in enumerate(classes):
        # Sample K examples for support
        examples = sample_class_examples(cls, K+15)
        support_x.append(examples[:K])
        support_y.extend([i] * K)
        
        # Rest for query
        query_x.append(examples[K:])
        query_y.extend([i] * 15)
    
    return support_x, support_y, query_x, query_y
```

## Meta-Learning for NLP

```python
# Few-shot text classification

# Support set (few examples)
support_texts = [
    "This product is amazing!",  # Positive
    "Terrible service",           # Negative
]
support_labels = [1, 0]

# Query (new text to classify)
query_text = "Best purchase ever!"

# MAML or Prototypical Networks on BERT embeddings
embeddings = bert_encode(support_texts + [query_text])
model = PrototypicalNetwork(embedding_dim=768)
prediction = model(embeddings[:2], support_labels, embeddings[2:])
```

## Applications

- **Few-shot classification**: New categories with few examples
- **Personalization**: Adapt to user with few interactions
- **Robot learning**: Learn new tasks quickly
- **Drug discovery**: Predict properties with limited data

## Challenges

- Still requires many tasks for training
- Computational cost
- Task distribution shift

## Remember

- Meta-learning = learning to learn
- Enables few-shot learning
- MAML most versatile
- Prototypical Networks simple and effective

Train models to learn quickly from few examples.

Meta-Learning (Learning to Learn)

Understand AI decisions.

## Why Explainability Matters

**Trust**: Users need to understand decisions
**Debugging**: Find model mistakes
**Compliance**: Laws require explanations (GDPR, etc.)
**Fairness**: Detect bias

## Types of Explanations

**Global**: How does model work overall?
**Local**: Why this specific prediction?

## LIME (Local Interpretable Model-agnostic Explanations)

Explain any model:

```python
from lime.lime_tabular import LimeTabularExplainer
import sklearn

# Train black-box model
model = RandomForestClassifier()
model.fit(X_train, y_train)

# Create explainer
explainer = LimeTabularExplainer(
    X_train.values,
    feature_names=X_train.columns,
    class_names=['Rejected', 'Approved'],
    mode='classification'
)

# Explain single prediction
i = 100
explanation = explainer.explain_instance(
    X_test.iloc[i].values,
    model.predict_proba,
    num_features=5
)

# Show explanation
explanation.show_in_notebook()

# Get feature contributions
explanation.as_list()
# [('age > 30', 0.45), ('income > 50000', 0.32), ...]
```

## SHAP (SHapley Additive exPlanations)

Based on game theory:

```python
import shap

# Train model
model = XGBClassifier()
model.fit(X_train, y_train)

# Create explainer
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)

# Visualizations

# 1. Force plot (single prediction)
shap.force_plot(
    explainer.expected_value,
    shap_values[0],
    X_test.iloc[0]
)

# 2. Summary plot (all predictions)
shap.summary_plot(shap_values, X_test)

# 3. Dependence plot (feature effect)
shap.dependence_plot('age', shap_values, X_test)

# 4. Waterfall plot
shap.waterfall_plot(shap.Explanation(
    values=shap_values[0],
    base_values=explainer.expected_value,
    data=X_test.iloc[0]
))
```

## Grad-CAM for Images

Visualize what CNN focuses on:

```python
import tensorflow as tf
from tensorflow.keras.models import Model

def make_gradcam_heatmap(img_array, model, last_conv_layer_name):
    # Gradient model
    grad_model = Model(
        inputs=[model.inputs],
        outputs=[model.get_layer(last_conv_layer_name).output, model.output]
    )
    
    with tf.GradientTape() as tape:
        conv_outputs, predictions = grad_model(img_array)
        class_channel = predictions[:, np.argmax(predictions[0])]
    
    # Gradient of class w.r.t. conv layer
    grads = tape.gradient(class_channel, conv_outputs)
    
    # Global average pooling
    pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))
    
    # Weight conv outputs by gradients
    conv_outputs = conv_outputs[0]
    heatmap = conv_outputs @ pooled_grads[..., tf.newaxis]
    heatmap = tf.squeeze(heatmap)
    
    # Normalize
    heatmap = tf.maximum(heatmap, 0) / tf.math.reduce_max(heatmap)
    return heatmap.numpy()

# Use it
heatmap = make_gradcam_heatmap(img, model, 'conv5_block3_out')

# Overlay on image
import matplotlib.pyplot as plt
import cv2

plt.imshow(img)
plt.imshow(heatmap, alpha=0.4, cmap='jet')
plt.show()

# Shows: Model focused on cat's face to classify it as "cat"
```

## Attention Visualization

For Transformers:

```python
from transformers import BertTokenizer, BertModel
import torch

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased', output_attentions=True)

text = "The restaurant in Austin was amazing"
inputs = tokenizer(text, return_tensors='pt')

# Get attention weights
outputs = model(**inputs)
attention = outputs.attentions  # (layers, heads, tokens, tokens)

# Visualize attention from last layer
import seaborn as sns

tokens = tokenizer.convert_ids_to_tokens(inputs['input_ids'][0])
last_layer_attention = attention[-1][0].mean(dim=0).detach().numpy()

plt.figure(figsize=(10, 10))
sns.heatmap(last_layer_attention, xticklabels=tokens, yticklabels=tokens)
plt.title("Attention Weights")
plt.show()

# Shows: "amazing" attends to "restaurant" and "Austin"
```

## Counterfactual Explanations

"What changes would change the prediction?"

```python
from dice_ml import Data, Model, Dice

# Setup
d = Data(dataframe=df, continuous_features=['age', 'income'], outcome_name='approved')
m = Model(model=model, backend='sklearn')
exp = Dice(d, m)

# Generate counterfactuals
query_instance = df[df['approved'] == 0].iloc[0]
dice_exp = exp.generate_counterfactuals(
    query_instance,
    total_CFs=3,
    desired_class=1
)

dice_exp.visualize_as_dataframe()

# Output:
# "If age was 35 instead of 28, would be approved"
# "If income was $60k instead of $45k, would be approved"
```

## Anchors

High-precision rules:

```python
from anchor import anchor_tabular

# Create explainer
explainer = anchor_tabular.AnchorTabularExplainer(
    class_names=['Rejected', 'Approved'],
    feature_names=X_train.columns,
    train_data=X_train.values
)

# Get anchor (rule that guarantees prediction)
explanation = explainer.explain_instance(
    X_test.iloc[0].values,
    model.predict,
    threshold=0.95
)

print(explanation.anchor)
# "IF age > 30 AND income > 50000 THEN Approved (95% confidence)"
```

## Feature Attribution

```python
from alibi.explainers import IntegratedGradients

# For neural networks
ig = IntegratedGradients(model)

# Get attributions
explanation = ig.explain(X_test[0:1])

# Plot feature importance
attributions = explanation.attributions[0]
plt.barh(feature_names, attributions)
plt.xlabel('Attribution')
plt.title('Feature Importance')
plt.show()
```

## Model Cards

Document model details:

```python
# model_card.md

## Model Details
- **Developed by**: AI Team, Company Name
- **Model date**: December 2025
- **Model type**: XGBoost Classifier
- **Version**: 1.2

## Intended Use
- **Primary use**: Credit approval decisions
- **Primary users**: Loan officers in Denver office
- **Out-of-scope**: Not for medical decisions

## Training Data
- **Dataset**: Customer data from 2020-2024
- **Size**: 100,000 samples
- **Features**: Age, income, credit score, employment

## Performance
- **Accuracy**: 87%
- **Precision**: 85%
- **Recall**: 82%
- **Tested on**: 20,000 holdout samples

## Limitations
- Lower accuracy for customers < 25 years old
- May not generalize to other regions
- Requires annual retraining

## Ethical Considerations
- Regular bias audits performed
- Explanations provided for all decisions
- Human review for edge cases
```

## Tools & Libraries

- **LIME**: Model-agnostic local explanations
- **SHAP**: Global and local explanations
- **Captum**: PyTorch interpretability
- **InterpretML**: Microsoft's library
- **What-If Tool**: Interactive visualizations

## Best Practices

1. Use multiple explanation methods
2. Validate explanations with domain experts
3. Document limitations clearly
4. Provide explanations to end users
5. Regular bias audits

## Remember

- Explainability builds trust
- Different stakeholders need different explanations
- No single perfect method
- Combine global and local explanations

Make AI decisions interpretable and transparent.

Explainable AI (XAI)

Privacy-preserving machine learning.

## What is Federated Learning?

Train model across many devices without centralizing data.

**Key idea**: Move code to data, not data to code!

## Why Federated Learning?

**Privacy**: Data stays on device
**Security**: No central data target
**Efficiency**: Use edge computing
**Regulation**: GDPR, HIPAA compliance

## How It Works

1. Server sends model to devices
2. Devices train locally on their data
3. Devices send updates (not data!) to server
4. Server averages updates
5. Repeat

## Simple Example

```python
import torch
import torch.nn as nn

# Central server
class Server:
    def __init__(self, model):
        self.global_model = model
    
    def aggregate(self, client_models):
        """Average client model weights"""
        global_dict = self.global_model.state_dict()
        
        for key in global_dict.keys():
            # Average weights from all clients
            global_dict[key] = torch.stack([
                client.state_dict()[key].float() 
                for client in client_models
            ]).mean(0)
        
        self.global_model.load_state_dict(global_dict)
        return self.global_model

# Client device
class Client:
    def __init__(self, model, data, device_id):
        self.model = model
        self.data = data
        self.device_id = device_id
    
    def train(self, epochs=1):
        """Train on local data"""
        optimizer = torch.optim.SGD(self.model.parameters(), lr=0.01)
        
        for epoch in range(epochs):
            for x, y in self.data:
                optimizer.zero_grad()
                output = self.model(x)
                loss = nn.CrossEntropyLoss()(output, y)
                loss.backward()
                optimizer.step()
        
        return self.model

# Federated training loop
def federated_learning(server, clients, rounds=10):
    for round in range(rounds):
        print(f"Round {round+1}/{rounds}")
        
        # 1. Server sends model to clients
        client_models = []
        
        for client in clients:
            # Copy global model to client
            client.model.load_state_dict(server.global_model.state_dict())
            
            # Client trains locally
            trained_model = client.train(epochs=5)
            client_models.append(trained_model)
        
        # 2. Server aggregates client updates
        server.aggregate(client_models)
        
        # 3. Evaluate global model
        accuracy = evaluate(server.global_model, test_data)
        print(f"Global accuracy: {accuracy:.2f}%")

# Usage
model = SimpleCNN()
server = Server(model)

# Create clients (e.g., user phones)
clients = [
    Client(copy.deepcopy(model), user1_data, "device_1"),
    Client(copy.deepcopy(model), user2_data, "device_2"),
    Client(copy.deepcopy(model), user3_data, "device_3"),
]

federated_learning(server, clients, rounds=50)
```

## Using PySyft

```python
import syft as sy
import torch

# Create virtual devices
hook = sy.TorchHook(torch)
alice = sy.VirtualWorker(hook, id="alice")
bob = sy.VirtualWorker(hook, id="bob")

# Send data to devices
alice_data = data[:50].send(alice)
bob_data = data[50:].send(bob)

# Define model
model = nn.Sequential(
    nn.Linear(784, 128),
    nn.ReLU(),
    nn.Linear(128, 10)
)

# Train federally
for epoch in range(10):
    # Train on Alice's device
    model = model.send(alice)
    optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
    
    for x, y in alice_data:
        optimizer.zero_grad()
        output = model(x)
        loss = F.nll_loss(output, y)
        loss.backward()
        optimizer.step()
    
    model = model.get()  # Retrieve from Alice
    
    # Train on Bob's device
    model = model.send(bob)
    # ... same training ...
    model = model.get()
```

## Using TensorFlow Federated

```python
import tensorflow_federated as tff

# Define model
def create_keras_model():
    return tf.keras.models.Sequential([
        tf.keras.layers.Dense(128, activation='relu', input_shape=(784,)),
        tf.keras.layers.Dense(10, activation='softmax')
    ])

# Convert to TFF model
def model_fn():
    keras_model = create_keras_model()
    return tff.learning.from_keras_model(
        keras_model,
        input_spec=input_spec,
        loss=tf.keras.losses.SparseCategoricalCrossentropy(),
        metrics=[tf.keras.metrics.SparseCategoricalAccuracy()]
    )

# Build federated averaging process
iterative_process = tff.learning.build_federated_averaging_process(model_fn)

# Simulate federated training
state = iterative_process.initialize()

for round in range(10):
    # Each client's dataset
    federated_data = [client1_data, client2_data, client3_data]
    
    # One round of federated training
    state, metrics = iterative_process.next(state, federated_data)
    
    print(f'Round {round}, Metrics: {metrics}')
```

## Differential Privacy

Add noise to protect individual privacy:

```python
from tensorflow_privacy.privacy.optimizers import dp_optimizer

# DP-SGD optimizer
optimizer = dp_optimizer.DPAdamGaussianOptimizer(
    l2_norm_clip=1.0,        # Clip gradients
    noise_multiplier=1.1,     # Amount of noise
    num_microbatches=1,
    learning_rate=0.001
)

# Train with differential privacy
model.compile(optimizer=optimizer, loss='sparse_categorical_crossentropy')
model.fit(X_train, y_train, epochs=10, batch_size=256)

# Privacy budget (epsilon)
# Lower epsilon = more privacy, less accuracy
```

## Secure Aggregation

Encrypt model updates:

```python
# Using PySyft's secure aggregation

# Each client encrypts their model update
encrypted_updates = []
for client in clients:
    update = client.train()
    encrypted = encrypt(update)
    encrypted_updates.append(encrypted)

# Server aggregates without seeing individual updates
aggregated = secure_aggregate(encrypted_updates)
decrypted = decrypt(aggregated)

# Server only sees average, not individual contributions
```

## Challenges

**Communication cost**: Sending models is expensive
- Solution: Compression, quantization

**Data heterogeneity**: Each device has different data
- Solution: Personalization layers

**Device availability**: Devices go offline
- Solution: Asynchronous updates

**Convergence**: Takes longer than centralized
- Solution: More sophisticated aggregation

## Real Applications

**Google Keyboard**: Learns typing patterns without seeing your messages

**Apple Siri**: Improves voice recognition privately

**Healthcare**: Train on hospital data without sharing patient records

**IoT**: Learn from sensors without centralizing all data

## Advanced Aggregation

```python
# Weighted averaging (by dataset size)
def weighted_aggregate(client_models, client_sizes):
    total_size = sum(client_sizes)
    weights = [size / total_size for size in client_sizes]
    
    global_dict = {}
    for key in client_models[0].state_dict().keys():
        global_dict[key] = sum(
            weight * model.state_dict()[key]
            for weight, model in zip(weights, client_models)
        )
    
    return global_dict

# FedProx (handles heterogeneity better)
def fedprox_train(model, data, global_model, mu=0.01):
    optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
    
    for x, y in data:
        optimizer.zero_grad()
        output = model(x)
        loss = F.cross_entropy(output, y)
        
        # Proximal term
        proximal_term = 0
        for param, global_param in zip(model.parameters(), global_model.parameters()):
            proximal_term += ((param - global_param) ** 2).sum()
        
        total_loss = loss + (mu / 2) * proximal_term
        total_loss.backward()
        optimizer.step()
```

## Best Practices

1. Use secure aggregation
2. Add differential privacy when needed
3. Handle stragglers (slow devices)
4. Monitor convergence carefully
5. Test communication efficiency

## Remember

- Federated learning enables privacy-preserving ML
- Data never leaves devices
- More complex than centralized training
- Growing in importance with privacy regulations

Federated Learning

Let AI build AI models for you.

## What is AutoML?

Automatically find best model and hyperparameters.

**Goal**: Make ML accessible to non-experts!

## Why AutoML?

**Speed**: Try hundreds of models automatically
**Accuracy**: Often beats manual tuning
**Efficiency**: Saves data scientist time

## AutoML with Auto-sklearn

Install:
```bash
pip install auto-sklearn
```

Use it:
```python
import autosklearn.classification
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_digits

# Load data
X, y = load_digits(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

# AutoML
automl = autosklearn.classification.AutoSklearnClassifier(
    time_left_for_this_task=120,  # 2 minutes
    per_run_time_limit=30
)
automl.fit(X_train, y_train)

# Test
score = automl.score(X_test, y_test)
print(f"Accuracy: {score:.2f}")

# See best model
print(automl.show_models())
```

## AutoML with TPOT

```python
from tpot import TPOTClassifier

# AutoML with genetic programming
tpot = TPOTClassifier(
    generations=5,
    population_size=20,
    verbosity=2,
    random_state=42
)
tpot.fit(X_train, y_train)

# Test
print(f"Score: {tpot.score(X_test, y_test):.2f}")

# Export best pipeline
tpot.export('best_model.py')
```

## Popular AutoML Tools

**Auto-sklearn**: Based on scikit-learn
**TPOT**: Genetic programming approach
**H2O AutoML**: Enterprise solution
**Google AutoML**: Cloud-based
**AutoKeras**: For deep learning

## When to Use AutoML

**Good for**: Quick prototypes, baseline models
**Not ideal for**: Very specialized problems, when you need full control

## Remember

- AutoML saves time on model selection
- Still need good data and feature engineering
- Great for getting started quickly
- Understand the basics before using AutoML

AutoML (Automated Machine Learning)

Two networks competing to create realistic data.

## What are GANs?

Generator creates fake data, Discriminator judges if it's real.

**Idea**: They improve each other through competition!

## How GANs Work

**Generator**: Creates fake images
**Discriminator**: Detects real vs fake
**Training**: Generator tries to fool Discriminator

## Simple GAN Example

```python
import tensorflow as tf
from tensorflow import keras

# Generator model
def make_generator():
    model = keras.Sequential([
        keras.layers.Dense(256, activation='relu', input_dim=100),
        keras.layers.Dense(512, activation='relu'),
        keras.layers.Dense(1024, activation='relu'),
        keras.layers.Dense(784, activation='sigmoid')  # 28x28 image
    ])
    return model

# Discriminator model
def make_discriminator():
    model = keras.Sequential([
        keras.layers.Dense(1024, activation='relu', input_dim=784),
        keras.layers.Dropout(0.3),
        keras.layers.Dense(512, activation='relu'),
        keras.layers.Dropout(0.3),
        keras.layers.Dense(256, activation='relu'),
        keras.layers.Dense(1, activation='sigmoid')  # Real or Fake
    ])
    return model

generator = make_generator()
discriminator = make_discriminator()

# Compile
discriminator.compile(optimizer='adam', loss='binary_crossentropy')
```

## Training GANs

```python
import numpy as np

# Prepare real data (example with MNIST)
(X_train, _), (_, _) = keras.datasets.mnist.load_data()
X_train = X_train.reshape(-1, 784) / 255.0

batch_size = 128
epochs = 10000

for epoch in range(epochs):
    # Train Discriminator
    # 1. Real images
    idx = np.random.randint(0, X_train.shape[0], batch_size)
    real_images = X_train[idx]
    real_labels = np.ones((batch_size, 1))
    
    # 2. Fake images
    noise = np.random.normal(0, 1, (batch_size, 100))
    fake_images = generator.predict(noise)
    fake_labels = np.zeros((batch_size, 1))
    
    # Train discriminator
    d_loss_real = discriminator.train_on_batch(real_images, real_labels)
    d_loss_fake = discriminator.train_on_batch(fake_images, fake_labels)
    
    # Train Generator
    noise = np.random.normal(0, 1, (batch_size, 100))
    misleading_labels = np.ones((batch_size, 1))  # Fool discriminator!
    
    # Combined model
    discriminator.trainable = False
    combined = keras.Sequential([generator, discriminator])
    combined.compile(optimizer='adam', loss='binary_crossentropy')
    g_loss = combined.train_on_batch(noise, misleading_labels)
    discriminator.trainable = True
    
    if epoch % 1000 == 0:
        print(f"Epoch {epoch}, D Loss: {(d_loss_real + d_loss_fake)/2:.4f}, G Loss: {g_loss:.4f}")
```

## Types of GANs

**DCGAN**: Uses convolutional layers
**StyleGAN**: High-quality face generation
**CycleGAN**: Image-to-image translation
**Pix2Pix**: Paired image translation

## Applications

- Generate realistic faces
- Create art and designs
- Data augmentation
- Image enhancement

## Remember

- GANs are hard to train (unstable)
- Require lots of data and compute
- Generator and Discriminator must balance
- Used for creative AI applications

GANs (Generative Adversarial Networks)

Agent learns by taking actions and getting rewards.

## What is Reinforcement Learning?

Agent interacts with environment to maximize rewards.

**Like training a dog**: Good behavior → Reward!

## Key Concepts

**Agent**: Learner (like a robot)
**Environment**: World agent interacts with
**State**: Current situation
**Action**: What agent can do
**Reward**: Feedback (positive or negative)

## Simple RL Example

```python
import gym
import numpy as np

# Create environment (simple grid world)
env = gym.make('FrozenLake-v1')

# Q-learning table
Q = np.zeros([env.observation_space.n, env.action_space.n])

# Parameters
learning_rate = 0.8
discount_factor = 0.95
epsilon = 0.1  # Exploration rate
episodes = 2000

# Training
for episode in range(episodes):
    state = env.reset()
    done = False
    
    while not done:
        # Choose action (explore vs exploit)
        if np.random.uniform(0, 1) < epsilon:
            action = env.action_space.sample()  # Explore
        else:
            action = np.argmax(Q[state, :])  # Exploit
        
        # Take action
        next_state, reward, done, info = env.step(action)
        
        # Update Q-table
        old_value = Q[state, action]
        next_max = np.max(Q[next_state, :])
        new_value = old_value + learning_rate * (reward + discount_factor * next_max - old_value)
        Q[state, action] = new_value
        
        state = next_state
    
    if episode % 100 == 0:
        print(f"Episode {episode} completed")

print("Training finished!")
```

## Test Trained Agent

```python
# Test the agent
state = env.reset()
done = False
total_reward = 0

while not done:
    action = np.argmax(Q[state, :])  # Use learned policy
    state, reward, done, info = env.step(action)
    total_reward += reward

print(f"Total reward: {total_reward}")
```

## RL Algorithms

**Q-Learning**: Learn action values
**SARSA**: On-policy learning
**DQN**: Deep Q-Network (uses neural networks)
**A3C**: Asynchronous Actor-Critic
**PPO**: Proximal Policy Optimization

## Applications

- Game playing (Chess, Go, video games)
- Robotics
- Self-driving cars
- Resource management

## Remember

- RL learns from trial and error
- Balance exploration vs exploitation
- Requires many episodes to learn
- Works when you can simulate environment

Reuse knowledge from one task for another.

## What is Transfer Learning?

Use model trained on big dataset for your smaller task.

**Like**: Learning guitar helps with learning ukulele!

## Why Transfer Learning?

**Less data needed**: Pretrained models already know features
**Faster training**: Start from good weights
**Better accuracy**: Benefit from large-scale training

## Transfer Learning Example

```python
from tensorflow.keras.applications import VGG16
from tensorflow.keras import layers, models
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Load pretrained model (without top layer)
base_model = VGG16(
    weights='imagenet',
    include_top=False,
    input_shape=(224, 224, 3)
)

# Freeze base model layers
base_model.trainable = False

# Add custom layers for your task
model = models.Sequential([
    base_model,
    layers.GlobalAveragePooling2D(),
    layers.Dense(256, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(10, activation='softmax')  # 10 classes
])

# Compile
model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)

print(model.summary())
```

## Train Your Model

```python
# Prepare data
train_datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True
)

train_generator = train_datagen.flow_from_directory(
    'data/train',
    target_size=(224, 224),
    batch_size=32,
    class_mode='categorical'
)

# Train
history = model.fit(
    train_generator,
    epochs=10,
    steps_per_epoch=100
)
```

## Fine-Tuning

Unfreeze some layers for better performance:

```python
# Unfreeze last few layers
base_model.trainable = True

# Freeze early layers, unfreeze last layers
for layer in base_model.layers[:-4]:
    layer.trainable = False

# Recompile with lower learning rate
model.compile(
    optimizer=keras.optimizers.Adam(1e-5),
    loss='categorical_crossentropy',
    metrics=['accuracy']
)

# Continue training
model.fit(train_generator, epochs=5)
```

## Popular Pretrained Models

**VGG16/VGG19**: Simple, effective
**ResNet50**: Very deep, skip connections
**InceptionV3**: Multiple filter sizes
**MobileNet**: Lightweight for mobile
**EfficientNet**: State-of-the-art efficiency

## When to Use Transfer Learning

**Small dataset**: Less than 10,000 images
**Similar task**: Your task similar to ImageNet
**Limited compute**: No GPU clusters

## Remember

- Transfer learning saves time and data
- Freeze early layers, train later layers
- Fine-tuning improves performance
- Works best with similar domains

Transfer Learning in Deep Learning

Let models focus on what matters.

## What is Attention?

Model learns which parts of input are important.

**Like reading**: You focus on key words, not every letter!

## Why Attention?

**Better performance**: Focus on relevant information
**Interpretability**: See what model focuses on
**Long sequences**: Handle long text better than RNNs

## Attention Score Calculation

Basic formula:
```
Attention(Q, K, V) = softmax(Q·K^T / sqrt(d_k)) · V
```

Where:
- Q = Query
- K = Key
- V = Value
- d_k = Dimension of key

## Simple Attention Example

```python
import numpy as np

def softmax(x):
    exp_x = np.exp(x - np.max(x))
    return exp_x / exp_x.sum(axis=-1, keepdims=True)

def attention(query, key, value):
    # Calculate attention scores
    scores = np.dot(query, key.T)
    
    # Scale by square root of dimension
    d_k = key.shape[-1]
    scores = scores / np.sqrt(d_k)
    
    # Apply softmax
    attention_weights = softmax(scores)
    
    # Apply weights to values
    output = np.dot(attention_weights, value)
    
    return output, attention_weights

# Example
query = np.array([[1, 0, 1]])
key = np.array([[1, 0, 1], [0, 1, 0], [1, 1, 0]])
value = np.array([[1, 2], [3, 4], [5, 6]])

output, weights = attention(query, key, value)
print("Output:", output)
print("Attention weights:", weights)
```

## Self-Attention

Each word attends to all words in sentence:

```python
import tensorflow as tf

class SelfAttention(tf.keras.layers.Layer):
    def __init__(self, d_model):
        super().__init__()
        self.d_model = d_model
        self.query_dense = tf.keras.layers.Dense(d_model)
        self.key_dense = tf.keras.layers.Dense(d_model)
        self.value_dense = tf.keras.layers.Dense(d_model)
    
    def call(self, inputs):
        # Linear projections
        query = self.query_dense(inputs)
        key = self.key_dense(inputs)
        value = self.value_dense(inputs)
        
        # Attention scores
        scores = tf.matmul(query, key, transpose_b=True)
        scores = scores / tf.sqrt(tf.cast(self.d_model, tf.float32))
        
        # Softmax
        weights = tf.nn.softmax(scores, axis=-1)
        
        # Apply to values
        output = tf.matmul(weights, value)
        return output

# Use it
attention_layer = SelfAttention(d_model=64)
```

## Multi-Head Attention

Multiple attention layers in parallel:

```python
class MultiHeadAttention(tf.keras.layers.Layer):
    def __init__(self, d_model, num_heads):
        super().__init__()
        self.num_heads = num_heads
        self.d_model = d_model
        self.depth = d_model // num_heads
        
        self.query_dense = tf.keras.layers.Dense(d_model)
        self.key_dense = tf.keras.layers.Dense(d_model)
        self.value_dense = tf.keras.layers.Dense(d_model)
        self.dense = tf.keras.layers.Dense(d_model)
    
    def split_heads(self, x, batch_size):
        x = tf.reshape(x, (batch_size, -1, self.num_heads, self.depth))
        return tf.transpose(x, perm=[0, 2, 1, 3])
    
    def call(self, inputs):
        batch_size = tf.shape(inputs)[0]
        
        # Linear projections and split heads
        query = self.split_heads(self.query_dense(inputs), batch_size)
        key = self.split_heads(self.key_dense(inputs), batch_size)
        value = self.split_heads(self.value_dense(inputs), batch_size)
        
        # Scaled dot-product attention
        scores = tf.matmul(query, key, transpose_b=True)
        scores = scores / tf.sqrt(tf.cast(self.depth, tf.float32))
        weights = tf.nn.softmax(scores, axis=-1)
        output = tf.matmul(weights, value)
        
        # Concatenate heads
        output = tf.transpose(output, perm=[0, 2, 1, 3])
        output = tf.reshape(output, (batch_size, -1, self.d_model))
        
        # Final linear projection
        return self.dense(output)
```

## Applications

- Transformers (BERT, GPT)
- Machine translation
- Image captioning
- Speech recognition

## Remember

- Attention helps focus on important parts
- Self-attention relates different positions
- Multi-head attention captures different relationships
- Core of modern NLP models

The architecture behind modern NLP.

## What are Transformers?

Neural network architecture using only attention mechanisms.

**Revolutionary**: No RNNs or CNNs needed!

## Transformer Components

**Encoder**: Processes input sequence
**Decoder**: Generates output sequence
**Attention**: Focus mechanism
**Positional Encoding**: Add position information

## Simple Transformer

```python
import tensorflow as tf

class TransformerBlock(tf.keras.layers.Layer):
    def __init__(self, d_model, num_heads, ff_dim, dropout=0.1):
        super().__init__()
        self.attention = tf.keras.layers.MultiHeadAttention(
            num_heads=num_heads,
            key_dim=d_model
        )
        self.ffn = tf.keras.Sequential([
            tf.keras.layers.Dense(ff_dim, activation='relu'),
            tf.keras.layers.Dense(d_model)
        ])
        self.layernorm1 = tf.keras.layers.LayerNormalization(epsilon=1e-6)
        self.layernorm2 = tf.keras.layers.LayerNormalization(epsilon=1e-6)
        self.dropout1 = tf.keras.layers.Dropout(dropout)
        self.dropout2 = tf.keras.layers.Dropout(dropout)
    
    def call(self, inputs, training=False):
        # Multi-head attention
        attn_output = self.attention(inputs, inputs)
        attn_output = self.dropout1(attn_output, training=training)
        out1 = self.layernorm1(inputs + attn_output)
        
        # Feed forward
        ffn_output = self.ffn(out1)
        ffn_output = self.dropout2(ffn_output, training=training)
        return self.layernorm2(out1 + ffn_output)
```

## Positional Encoding

Add position information:

```python
def positional_encoding(length, depth):
    positions = np.arange(length)[:, np.newaxis]
    depths = np.arange(depth)[np.newaxis, :] / depth
    
    angle_rates = 1 / (10000 ** depths)
    angle_rads = positions * angle_rates
    
    pos_encoding = np.concatenate([
        np.sin(angle_rads[:, 0::2]),
        np.cos(angle_rads[:, 1::2])
    ], axis=-1)
    
    return tf.cast(pos_encoding, dtype=tf.float32)

# Add to embeddings
embeddings = embeddings + positional_encoding(seq_length, d_model)
```

## Full Transformer Model

```python
class Transformer(tf.keras.Model):
    def __init__(self, num_layers, d_model, num_heads, ff_dim, 
                 vocab_size, max_length, dropout=0.1):
        super().__init__()
        
        self.embedding = tf.keras.layers.Embedding(vocab_size, d_model)
        self.pos_encoding = positional_encoding(max_length, d_model)
        
        self.encoder_layers = [
            TransformerBlock(d_model, num_heads, ff_dim, dropout)
            for _ in range(num_layers)
        ]
        
        self.dropout = tf.keras.layers.Dropout(dropout)
        self.final_layer = tf.keras.layers.Dense(vocab_size)
    
    def call(self, inputs, training=False):
        # Embedding and position encoding
        x = self.embedding(inputs)
        x *= tf.sqrt(tf.cast(self.d_model, tf.float32))
        x += self.pos_encoding[:tf.shape(inputs)[1], :]
        x = self.dropout(x, training=training)
        
        # Encoder layers
        for encoder_layer in self.encoder_layers:
            x = encoder_layer(x, training=training)
        
        # Output
        return self.final_layer(x)

# Create model
model = Transformer(
    num_layers=4,
    d_model=128,
    num_heads=8,
    ff_dim=512,
    vocab_size=10000,
    max_length=100
)
```

## Training Transformer

```python
# Compile
model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

# Train
model.fit(
    train_dataset,
    epochs=10,
    validation_data=val_dataset
)
```

## Famous Transformers

**BERT**: Bidirectional encoder
**GPT**: Generative pre-training
**T5**: Text-to-text transfer
**RoBERTa**: Optimized BERT
**XLNet**: Permutation language modeling

## Applications

- Text generation
- Translation
- Question answering
- Text classification
- Summarization

## Remember

- Transformers use only attention
- Process sequences in parallel (fast!)
- Need positional encoding
- Foundation of modern NLP

Transformers Architecture

Powerful pre-trained language understanding.

## What is BERT?

Bidirectional Encoder Representations from Transformers.

**Key**: Reads text both ways (left-to-right and right-to-left)!

## Why BERT?

**Pre-trained**: Learned from millions of books
**Context-aware**: Understands word meaning from context
**Fine-tunable**: Adapt to your specific task

## Using BERT

```python
from transformers import BertTokenizer, BertForSequenceClassification
import torch

# Load pre-trained model
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForSequenceClassification.from_pretrained(
    'bert-base-uncased',
    num_labels=2  # Binary classification
)

# Tokenize text
text = "This movie is fantastic!"
inputs = tokenizer(
    text,
    padding=True,
    truncation=True,
    return_tensors='pt'
)

# Get prediction
with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
    print(predictions)
```

## Fine-tuning BERT

```python
from transformers import BertForSequenceClassification, Trainer, TrainingArguments
from datasets import load_dataset

# Load dataset
dataset = load_dataset('imdb')

# Tokenize dataset
def tokenize_function(examples):
    return tokenizer(
        examples['text'],
        padding='max_length',
        truncation=True
    )

tokenized_datasets = dataset.map(tokenize_function, batched=True)

# Training arguments
training_args = TrainingArguments(
    output_dir='./results',
    num_train_epochs=3,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    warmup_steps=500,
    weight_decay=0.01,
    logging_dir='./logs',
)

# Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_datasets['train'],
    eval_dataset=tokenized_datasets['test']
)

# Train
trainer.train()
```

## BERT for Different Tasks

### Text Classification
```python
from transformers import pipeline

classifier = pipeline('sentiment-analysis')
result = classifier("I love this product!")
print(result)
```

### Named Entity Recognition
```python
ner = pipeline('ner')
result = ner("John Smith lives in New York")
print(result)
```

### Question Answering
```python
qa = pipeline('question-answering')
result = qa(
    question="Where does John live?",
    context="John Smith lives in New York and works at Google."
)
print(result)
```

## BERT Variants

**RoBERTa**: Robustly optimized BERT
**ALBERT**: Lighter BERT
**DistilBERT**: Faster, smaller BERT
**ELECTRA**: More efficient pre-training
**DeBERTa**: Improved attention mechanism

## Best Practices

**Batch size**: Use smaller batches (8-16)
**Learning rate**: Start with 2e-5 to 5e-5
**Epochs**: Usually 2-4 epochs enough
**Max length**: 512 tokens maximum

## Remember

- BERT is pre-trained on massive text
- Fine-tune for your specific task
- Very powerful for NLP tasks
- Consider lighter versions for speed

BERT and Pre-trained Language Models

Generate human-like text with GPT.

## What is GPT?

Generative Pre-trained Transformer.

**Goal**: Predict next word in sequence!

## How GPT Works

Trained to complete text:
- Input: "The weather is"
- Output: "nice today"

## Using GPT-2

```python
from transformers import GPT2LMHeadModel, GPT2Tokenizer

# Load model
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2LMHeadModel.from_pretrained('gpt2')

# Generate text
def generate_text(prompt, max_length=50):
    inputs = tokenizer.encode(prompt, return_tensors='pt')
    
    outputs = model.generate(
        inputs,
        max_length=max_length,
        num_return_sequences=1,
        no_repeat_ngram_size=2,
        temperature=0.7,
        top_k=50,
        top_p=0.95
    )
    
    text = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return text

# Test
prompt = "In the year 2050,"
result = generate_text(prompt)
print(result)
```

## Control Generation

```python
# More creative (higher temperature)
outputs = model.generate(
    inputs,
    max_length=100,
    temperature=1.0,  # More random
    do_sample=True
)

# More focused (lower temperature)
outputs = model.generate(
    inputs,
    max_length=100,
    temperature=0.3,  # More deterministic
    do_sample=True
)

# Top-k sampling
outputs = model.generate(
    inputs,
    max_length=100,
    top_k=40,  # Consider top 40 words
    do_sample=True
)

# Top-p (nucleus) sampling
outputs = model.generate(
    inputs,
    max_length=100,
    top_p=0.9,  # Cumulative probability 0.9
    do_sample=True
)
```

## Fine-tuning GPT-2

```python
from transformers import TextDataset, DataCollatorForLanguageModeling
from transformers import Trainer, TrainingArguments

# Prepare dataset
def load_dataset(file_path, tokenizer):
    dataset = TextDataset(
        tokenizer=tokenizer,
        file_path=file_path,
        block_size=128
    )
    return dataset

train_dataset = load_dataset('train.txt', tokenizer)

data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer,
    mlm=False  # GPT uses causal LM, not masked LM
)

# Training
training_args = TrainingArguments(
    output_dir='./gpt2-finetuned',
    overwrite_output_dir=True,
    num_train_epochs=3,
    per_device_train_batch_size=4,
    save_steps=500,
    save_total_limit=2,
)

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=train_dataset,
)

trainer.train()
```

## Practical Applications

### Story Generation
```python
prompt = "Once upon a time in Boston, there was"
story = generate_text(prompt, max_length=200)
print(story)
```

### Code Generation
```python
prompt = "def calculate_fibonacci(n):"
code = generate_text(prompt, max_length=150)
print(code)
```

### Chatbot
```python
def chat(message, history=""):
    prompt = f"{history}
User: {message}
Bot:"
    response = generate_text(prompt, max_length=100)
    return response.split("Bot:")[-1].split("
")[0]

# Use it
response = chat("Hello! How are you?")
print(response)
```

## GPT Versions

**GPT-2**: 1.5B parameters, open source
**GPT-3**: 175B parameters, API access
**GPT-3.5**: ChatGPT base model
**GPT-4**: Most advanced, multimodal

## Generation Parameters

**Temperature**: Randomness (0.7-1.0 good)
**Top-k**: Limit to k most likely words
**Top-p**: Nucleus sampling threshold
**Max length**: Maximum tokens to generate
**Repetition penalty**: Avoid repeating text

## Remember

- GPT generates text token by token
- Temperature controls creativity
- Fine-tune for domain-specific tasks
- Larger models = better quality

GPT and Language Generation

Find and label objects in images.

## What is Object Detection?

Find where objects are and what they are.

**Output**: Bounding boxes + class labels!

## YOLO (You Only Look Once)

Fast, real-time object detection.

```python
# Install
# pip install ultralytics

from ultralytics import YOLO
import cv2

# Load pre-trained model
model = YOLO('yolov8n.pt')  # nano model (fastest)

# Detect objects in image
results = model('image.jpg')

# Show results
for result in results:
    boxes = result.boxes
    for box in boxes:
        # Get coordinates
        x1, y1, x2, y2 = box.xyxy[0]
        confidence = box.conf[0]
        class_id = box.cls[0]
        class_name = model.names[int(class_id)]
        
        print(f"{class_name}: {confidence:.2f}")
        print(f"Box: ({x1:.0f}, {y1:.0f}) to ({x2:.0f}, {y2:.0f})")

# Display results
result.show()
```

## Real-time Video Detection

```python
import cv2

# Open webcam
cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    if not ret:
        break
    
    # Detect objects
    results = model(frame)
    
    # Draw boxes
    annotated_frame = results[0].plot()
    
    # Display
    cv2.imshow('YOLO Detection', annotated_frame)
    
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()
```

## Custom Object Detection

```python
# Train on custom dataset
model = YOLO('yolov8n.pt')

# Train
results = model.train(
    data='custom_data.yaml',  # Dataset config
    epochs=100,
    imgsz=640,
    batch=16,
    device=0  # GPU
)

# Validate
metrics = model.val()

# Use trained model
model = YOLO('runs/detect/train/weights/best.pt')
results = model('test_image.jpg')
```

## Dataset Format (YOLO)

Create custom_data.yaml file:
```yaml
train: /path/to/train/images
val: /path/to/val/images

nc: 3  # number of classes
names: ['person', 'car', 'bicycle']
```

Label format (one .txt file per image):
```
# class_id center_x center_y width height (normalized 0-1)
0 0.5 0.5 0.3 0.4
1 0.7 0.3 0.2 0.3
```

## Faster R-CNN

More accurate but slower:

```python
import torchvision
from torchvision.models.detection import fasterrcnn_resnet50_fpn

# Load model
model = fasterrcnn_resnet50_fpn(pretrained=True)
model.eval()

# Prepare image
import torch
from PIL import Image
from torchvision import transforms

image = Image.open('image.jpg')
transform = transforms.Compose([transforms.ToTensor()])
image_tensor = transform(image).unsqueeze(0)

# Detect
with torch.no_grad():
    predictions = model(image_tensor)

# Process results
boxes = predictions[0]['boxes'].cpu().numpy()
labels = predictions[0]['labels'].cpu().numpy()
scores = predictions[0]['scores'].cpu().numpy()

for box, label, score in zip(boxes, labels, scores):
    if score > 0.5:  # Confidence threshold
        print(f"Label: {label}, Score: {score:.2f}")
        print(f"Box: {box}")
```

## Comparison

**YOLO**:
- Very fast (real-time)
- Good for video
- Slightly less accurate

**Faster R-CNN**:
- More accurate
- Slower
- Better for images

**EfficientDet**:
- Balance of speed and accuracy
- Efficient architecture

## Applications

- Self-driving cars
- Security cameras
- Retail analytics
- Sports tracking
- Wildlife monitoring

## Remember

- YOLO is fast, R-CNN is accurate
- Need labeled bounding box data
- More data = better detection
- Use pre-trained models when possible

Object Detection (YOLO, R-CNN)

Label every pixel in an image.

## What is Image Segmentation?

Classify each pixel (not just boxes).

**Types**:
- Semantic: Label categories
- Instance: Separate individual objects

## U-Net for Segmentation

```python
import tensorflow as tf

def unet_model(input_size=(256, 256, 3)):
    inputs = tf.keras.Input(input_size)
    
    # Encoder (downsampling)
    c1 = tf.keras.layers.Conv2D(64, 3, activation='relu', padding='same')(inputs)
    c1 = tf.keras.layers.Conv2D(64, 3, activation='relu', padding='same')(c1)
    p1 = tf.keras.layers.MaxPooling2D()(c1)
    
    c2 = tf.keras.layers.Conv2D(128, 3, activation='relu', padding='same')(p1)
    c2 = tf.keras.layers.Conv2D(128, 3, activation='relu', padding='same')(c2)
    p2 = tf.keras.layers.MaxPooling2D()(c2)
    
    # Bottleneck
    c3 = tf.keras.layers.Conv2D(256, 3, activation='relu', padding='same')(p2)
    c3 = tf.keras.layers.Conv2D(256, 3, activation='relu', padding='same')(c3)
    
    # Decoder (upsampling)
    u1 = tf.keras.layers.Conv2DTranspose(128, 2, strides=2, padding='same')(c3)
    u1 = tf.keras.layers.concatenate([u1, c2])
    c4 = tf.keras.layers.Conv2D(128, 3, activation='relu', padding='same')(u1)
    c4 = tf.keras.layers.Conv2D(128, 3, activation='relu', padding='same')(c4)
    
    u2 = tf.keras.layers.Conv2DTranspose(64, 2, strides=2, padding='same')(c4)
    u2 = tf.keras.layers.concatenate([u2, c1])
    c5 = tf.keras.layers.Conv2D(64, 3, activation='relu', padding='same')(u2)
    c5 = tf.keras.layers.Conv2D(64, 3, activation='relu', padding='same')(c5)
    
    # Output
    outputs = tf.keras.layers.Conv2D(1, 1, activation='sigmoid')(c5)
    
    model = tf.keras.Model(inputs=[inputs], outputs=[outputs])
    return model

model = unet_model()
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
```

## Training Segmentation Model

```python
import numpy as np
from tensorflow.keras.preprocessing.image import load_img, img_to_array

# Load data
def load_images_and_masks(image_dir, mask_dir, img_size=(256, 256)):
    images = []
    masks = []
    
    for img_file in os.listdir(image_dir):
        # Load image
        img = load_img(f"{image_dir}/{img_file}", target_size=img_size)
        img = img_to_array(img) / 255.0
        images.append(img)
        
        # Load mask
        mask = load_img(f"{mask_dir}/{img_file}", target_size=img_size, color_mode='grayscale')
        mask = img_to_array(mask) / 255.0
        masks.append(mask)
    
    return np.array(images), np.array(masks)

X_train, y_train = load_images_and_masks('train/images', 'train/masks')

# Train
history = model.fit(
    X_train, y_train,
    batch_size=16,
    epochs=50,
    validation_split=0.2
)
```

## Prediction and Visualization

```python
import matplotlib.pyplot as plt

# Predict
test_img = load_img('test.jpg', target_size=(256, 256))
test_img_array = img_to_array(test_img) / 255.0
test_img_array = np.expand_dims(test_img_array, axis=0)

prediction = model.predict(test_img_array)[0]

# Visualize
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
axes[0].imshow(test_img)
axes[0].set_title('Original Image')
axes[1].imshow(prediction.squeeze(), cmap='gray')
axes[1].set_title('Predicted Mask')
axes[2].imshow(test_img)
axes[2].imshow(prediction.squeeze(), alpha=0.5, cmap='jet')
axes[2].set_title('Overlay')
plt.show()
```

## DeepLab v3+

State-of-the-art segmentation:

```python
# Install
# pip install segmentation-models-pytorch

import segmentation_models_pytorch as smp

# Load pre-trained model
model = smp.DeepLabV3Plus(
    encoder_name='resnet50',
    encoder_weights='imagenet',
    classes=1,
    activation='sigmoid'
)

# Use with PyTorch
import torch
model.eval()

image = torch.randn(1, 3, 256, 256)
mask = model(image)
print(mask.shape)  # (1, 1, 256, 256)
```

## Mask R-CNN (Instance Segmentation)

```python
# Install
# pip install detectron2

from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer

# Setup
cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")

predictor = DefaultPredictor(cfg)

# Predict
import cv2
image = cv2.imread('image.jpg')
outputs = predictor(image)

# Visualize
v = Visualizer(image[:, :, ::-1])
out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
cv2.imshow('Segmentation', out.get_image()[:, :, ::-1])
cv2.waitKey(0)
```

## Applications

- Medical imaging (tumor detection)
- Autonomous driving (road segmentation)
- Satellite imagery
- Photo editing
- Agriculture (crop monitoring)

## Remember

- Semantic segmentation: Label categories
- Instance segmentation: Separate objects
- U-Net popular for medical images
- Needs pixel-wise labeled data

Image Segmentation

Turn spoken words into text.

## What is Speech Recognition?

Convert audio into written text.

**Use cases**: Voice assistants, transcription, accessibility!

## Using Speech Recognition

```python
# Install
# pip install SpeechRecognition

import speech_recognition as sr

# Create recognizer
recognizer = sr.Recognizer()

# From microphone
with sr.Microphone() as source:
    print("Speak something...")
    audio = recognizer.listen(source)
    
    try:
        text = recognizer.recognize_google(audio)
        print(f"You said: {text}")
    except sr.UnknownValueError:
        print("Could not understand audio")
    except sr.RequestError as e:
        print(f"Error: {e}")
```

## From Audio File

```python
# Load audio file
with sr.AudioFile('audio.wav') as source:
    audio = recognizer.record(source)
    text = recognizer.recognize_google(audio)
    print(text)
```

## Whisper (OpenAI)

State-of-the-art speech recognition:

```python
# Install
# pip install openai-whisper

import whisper

# Load model
model = whisper.load_model("base")  # tiny, base, small, medium, large

# Transcribe
result = model.transcribe("audio.mp3")
print(result["text"])

# With timestamps
segments = result["segments"]
for segment in segments:
    start = segment["start"]
    end = segment["end"]
    text = segment["text"]
    print(f"[{start:.2f}s - {end:.2f}s]: {text}")
```

## Real-time Transcription

```python
import pyaudio
import wave

# Record audio
def record_audio(filename, duration=5):
    CHUNK = 1024
    FORMAT = pyaudio.paInt16
    CHANNELS = 1
    RATE = 16000
    
    p = pyaudio.PyAudio()
    
    stream = p.open(
        format=FORMAT,
        channels=CHANNELS,
        rate=RATE,
        input=True,
        frames_per_buffer=CHUNK
    )
    
    print("Recording...")
    frames = []
    
    for _ in range(0, int(RATE / CHUNK * duration)):
        data = stream.read(CHUNK)
        frames.append(data)
    
    print("Done recording")
    
    stream.stop_stream()
    stream.close()
    p.terminate()
    
    # Save
    wf = wave.open(filename, 'wb')
    wf.setnchannels(CHANNELS)
    wf.setsampwidth(p.get_sample_size(FORMAT))
    wf.setframerate(RATE)
    wf.writeframes(b''.join(frames))
    wf.close()

# Use it
record_audio("recording.wav", duration=5)
result = model.transcribe("recording.wav")
print(result["text"])
```

## Different Languages

```python
# Specify language
result = model.transcribe("audio.mp3", language="es")  # Spanish
print(result["text"])

# Auto-detect language
result = model.transcribe("audio.mp3")
print(f"Detected language: {result['language']}")
```

## Voice Commands

```python
def listen_for_command():
    with sr.Microphone() as source:
        print("Listening...")
        audio = recognizer.listen(source)
        
        try:
            command = recognizer.recognize_google(audio).lower()
            return command
        except:
            return None

# Command handler
while True:
    command = listen_for_command()
    
    if command:
        if "hello" in command:
            print("Hello there!")
        elif "time" in command:
            from datetime import datetime
            now = datetime.now()
            print(f"Current time: {now.strftime('%H:%M')}")
        elif "stop" in command:
            print("Goodbye!")
            break
        else:
            print(f"You said: {command}")
```

## Speaker Diarization

Identify who is speaking:

```python
# Install
# pip install pyannote.audio

from pyannote.audio import Pipeline

pipeline = Pipeline.from_pretrained("pyannote/speaker-diarization")

# Diarize
diarization = pipeline("audio.wav")

# Print results
for turn, _, speaker in diarization.itertracks(yield_label=True):
    print(f"Speaker {speaker}: {turn.start:.1f}s - {turn.end:.1f}s")
```

## Best Practices

**Good audio**: Minimize background noise
**Sample rate**: Use 16kHz for most models
**Segment long audio**: Break into smaller chunks
**Post-processing**: Clean up text output

## Remember

- Whisper is most accurate
- Works with multiple languages
- Needs clear audio for best results
- Can run locally (no API needed)

Speech Recognition with AI

Build responsible AI systems.

## What is AI Bias?

When AI makes unfair decisions based on protected attributes.

**Problem**: AI learns from biased data!

## Types of Bias

**Data bias**: Training data not representative
**Algorithm bias**: Model design favors certain groups
**Evaluation bias**: Test data doesn't reflect real world
**Deployment bias**: AI used inappropriately

## Detecting Bias

```python
# Install
# pip install fairlearn

from fairlearn.metrics import MetricFrame
from sklearn.metrics import accuracy_score
import pandas as pd

# Assume you have predictions and sensitive features
y_true = [0, 1, 1, 0, 1, 0]
y_pred = [0, 1, 0, 0, 1, 1]
gender = ['M', 'F', 'F', 'M', 'F', 'M']

# Calculate metrics by group
metric_frame = MetricFrame(
    metrics=accuracy_score,
    y_true=y_true,
    y_pred=y_pred,
    sensitive_features=gender
)

print("Overall accuracy:", metric_frame.overall)
print("By group:")
print(metric_frame.by_group)
```

## Fairness Metrics

```python
from fairlearn.metrics import (
    demographic_parity_difference,
    equalized_odds_difference
)

# Demographic parity
dp = demographic_parity_difference(
    y_true, y_pred,
    sensitive_features=gender
)
print(f"Demographic parity difference: {dp:.3f}")

# Equalized odds
eo = equalized_odds_difference(
    y_true, y_pred,
    sensitive_features=gender
)
print(f"Equalized odds difference: {eo:.3f}")
```

## Mitigating Bias

### 1. Better Data Collection
```python
# Check data distribution
df = pd.DataFrame({'gender': gender, 'outcome': y_true})
print(df.groupby('gender')['outcome'].value_counts())

# Balance dataset
from sklearn.utils import resample

# Oversample minority group
minority = df[df['gender'] == 'F']
majority = df[df['gender'] == 'M']

minority_upsampled = resample(
    minority,
    replace=True,
    n_samples=len(majority),
    random_state=42
)

balanced_df = pd.concat([majority, minority_upsampled])
```

### 2. Fair Model Training
```python
from fairlearn.reductions import ExponentiatedGradient, DemographicParity
from sklearn.linear_model import LogisticRegression

# Fair classifier
model = LogisticRegression()
constraint = DemographicParity()

mitigator = ExponentiatedGradient(model, constraint)
mitigator.fit(X_train, y_train, sensitive_features=sensitive_train)

# Predict
y_pred = mitigator.predict(X_test)
```

### 3. Post-processing
```python
from fairlearn.postprocessing import ThresholdOptimizer

# Optimize thresholds per group
optimizer = ThresholdOptimizer(
    estimator=model,
    constraints="demographic_parity"
)

optimizer.fit(X_train, y_train, sensitive_features=sensitive_train)
y_pred_fair = optimizer.predict(X_test, sensitive_features=sensitive_test)
```

## Explainability

Make AI decisions transparent:

```python
# Install
# pip install shap

import shap

# Explain predictions
explainer = shap.Explainer(model, X_train)
shap_values = explainer(X_test)

# Visualize
shap.plots.waterfall(shap_values[0])
shap.plots.beeswarm(shap_values)
```

## Ethical Guidelines

**Transparency**: Explain how AI makes decisions
**Fairness**: Treat all groups equally
**Privacy**: Protect user data
**Accountability**: Take responsibility for AI actions
**Safety**: Ensure AI doesn't cause harm

## Fairness Trade-offs

**Accuracy vs Fairness**: More fair model may be less accurate
**Group fairness vs Individual fairness**: Can't always satisfy both
**Different fairness definitions**: May conflict with each other

## Best Practices

1. **Diverse teams**: Include different perspectives
2. **Regular audits**: Check for bias continuously
3. **User feedback**: Listen to affected communities
4. **Documentation**: Record decisions and trade-offs
5. **Testing**: Test on diverse populations

## Real-world Examples

**Hiring**: Don't discriminate by gender/race
**Lending**: Fair loan approval
**Criminal justice**: Unbiased risk assessment
**Healthcare**: Equal treatment recommendations

## Remember

- All AI systems have bias - minimize it!
- Test fairness on multiple groups
- Balance fairness with accuracy
- Be transparent about limitations
- Ethics is ongoing, not one-time

Take AI models from notebook to production.

## What is MLOps?

DevOps practices for machine learning.

**Goal**: Reliably deploy and maintain ML models!

## Model Deployment Pipeline

1. Train model
2. Version control
3. Test model
4. Package model
5. Deploy to production
6. Monitor performance

## Save and Load Models

```python
# Scikit-learn
import joblib

# Save
joblib.dump(model, 'model.pkl')

# Load
model = joblib.load('model.pkl')

# TensorFlow/Keras
model.save('model.h5')
loaded_model = tf.keras.models.load_model('model.h5')

# PyTorch
torch.save(model.state_dict(), 'model.pth')
model.load_state_dict(torch.load('model.pth'))
```

## Create API with Flask

```python
from flask import Flask, request, jsonify
import joblib
import numpy as np

app = Flask(__name__)

# Load model
model = joblib.load('model.pkl')

@app.route('/predict', methods=['POST'])
def predict():
    # Get data from request
    data = request.get_json()
    features = np.array(data['features']).reshape(1, -1)
    
    # Predict
    prediction = model.predict(features)
    probability = model.predict_proba(features)
    
    # Return response
    return jsonify({
        'prediction': int(prediction[0]),
        'probability': float(probability[0][1])
    })

@app.route('/health', methods=['GET'])
def health():
    return jsonify({'status': 'healthy'})

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)
```

## FastAPI (Faster Alternative)

```python
from fastapi import FastAPI
from pydantic import BaseModel
import joblib

app = FastAPI()
model = joblib.load('model.pkl')

class PredictionRequest(BaseModel):
    features: list

@app.post("/predict")
async def predict(request: PredictionRequest):
    features = np.array(request.features).reshape(1, -1)
    prediction = model.predict(features)
    return {"prediction": int(prediction[0])}

# Run with: uvicorn main:app --reload
```

## Docker Containerization

Create Dockerfile:
```dockerfile
FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .

CMD ["python", "app.py"]
```

Build and run:
```bash
docker build -t ml-model .
docker run -p 5000:5000 ml-model
```

## Model Versioning with MLflow

```python
# Install
# pip install mlflow

import mlflow
import mlflow.sklearn

# Start experiment
mlflow.set_experiment("my-experiment")

with mlflow.start_run():
    # Train model
    model.fit(X_train, y_train)
    
    # Log parameters
    mlflow.log_param("max_depth", 5)
    mlflow.log_param("n_estimators", 100)
    
    # Log metrics
    accuracy = model.score(X_test, y_test)
    mlflow.log_metric("accuracy", accuracy)
    
    # Log model
    mlflow.sklearn.log_model(model, "model")

# View experiments: mlflow ui
```

## Load Model from MLflow

```python
# Load by run ID
run_id = "abc123"
model = mlflow.sklearn.load_model(f"runs:/{run_id}/model")

# Or use model registry
model = mlflow.sklearn.load_model("models:/my-model/production")
```

## Monitoring in Production

```python
import logging
from datetime import datetime

# Setup logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

@app.route('/predict', methods=['POST'])
def predict():
    start_time = datetime.now()
    
    data = request.get_json()
    features = np.array(data['features']).reshape(1, -1)
    
    # Predict
    prediction = model.predict(features)
    
    # Calculate latency
    latency = (datetime.now() - start_time).total_seconds()
    
    # Log
    logger.info(f"Prediction: {prediction[0]}, Latency: {latency:.3f}s")
    
    return jsonify({'prediction': int(prediction[0])})
```

## Data Drift Detection

```python
# Install
# pip install evidently

from evidently.dashboard import Dashboard
from evidently.tabs import DataDriftTab

# Create drift report
dashboard = Dashboard(tabs=[DataDriftTab()])
dashboard.calculate(reference_data, production_data)
dashboard.save("drift_report.html")
```

## A/B Testing

```python
import random

# Two models
model_a = joblib.load('model_a.pkl')
model_b = joblib.load('model_b.pkl')

@app.route('/predict', methods=['POST'])
def predict():
    data = request.get_json()
    features = np.array(data['features']).reshape(1, -1)
    
    # Randomly choose model (50/50 split)
    if random.random() < 0.5:
        prediction = model_a.predict(features)
        model_version = 'A'
    else:
        prediction = model_b.predict(features)
        model_version = 'B'
    
    # Log which model was used
    logger.info(f"Model {model_version} used")
    
    return jsonify({
        'prediction': int(prediction[0]),
        'model_version': model_version
    })
```

## Cloud Deployment

### AWS SageMaker
```python
import sagemaker
from sagemaker.sklearn import SKLearnModel

# Deploy to SageMaker
sklearn_model = SKLearnModel(
    model_data='s3://bucket/model.tar.gz',
    role=role,
    entry_point='inference.py',
    framework_version='0.23-1'
)

predictor = sklearn_model.deploy(
    initial_instance_count=1,
    instance_type='ml.m5.large'
)

# Predict
result = predictor.predict(data)
```

## Best Practices

1. **Version everything**: Code, data, models
2. **Automate testing**: Unit tests, integration tests
3. **Monitor continuously**: Accuracy, latency, errors
4. **Gradual rollout**: Canary deployment, A/B testing
5. **Easy rollback**: Keep old model versions
6. **Document**: API docs, model cards

## Remember

- MLOps is essential for production AI
- Use Docker for consistency
- Monitor model performance over time
- Plan for model retraining
- Security: Validate inputs, use HTTPS

Deploy and manage AI models in production.

MLOps and Model Deployment

Forecast future trends with AI.

## What is Time Series?

Data points ordered by time.

**Examples**: Stock prices, weather, sales!

## Basic Time Series with ARIMA

```python
# Install
# pip install statsmodels

import pandas as pd
from statsmodels.tsa.arima.model import ARIMA
import matplotlib.pyplot as plt

# Load data
df = pd.read_csv('sales.csv', parse_dates=['date'], index_col='date')

# Train ARIMA model
model = ARIMA(df['sales'], order=(1, 1, 1))
fitted_model = model.fit()

# Forecast
forecast = fitted_model.forecast(steps=30)  # 30 days ahead

# Plot
plt.figure(figsize=(12, 6))
plt.plot(df['sales'], label='Historical')
plt.plot(forecast, label='Forecast', color='red')
plt.legend()
plt.show()
```

## LSTM for Time Series

```python
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, Dropout

# Prepare data
def create_sequences(data, seq_length):
    X, y = [], []
    for i in range(len(data) - seq_length):
        X.append(data[i:i+seq_length])
        y.append(data[i+seq_length])
    return np.array(X), np.array(y)

# Normalize data
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
scaled_data = scaler.fit_transform(df['sales'].values.reshape(-1, 1))

# Create sequences
seq_length = 30
X, y = create_sequences(scaled_data, seq_length)

# Reshape for LSTM
X = X.reshape((X.shape[0], X.shape[1], 1))

# Build model
model = Sequential([
    LSTM(50, activation='relu', return_sequences=True, input_shape=(seq_length, 1)),
    Dropout(0.2),
    LSTM(50, activation='relu'),
    Dropout(0.2),
    Dense(1)
])

model.compile(optimizer='adam', loss='mse')

# Train
history = model.fit(X, y, epochs=50, batch_size=32, validation_split=0.2)
```

## Make Predictions

```python
# Predict next 30 days
last_sequence = scaled_data[-seq_length:]
predictions = []

for _ in range(30):
    # Reshape for prediction
    input_seq = last_sequence.reshape((1, seq_length, 1))
    
    # Predict
    pred = model.predict(input_seq, verbose=0)
    predictions.append(pred[0, 0])
    
    # Update sequence
    last_sequence = np.append(last_sequence[1:], pred)

# Inverse transform
predictions = scaler.inverse_transform(np.array(predictions).reshape(-1, 1))

# Plot
plt.figure(figsize=(12, 6))
plt.plot(df['sales'].values, label='Historical')
plt.plot(range(len(df), len(df) + 30), predictions, label='Forecast', color='red')
plt.legend()
plt.show()
```

## Prophet (Facebook)

Easy, powerful forecasting:

```python
# Install
# pip install prophet

from prophet import Prophet

# Prepare data (needs 'ds' and 'y' columns)
df_prophet = df.reset_index()
df_prophet.columns = ['ds', 'y']

# Create and fit model
model = Prophet(
    yearly_seasonality=True,
    weekly_seasonality=True,
    daily_seasonality=False
)
model.fit(df_prophet)

# Make future dataframe
future = model.make_future_dataframe(periods=30)

# Forecast
forecast = model.predict(future)

# Plot
model.plot(forecast)
model.plot_components(forecast)
plt.show()
```

## Handle Seasonality

```python
# Custom seasonality
model = Prophet()
model.add_seasonality(name='monthly', period=30.5, fourier_order=5)

# Holidays
from prophet.make_holidays import make_holidays_df
holidays = make_holidays_df(
    year_list=[2023, 2024],
    country='US'
)
model = Prophet(holidays=holidays)
model.fit(df_prophet)
```

## Evaluate Forecast

```python
from sklearn.metrics import mean_absolute_error, mean_squared_error

# Split data
train_size = int(len(df) * 0.8)
train, test = df[:train_size], df[train_size:]

# Train and predict
model = Prophet()
model.fit(train.reset_index().rename(columns={'date': 'ds', 'sales': 'y'}))

future = model.make_future_dataframe(periods=len(test))
forecast = model.predict(future)

# Get predictions for test period
predictions = forecast.iloc[-len(test):]['yhat'].values

# Calculate metrics
mae = mean_absolute_error(test['sales'], predictions)
rmse = np.sqrt(mean_squared_error(test['sales'], predictions))

print(f"MAE: {mae:.2f}")
print(f"RMSE: {rmse:.2f}")
```

## Multivariate Forecasting

```python
# Multiple features
from tensorflow.keras.layers import Input, Concatenate

# Prepare data with multiple features
def prepare_multivariate_data(df, features, seq_length):
    data = df[features].values
    scaler = MinMaxScaler()
    scaled_data = scaler.fit_transform(data)
    
    X, y = [], []
    for i in range(len(scaled_data) - seq_length):
        X.append(scaled_data[i:i+seq_length])
        y.append(scaled_data[i+seq_length, 0])  # Predict first feature
    
    return np.array(X), np.array(y), scaler

features = ['sales', 'temperature', 'day_of_week']
X, y, scaler = prepare_multivariate_data(df, features, seq_length=30)

# LSTM for multiple features
model = Sequential([
    LSTM(100, activation='relu', input_shape=(30, len(features))),
    Dropout(0.2),
    Dense(50, activation='relu'),
    Dense(1)
])

model.compile(optimizer='adam', loss='mse')
model.fit(X, y, epochs=50, batch_size=32)
```

## Applications

- Stock price prediction
- Sales forecasting
- Energy demand prediction
- Weather forecasting
- Website traffic prediction

## Remember

- Time series needs sequential data
- ARIMA for simple patterns
- LSTM for complex patterns
- Prophet for business forecasting
- Always validate on test set

AI for Time Series Forecasting

Complete guide to AI project success.

## Project Phases

1. **Problem definition**
2. **Data collection**
3. **Exploratory analysis**
4. **Model development**
5. **Evaluation**
6. **Deployment**
7. **Monitoring**

## Phase 1: Define Problem

Ask key questions:

**What problem are we solving?**
- Clear, specific goal
- Measurable success criteria

**Is AI the right solution?**
- Need enough data
- Problem must be learnable

**What's the business impact?**
- Cost savings
- Revenue increase
- User experience improvement

## Phase 2: Data Strategy

```python
import pandas as pd

# Data checklist
checklist = {
    'quantity': 'At least 1000 samples',
    'quality': 'Clean, accurate labels',
    'relevance': 'Matches real-world use',
    'balance': 'Equal class distribution',
    'privacy': 'Complies with regulations'
}

# Initial data exploration
df = pd.read_csv('data.csv')

print("Shape:", df.shape)
print("
Missing values:")
print(df.isnull().sum())
print("
Class distribution:")
print(df['target'].value_counts())
```

## Phase 3: EDA (Exploratory Data Analysis)

```python
import matplotlib.pyplot as plt
import seaborn as sns

# Distribution plots
fig, axes = plt.subplots(2, 2, figsize=(12, 10))

# Feature distributions
df['feature1'].hist(ax=axes[0, 0])
axes[0, 0].set_title('Feature 1 Distribution')

# Target distribution
df['target'].value_counts().plot(kind='bar', ax=axes[0, 1])
axes[0, 1].set_title('Target Distribution')

# Correlation heatmap
corr = df.corr()
sns.heatmap(corr, annot=True, ax=axes[1, 0])
axes[1, 0].set_title('Correlation Matrix')

# Box plot for outliers
df.boxplot(column='feature1', by='target', ax=axes[1, 1])
axes[1, 1].set_title('Feature by Target')

plt.tight_layout()
plt.show()
```

## Phase 4: Model Development

Start simple, then improve:

```python
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report

# Split data
X = df.drop('target', axis=1)
y = df['target']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Baseline model (simple)
baseline = LogisticRegression()
baseline.fit(X_train, y_train)
baseline_score = baseline.score(X_test, y_test)
print(f"Baseline accuracy: {baseline_score:.3f}")

# Advanced model
advanced = RandomForestClassifier(n_estimators=100)
advanced.fit(X_train, y_train)
advanced_score = advanced.score(X_test, y_test)
print(f"Advanced accuracy: {advanced_score:.3f}")

# Detailed metrics
y_pred = advanced.predict(X_test)
print(classification_report(y_test, y_pred))
```

## Phase 5: Proper Evaluation

```python
from sklearn.model_selection import cross_val_score

# Cross-validation
cv_scores = cross_val_score(advanced, X, y, cv=5)
print(f"CV scores: {cv_scores}")
print(f"Mean CV score: {cv_scores.mean():.3f} (+/- {cv_scores.std():.3f})")

# Test on unseen data
# Load fresh test set
test_df = pd.read_csv('new_test_data.csv')
X_new_test = test_df.drop('target', axis=1)
y_new_test = test_df['target']

final_score = advanced.score(X_new_test, y_new_test)
print(f"Final test score: {final_score:.3f}")
```

## Phase 6: Documentation

Create a model card:

```python
model_card = {
    'model_details': {
        'name': 'Customer Churn Predictor',
        'version': '1.0',
        'date': '2025-12-18',
        'type': 'Random Forest Classifier'
    },
    'intended_use': {
        'primary': 'Predict customer churn',
        'out_of_scope': 'Not for credit decisions'
    },
    'performance': {
        'accuracy': 0.87,
        'precision': 0.85,
        'recall': 0.89
    },
    'data': {
        'training_data': '50,000 customer records',
        'features': ['age', 'tenure', 'monthly_charges'],
        'date_range': '2023-2024'
    },
    'limitations': [
        'Works best for USA customers',
        'Accuracy drops for new customers (<3 months)',
        'Requires monthly updates'
    ],
    'ethical_considerations': [
        'No protected attributes used',
        'Regular bias audits',
        'Human review for high-risk decisions'
    ]
}

import json
with open('model_card.json', 'w') as f:
    json.dump(model_card, f, indent=2)
```

## Phase 7: Deployment Checklist

```python
deployment_checklist = {
    'model': {
        'saved': '✓ model.pkl',
        'tested': '✓ Unit tests pass',
        'versioned': '✓ v1.0 in MLflow'
    },
    'api': {
        'endpoint': '✓ /predict created',
        'docs': '✓ Swagger docs',
        'auth': '✓ API key required'
    },
    'infrastructure': {
        'docker': '✓ Dockerfile ready',
        'ci_cd': '✓ GitHub Actions',
        'monitoring': '✓ Prometheus + Grafana'
    },
    'security': {
        'input_validation': '✓ Implemented',
        'rate_limiting': '✓ 100 req/min',
        'https': '✓ SSL certificate'
    }
}
```

## Common Mistakes to Avoid

**Data leakage**: Test data in training
**Overfitting**: Model memorizes training data
**Wrong metric**: Accuracy for imbalanced data
**No baseline**: Nothing to compare against
**Ignoring deployment**: Model stuck in notebook
**No monitoring**: Performance degrades unseen

## Project Organization

```
project/
├── data/
│   ├── raw/
│   ├── processed/
│   └── external/
├── notebooks/
│   ├── 01_exploration.ipynb
│   ├── 02_modeling.ipynb
│   └── 03_evaluation.ipynb
├── src/
│   ├── data/
│   ├── features/
│   ├── models/
│   └── visualization/
├── tests/
├── models/
├── requirements.txt
└── README.md
```

## Success Metrics

Track these:

```python
metrics_to_track = {
    'model_metrics': {
        'accuracy': 0.87,
        'latency_ms': 45,
        'throughput': '1000 req/sec'
    },
    'business_metrics': {
        'revenue_impact': '$50,000/month',
        'cost_savings': '$20,000/month',
        'user_satisfaction': '4.5/5'
    },
    'operational_metrics': {
        'uptime': '99.9%',
        'errors': '0.1%',
        'data_drift': 'None detected'
    }
}
```

## Remember

- Start with clear problem definition
- Invest time in data quality
- Build baseline before complex models
- Document everything
- Monitor after deployment
- Iterate based on feedback
- Focus on business value!

Build successful AI projects from start to finish.

AI Project Best Practices

Deep learning is powering the AI revolution. From image recognition to language translation, deep neural networks are behind most modern AI breakthroughs. Let's understand how they work.

## What is Deep Learning?

Deep learning uses neural networks with multiple hidden layers (hence "deep"). Each layer learns increasingly complex features. The first layers might detect edges, middle layers detect shapes, and final layers detect objects or concepts.

## Why Deep Learning Works

Deep networks can automatically learn hierarchical representations from data. You don't need to manually engineer features - the network learns them. This is why deep learning excels at tasks like image and speech recognition.

## Common Architectures

Learn about feedforward networks, CNNs for images, RNNs for sequences, and transformers for language. Each architecture is designed for specific types of data and problems.

## Getting Started

I'll show you how to build your first deep learning model using popular frameworks. Don't worry, modern tools make it much easier than it sounds. You'll be building neural networks in no time.

Understand deep learning from the ground up. Learn how deep neural networks work, why they're powerful, and when to use them. Essential foundation for modern AI.

Deep Learning Basics: From Neurons to Networks

NLP is how computers understand human language. It's behind chatbots, translation apps, sentiment analysis, and modern language models. Understanding NLP opens up countless possibilities.

## What is NLP?

Natural Language Processing is the field of AI that focuses on the interaction between computers and human language. It involves teaching machines to read, understand, and generate human language in a valuable way.

## Common NLP Tasks

Text classification (spam detection, sentiment analysis), named entity recognition (finding names, places), machine translation, question answering, and text generation. Each task uses different techniques.

## Modern Approaches

Traditional NLP used rule-based systems and statistical methods. Modern NLP uses deep learning, especially transformers and language models. These approaches understand context and meaning much better.

## Practical Applications

I'll show you how to build practical NLP applications - sentiment analysis, text classification, and basic chatbots. These are skills you can use in real projects right away.

Learn NLP fundamentals - how AI understands and processes human language. From text classification to sentiment analysis, master the techniques behind chatbots and language models.

Natural Language Processing: Understanding Text with AI

Computer vision lets AI understand images and videos. It's behind facial recognition, self-driving cars, medical imaging, and so much more. If you want to work with visual data, you need to understand computer vision.

## What is Computer Vision?

Computer vision is the field of AI that trains computers to interpret and understand the visual world. Using digital images and videos, AI can identify and classify objects, detect faces, and even understand scenes.

## Key Tasks

Image classification (what's in the image), object detection (where are objects), image segmentation (pixel-level understanding), and image generation. Each requires different techniques and models.

## Convolutional Neural Networks

CNNs are the backbone of computer vision. They're designed to process pixel data and automatically learn features like edges, shapes, and objects. Understanding CNNs is essential for computer vision.

## Real-World Applications

I'll show you practical examples - building image classifiers, detecting objects in images, and working with pre-trained models. These skills are in high demand and have countless applications.

Master computer vision - how AI processes and understands images. Learn image classification, object detection, and image segmentation. Essential for building AI that works with visual data.

Computer Vision: Teaching AI to See

Training deep learning models from scratch takes time, data, and computational power. Transfer learning lets you use models already trained on huge datasets. You can get great results with much less effort.

## What is Transfer Learning?

Transfer learning is using a model trained on one task and adapting it for a different but related task. For example, using a model trained on ImageNet (millions of images) for your specific image classification problem.

## Why It Works

Pre-trained models have learned useful features from large datasets. You can use these features and fine-tune them for your specific problem. This is much faster and requires less data than training from scratch.

## How to Do It

Freeze the early layers (they've learned general features), replace the final layers with your own, and train only the new layers. Or fine-tune the entire model with a low learning rate. I'll show you both approaches.

## Practical Examples

I'll walk you through using pre-trained models for image classification, NLP tasks, and more. This is the standard approach in industry - why train from scratch when you can build on proven models?

Master transfer learning - use pre-trained models to solve your problems quickly. Instead of training from scratch, leverage models trained on millions of images. This is how professionals build AI in 2025.

AI Tutorial

What is AI?

AI vs Machine Learning

How AI Works

Types of Machine Learning

AI in Daily Life

Getting Started with AI

Understanding Data for AI

AI Models Explained

Neural Networks Basics

Training and Testing AI

AI Ethics and Bias

AI Tools and Platforms

Computer Vision Basics

Natural Language Processing Basics

AI Project Ideas for Beginners

Linear Regression

Classification Algorithms

Decision Trees

Random Forests

K-Nearest Neighbors

Support Vector Machines

Clustering with K-Means

Principal Component Analysis

Feature Engineering

Cross-Validation

Hyperparameter Tuning

Handling Imbalanced Data

Ensemble Learning

Convolutional Neural Networks

Recurrent Neural Networks

Natural Language Processing Tasks

Word Embeddings

Time Series Forecasting

Anomaly Detection

Recommendation Systems

Model Deployment Basics

Model Interpretability

A/B Testing for AI

Data Augmentation

MLOps Basics

Transfer Learning

Generative Adversarial Networks

Transformer Architecture

Reinforcement Learning Basics

AutoML and Neural Architecture Search

Attention Mechanisms

Graph Neural Networks

Meta-Learning (Learning to Learn)

Explainable AI (XAI)

Federated Learning

AutoML (Automated Machine Learning)

GANs (Generative Adversarial Networks)

Reinforcement Learning Basics

Transfer Learning in Deep Learning

Attention Mechanisms

Transformers Architecture

BERT and Pre-trained Language Models

GPT and Language Generation

Object Detection (YOLO, R-CNN)

Image Segmentation

Speech Recognition with AI

AI Ethics and Bias

MLOps and Model Deployment

AI for Time Series Forecasting

AI Project Best Practices

Deep Learning Basics: From Neurons to Networks

Natural Language Processing: Understanding Text with AI

Computer Vision: Teaching AI to See

Transfer Learning: Using Pre-trained Models

What is AI?

What is AI?

Simple Example

Where Do We Use AI?

Types of AI

How AI Learns

Remember