HTTP Requests Basics

Scraping starts with sending HTTP requests. HTTP is the language computers use to talk on the web. ## What is a request A request asks: “Please give me this page.” A response says: “Here is the HTML.” ## First request with Python ```python import requests url = "https://example.com" res = requests.get(url) print(res.status_code) print(res.text[:200]) ``` ## Status codes (important) - 200: OK - 404: Not found - 403: Forbidden - 500: Server error Always check status before scraping. ## Headers: act like a browser Some sites block unknown bots. ```python headers = { "User-Agent": "Mozilla/5.0" } res = requests.get(url, headers=headers) ``` ## Why headers matter They tell server: - who you are - what you accept - how to respond ## Graph: request cycle ```mermaid flowchart LR A[Python] --> B[HTTP GET] B --> C[Server] C --> D[Status + HTML] D --> A ``` ## Common mistakes - Ignoring status codes - No User-Agent - Sending too many requests fast ## Remember - requests library is your base tool - Always check status - Use headers to mimic browser