HTTP Requests Basics

Scraping starts with sending HTTP requests.

HTTP is the language computers use to talk on the web.

What is a request

A request asks:
“Please give me this page.”

A response says:
“Here is the HTML.”

First request with Python

import requests

url = "https://example.com"
res = requests.get(url)

print(res.status_code)
print(res.text[:200])

Status codes (important)

• 200: OK
• 404: Not found
• 403: Forbidden
• 500: Server error

Always check status before scraping.

Headers: act like a browser

Some sites block unknown bots.

headers = {
  "User-Agent": "Mozilla/5.0"
}
res = requests.get(url, headers=headers)

Why headers matter

They tell server:

• who you are
• what you accept
• how to respond

Graph: request cycle

flowchart LR
  A[Python] --> B[HTTP GET]
  B --> C[Server]
  C --> D[Status + HTML]
  D --> A

Common mistakes

• Ignoring status codes
• No User-Agent
• Sending too many requests fast

Remember

• requests library is your base tool
• Always check status
• Use headers to mimic browser