Query Params and Search Forms

Most websites allow searching or filtering:

• product by category
• jobs by city
• news by keyword

These filters are usually sent as query parameters.

Example URL:
site.com/search?q=python&page=2

Here:
q = python
page = 2

Why this matters

Instead of clicking forms manually, you can:

• change URL parameters
• loop over values
• collect large datasets

---

Inspect query params

Open browser DevTools → Network → search request.

You may see:
?q=laptop&min_price=500

---

Using params with requests

import requests

url = "https://example.com/search"
params = {
    "q": "python",
    "page": 1
}

res = requests.get(url, params=params)
print(res.url)  # full URL with params

---

Loop over search pages

for page in range(1, 4):
    params = {"q": "python", "page": page}
    res = requests.get("https://example.com/search", params=params)
    print("Scraping:", res.url)

---

Graph: query param flow

flowchart LR
  A[Search Form] --> B[URL Params]
  B --> C[requests.get()]
  C --> D[Filtered Results]

Remember

• Forms usually map to URL parameters
• You can build URLs programmatically
• This unlocks large datasets