JavaScript Rendered Pages

Some pages load data using JavaScript.

If you fetch them with requests: You may see empty HTML.

Why this happens Browser loads page → JavaScript runs → JS fetches data → Page updates

Requests only gets initial HTML.

How to detect JS pages - View page source: data missing - Inspect in browser: data appears - Network tab shows API calls

Solution: use browser automation Tools: - Selenium - Playwright

Simple Selenium example ```python from selenium import webdriver from selenium.webdriver.common.by import By

driver = webdriver.Chrome() driver.get("https://example.com")

title = driver.find_element(By.CLASS_NAME, "title").text print(title)

driver.quit() ```

Graph: JS rendering ```mermaid flowchart TD A[HTML] --> B[Browser] B --> C[Run JS] C --> D[Final DOM] D --> E[Scrape] ```

Remember - requests cannot run JS - Use Selenium only when needed - Always check if API exists first