Web Scraping30 min read
Configuration and Secrets
Learn how to manage settings and protect secrets like database URLs and API keys using config files and environment variables.
David Miller
December 21, 2025
0.0k0
Never hardcode: - database passwords - API keys - URLs
They change and must be protected.
---
Use environment variables
```bash export DB_URL="sqlite:///scraped.db" export API_KEY="secret123" ```
```python import os
db_url = os.getenv("DB_URL") api_key = os.getenv("API_KEY") ```
---
Use .env file (with python-dotenv)
``` DB_URL=sqlite:///scraped.db API_KEY=secret123 ```
```python from dotenv import load_dotenv load_dotenv() ```
---
Why config matters - safe secrets - different envs (dev, prod) - easy changes without code edit
---
Graph: config flow
```mermaid flowchart LR A[Env / Config File] --> B[App] B --> C[DB / APIs] ```
---
Remember - Never commit secrets to git - Use env vars for sensitive data - Keep config separate from logic
#Python#Advanced#Config