Web Scraping30 min read
Configuration and Secrets
Learn how to manage settings and protect secrets like database URLs and API keys using config files and environment variables.
David Miller
December 17, 2025
1.4k41
Never hardcode:
- database passwords
- API keys
- URLs
They change and must be protected.
Use environment variables
export DB_URL="sqlite:///scraped.db"
export API_KEY="secret123"
import os
db_url = os.getenv("DB_URL")
api_key = os.getenv("API_KEY")
Use .env file (with python-dotenv)
DB_URL=sqlite:///scraped.db
API_KEY=secret123
from dotenv import load_dotenv
load_dotenv()
Why config matters
- safe secrets
- different envs (dev, prod)
- easy changes without code edit
Graph: config flow
flowchart LR
A[Env / Config File] --> B[App]
B --> C[DB / APIs]
Remember
- Never commit secrets to git
- Use env vars for sensitive data
- Keep config separate from logic
#Python#Advanced#Config