HTTP requests, venv and packaging

HTTP with requests: the four things you must not skip

requests is Python’s de-facto HTTP client — it is to Python what fetch is to the browser, except synchronous and far more ergonomic than the standard-library urllib. A GET with query params and a JSON response is three lines:

import requests

resp = requests.get(
    "https://api.example.com/search",
    params={"q": "python", "limit": 10},
    headers={"Accept": "application/json"},
    timeout=10,
)
resp.raise_for_status()          # turn a 4xx/5xx into an exception
data = resp.json()               # parsed dict, like fetch's await resp.json()

A POST that sends JSON is the same shape — pass json= and requests serialises the body and sets Content-Type: application/json for you:

resp = requests.post(url, json={"name": "ada"}, timeout=10)

Two habits separate a script that survives production from one that hangs or lies:

• Always pass timeout=. This is the senior point, and the one beginners miss. By default requests waits forever. A single slow or dead upstream will freeze your whole script — or every worker in a pool — with no error and no recovery. fetch has the same gap (you need an AbortController); in Python you just pass a number.
• Call raise_for_status(). Unlike fetch, which rejects only on network failure and happily resolves a 500, requests does not raise on a 4xx/5xx by default — resp looks fine and resp.json() may return an error body you treat as data. raise_for_status() converts a bad status into an HTTPError you can catch. (fetch is the mirror image: you must check resp.ok yourself.)

For many parallel calls or an async app, reach for httpx — same API surface as requests, but it adds async/await and HTTP/2. A retry with backoff is worth wiring in for any real upstream: on a 429 or 503, wait 0.5s, 1s, 2s… and retry a few times rather than failing on the first blip. (Libraries like tenacity, or urllib3’s Retry adapter, do this for you.)

The AI-glue payoff: calling an LLM HTTP API safely

Most “AI integration” is exactly this pattern: POST a JSON body to a provider endpoint with an auth header, then parse the JSON that comes back. The senior detail is where the key lives — never in source.

import os
import requests

api_key = os.environ.get("LLM_API_KEY")  # read from the environment, not the file
if not api_key:
    raise RuntimeError("LLM_API_KEY is not set")

resp = requests.post(
    "https://api.provider.example/v1/chat/completions",
    headers={"Authorization": f"Bearer {api_key}"},
    json={
        "model": "some-model",
        "messages": [{"role": "user", "content": "Summarise this ticket: ..."}],
    },
    timeout=30,                          # LLMs are slow; give them room, but bound it
)
resp.raise_for_status()
reply = resp.json()["choices"][0]["message"]["content"]

os.environ["LLM_API_KEY"] is Python’s process.env.LLM_API_KEY. You set it in your shell, a .env loaded by python-dotenv, or your deploy platform’s secret store — and the key never appears in the repo, in git history, or in a screen-share. Hard-coding it is the single most common way credentials leak. The generous timeout=30 matters here too: model responses are slow, so a tight timeout cancels good requests, but no timeout means one stuck generation hangs your tool indefinitely.

Environments and packaging: why venv exists

Ask yourself: how many Python projects do you or your teammates have on one machine? Each one has opinions about which library versions it needs — and by default Python lets them all fight over the same global install folder. Here is the teammate’s failure from the hook. Python, by default, installs packages into one global site-packages shared by every project on the machine. Project A needs urllib3==1.26; project B needs urllib3==2.x. Installed globally, the second pip install overwrites the first — and now whichever project loads the wrong version breaks. There is no node_modules per project to keep them apart.

A virtual environment (venv) is that per-project boundary. It is a lightweight, isolated copy of the interpreter with its own site-packages, so each project’s dependencies live in their own folder and never collide:

python -m venv .venv          # create an isolated env in ./.venv
source .venv/bin/activate     # macOS/Linux  (Windows: .venv\Scripts\activate)
pip install requests httpx    # installs INTO .venv, not globally

While activated, python and pip point at .venv — analogous to how node_modules scopes packages to one project, except a venv isolates the interpreter itself, not just the libraries. You .gitignore the .venv/ folder (like node_modules/) and commit a manifest instead so anyone can recreate it.

Pinning: requirements.txt and the move to pyproject.toml

Isolation is half the story; reproducibility is the other half. If you pip install requests today and your teammate does it next month, you may get different versions. Pin them. The classic file is requirements.txt:

requests==2.32.3
httpx==0.27.2

Recreate the exact set with pip install -r requirements.txt inside a fresh venv. This is package.json + a lockfile, split across convention rather than one file. The modern, standardised home for project metadata and dependencies is pyproject.toml (PEP 621) — declaring deps there, and using a fast resolver like uv or poetry, gives you a real lockfile and one-command setup, much closer to the npm install experience. For a quick script requirements.txt is fine; for anything you ship or share, prefer pyproject.toml.