Policy engines and authz at scale

Why scattered authorization decays

Authorization starts simple. One service, a couple of roles, an if user.role == "admin" near the top of the handler. It works, so the next service copies the pattern, and the one after that. Nothing about this is a mistake on any given day — it’s the path of least resistance, and each team is locally correct. The failure is emergent: after a year you have the same rule expressed forty subtly different ways, in four languages, against three slightly different role models, and no single place that can answer a policy question.

That decay has three concrete costs a senior feels in production. First, drift: when “managers can approve their own team’s expenses up to $5k” changes to “$10k,” you must find and edit it in every service that enforces it — and you will miss one, silently, because nothing fails when a check is merely stale rather than absent. Second, unauditability: a compliance or incident question like “who can delete a customer record” has no authoritative answer; you grep forty repos and hope. Third, inconsistent enforcement: the refund rule that allows team leads in the CSR tool but not the billing service isn’t a feature — it’s a vulnerability that nobody decided to ship. Scattered authz isn’t just messy; it is the structural cause of the broken-access-control bugs you’ve already met, multiplied by your service count.

Externalizing the decision: PDP and PEP

The fix is to separate deciding from enforcing. This is the PDP/PEP split, and it’s the mental model that makes everything else click:

• The Policy Enforcement Point (PEP) lives in your service. It’s the code at the request boundary that says “I’m about to do action X on resource Y for user Z — am I allowed?” It does not contain the rule. It asks.
• The Policy Decision Point (PDP) is the policy engine. It holds the rules as data, receives the question plus relevant context (the user’s roles, the resource’s owner, attributes), evaluates the policy, and returns allow or deny.

Your application code shrinks to one line: gather the input, call the PDP, honor the answer. The logic — every condition, role, and exception — moves into versioned policy-as-code that lives in one repository, gets code-reviewed, gets tested in CI, and is deployed independently of the services that consume it. The win is direct: the auditor’s question now has exactly one place to look, the refund rule changes in exactly one file, and enforcement is identical everywhere because every service asks the same brain.

OPA and Rego in practice

The reference implementation of this pattern is OPA (Open Policy Agent), a CNCF-graduated, general-purpose policy engine. You don’t write OPA policies in your application language; you write them in Rego, a declarative query language built for exactly this shape of problem — “given this input, does any rule conclude allow?” A trivial Rego policy looks like this:

package authz

default allow := false

allow if {
    input.action == "refund"
    input.user.role in {"billing_admin", "team_lead"}
}

Two things matter here. default allow := false is deny by default expressed as one line — if no rule fires, the answer is no, which is exactly the reflex access control demands. And the rule is data the engine evaluates, not control flow you maintain: you change who can refund by editing this file, not by hunting through service code. OPA isn’t only for service authz, either — the same engine and language enforce Kubernetes admission control (via Gatekeeper), Terraform plan checks, and CI gates, which is why investing in Rego pays off across the platform, not just one API.

The operational question is where the PDP runs, and this is where seniors earn their pay:

The dominant production answer is the sidecar / local-daemon model: OPA runs next to each service (same pod, same host), policy is distributed as signed bundles the agent pulls periodically, and the decision is a loopback call measured in microseconds. You trade a network round-trip you can’t afford on every request for eventual consistency of policy — a changed rule takes a bundle-poll interval to reach every node. For authorization that’s usually fine; for revoking a compromised admin right now, you must understand that the propagation lag is real and design for it.

What a senior gets wrong (and right)

The seductive mistake is treating “we adopted OPA” as the finish line. Externalizing the engine doesn’t externalize the data the policy needs: if the policy must know that user 42 owns invoice 4815, that ownership fact has to reach the PDP somehow — passed in the input, or replicated into the engine, or fetched mid-decision. Each option is a real tradeoff (input bloat vs. a stale replica vs. a decision-time dependency you were trying to avoid). And the engine is now on your critical path: you need a defined behavior when the PDP is unreachable. Fail-closed (deny on engine error) is the secure default and the right call for sensitive actions; fail-open is occasionally justified for low-risk reads where availability beats strictness — but that is a decision you make explicitly, per action, with eyes open, never a fallback you discover during an outage.