Environments and gates: required reviewers, wait timers, and branch protection
A GitHub Environment scopes secrets and wraps a deploy job in protection rules: required reviewers force human approval, a wait timer creates a soak window, and deployment-branch rules plus branch protection ensure only reviewed code from main can ever reach production.
At 2 a.m. an engineer pushed a hotfix branch and, to save time, pointed the deploy workflow straight at it. The job had the production deploy token in scope, so it shipped — straight to prod, from an unreviewed branch, with no second pair of eyes and no soak. The hotfix fixed the symptom and introduced a worse bug, and because nothing stood between “push” and “production” it reached 100% of users in ninety seconds. The retro found the real defect was not the bad code; it was that production was reachable by anyone who could trigger a workflow, from any branch, instantly. The fix was to put production behind a gate it could not skip: a named environment that holds the deploy secret, demands a human approval the deployer cannot self-grant, enforces a short wait so a bad release can be caught before it widens, and refuses to deploy anything that is not reviewed code on main.
The environment is a scoped, gated boundary
A GitHub Environment (production, staging) is two things at once: a scope for secrets and a set of protection rules around any job that targets it. A job opts in with environment: production, and at that moment three things happen. First, the job gains access to environment-scoped secrets — the prod deploy token lives on the environment, not loose in the repo, so only a job that successfully enters production can ever see it. Second, the run pauses before the job starts until every protection rule is satisfied. Third, the deployment is recorded against the environment, giving the audit trail of who deployed what, when. The Hook’s incident is impossible the moment the prod token is environment-scoped and the environment is gated, because there is no longer a path from “trigger a workflow” to “hold the prod credential” that does not pass through the rules.
jobs:
deploy-prod:
runs-on: ubuntu-latest
environment: production # scopes secrets + triggers protection rules
steps:
- run: ./deploy.sh
env:
TOKEN: ${{ secrets.PROD_DEPLOY_TOKEN }} # only readable inside `production`The three protection rules
Each rule closes a different kind of gap. When you see the 2 a.m. incident in the Hook, notice which specific rule would have stopped it — and notice that removing any single rule recreates a version of that gap. The environment’s rules are what convert “a job that can deploy” into “a governed deploy”:
- Required reviewers: the run blocks until a named person or team approves the deployment in the GitHub UI. Critically, the approval is separate from the code review — it gates the act of deploying this run to this environment, and a reviewer who is the deployer typically cannot self-approve, so shipping always needs a second human. This is the rule that the 2 a.m. push bypassed.
- Wait timer: a configurable delay (0–43,200 minutes, i.e. up to 30 days) inserted before the job runs after approval. A short timer — say 10 minutes — creates a deliberate soak/abort window: an obviously-bad release can be cancelled before it touches production, and a canary in front of it has time to report.
- Deployment branches: restrict which branches may deploy to the environment — typically
mainonly (or a release-tag pattern). Combined with branch protection onmain(required PR review, required status checks, no direct pushes), this guarantees the only code that can reach production is code that was reviewed and passed CI on the protected branch. The hotfix branch from the Hook would be refused outright.
Together these three rules mean production is not just password-protected — it is structurally unreachable unless code quality, human judgment, and timing all clear. Remove any one and a gap opens — direct-push code, a self-approved deploy, an instant blast — which is precisely the gap the incident fell through.
A deploy job sets `environment: production`, which has required reviewers configured. Why does scoping the prod deploy token to that environment (rather than storing it as a repo secret) close the 2 a.m. hotfix hole even before anyone reviews the code?
Required reviewers vs branch protection: two different gates
A common confusion is treating branch protection and required reviewers as redundant. They guard different things. Branch protection governs what code is allowed onto main — it is about the artifact’s trustworthiness and fires at merge time. Required reviewers on the environment governs whether this specific run may deploy now — it is about the act of releasing and fires at deploy time. A change can be perfectly reviewed and merged (branch protection satisfied) and still need a deliberate, separately-approved decision to push it to production during business hours with someone watching (environment reviewer satisfied). You want both because they fail differently: branch protection alone lets any green main commit auto-deploy with no human in the loop at release time; environment review alone lets a human approve deploying code that was never reviewed. The wait timer then adds time to the human gate, and deployment-branch rules bind the two together by ensuring the run being approved actually originates from the protected branch.
Branch protection on `main` already requires PR review before merge. Why also configure required reviewers on the `production` environment — isn't that the same approval twice?
▸Why this works
Why separate the deploy approval from the code review at all — wouldn’t one approval be simpler? Because they answer different questions at different times. The PR review asks “is this change correct?” when the code is proposed; the deploy approval asks “should we release this, now, to these users?” when the artifact is about to ship — a decision that depends on the current state of production, the on-call schedule, an ongoing incident, or a change freeze, none of which the original reviewer could know. Collapsing them means either you lose the release-time decision (code auto-ships the moment it merges) or you lose the merge-time decision (no diff review). The two-gate design is what lets a team merge freely into a protected main yet still hold a deliberate, human, abortable moment before each production release.
- 01Explain how scoping the prod deploy secret to a GitHub Environment, combined with required reviewers, makes an unreviewed instant deploy impossible.
- 02Why are branch protection and environment required-reviewers not redundant, and what does each protect?
A GitHub Environment is simultaneously a secret scope and a gate: a job opts in with environment: production, and only then can it read the environment-scoped deploy token, only after the run pauses for every protection rule, and the deployment is recorded for audit. Scoping the prod token to the environment is itself a control — there is no path from triggering a workflow to holding the prod credential that does not pass the rules, which is what makes the 2 a.m. unreviewed-hotfix-straight-to-prod incident structurally impossible. The three rules compose a chain. Required reviewers force a human approval of the deploy act, separate from the PR code review and not self-grantable, so every release needs a second person at release time. A wait timer — up to 30 days, but in practice a short soak like 10 minutes — inserts an abort window after approval so a bad release can be cancelled and a canary can report before it widens. Deployment-branch rules restrict which branches may deploy, typically main or a release tag, and combined with branch protection on main (required review, required checks, no direct push) guarantee only reviewed, CI-passed code reaches production. Branch protection and environment review are not redundant: one governs what code enters main at merge time, the other governs whether a run may deploy now at release time, and they fail in different ways, so you keep both. Remove any link — direct-push code, a self-approved deploy, an instant blast with no soak — and the exact gap the incident fell through reopens. Now when you review a colleague’s pipeline and see a loose repo secret wired directly into a deploy job, you know exactly what is missing and why it matters at 2 a.m.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.