open atlas
↑ Back to track
CI/CD pipelines CICD · 03 · 01

Continuous delivery vs deployment, and release strategies

Continuous delivery keeps a release-ready artifact and a human clicks deploy; continuous deployment auto-ships every green commit. Then pick a rollout: blue-green, canary, rolling, or flag-gated — each with its own rollback speed, infra cost, and DB compatibility trap.

CICD Middle ◷ 17 min
Level
FoundationsJuniorMiddleSenior

At 14:02 a team flips their blue-green switch: traffic moves from the old environment to the new one in a single load-balancer change. Latency is flat, error rate is flat — the deploy looks perfect. At 14:09 the support queue explodes. The new release ran a migration that dropped a column the old code still wrote to, and the “instant rollback” everyone bragged about is now useless: flipping back sends live traffic to code that queries a column the database no longer has. The deploy strategy was sound. The database compatibility was not, and that is the line where most release strategies actually fail.

Two D’s: delivery decides who presses go, deployment removes the button

These two terms are constantly confused, and the difference is one sentence. Continuous delivery means every change that passes the pipeline produces an artifact that is always release-ready; getting it to production is a deliberate human action — someone clicks “deploy”. Continuous deployment removes that click: every commit that goes green through the full pipeline ships to production automatically, no human in the loop.

The gap between them is not tooling, it is trust. The same pipeline backs both; deployment is just delivery with enough test coverage, monitoring, and fast rollback that you let the machine press go. Crucially, delivery vs deployment is a separate axis from which rollout strategy you use. You decide who presses go (delivery vs deployment), and then — independently — how the new version reaches users (blue-green, canary, rolling, or flag-gated). A team can run continuous deployment with a canary rollout, or continuous delivery with a blue-green flip; the two choices compose.

Blue-green: two environments, flip at the load balancer

Blue-green keeps two identical production environments. Blue serves live traffic; you deploy the new version to the idle Green, smoke-test it out of band, then flip the load balancer so all traffic moves to Green at once. Rollback is the same flip in reverse — point back at Blue — so it is effectively instant, on the order of seconds, with no rebuild or redeploy.

The bill is two parts. First, cost: you run roughly 2× the infrastructure during the overlap, because both full environments exist at once (cloud autoscaling softens this, but the idle environment is real spend). Second, and the part that bites: the database is shared, and the schema must be compatible with both versions simultaneously. The flip swaps app code in an instant; it does not fork the database. If Green’s release runs a destructive migration — drops a column, renames a table, narrows a type — then the moment you need to roll back to Blue, Blue’s still-running code hits a schema it no longer understands and your instant rollback evaporates. The discipline that makes blue-green safe is the expand/contract (parallel-change) migration pattern: expand the schema additively first (add the new column, keep the old), deploy code that reads both, migrate the data, and only contract — drop the old column — a release later, once nothing reads it. Never ship a backward-incompatible schema change in the same release you flip.

Canary: route a slice, watch the metrics, ramp or roll back

A canary release sends the new version to a small fraction of traffic first — a common ramp is 1% → 5% → 25% → 50% → 100% — while you compare its error rate and latency against the old version on the rest. If the canary looks healthy you ramp up; if it breaches a threshold you route everyone back to the stable version, having exposed only that small slice to the bug.

The single most important word here is automated. A canary without an automated metric gate is just a manual deploy you are babysitting: an engineer squinting at a Grafana dashboard, who blinks, goes to lunch, or reads noise as signal. Real canarying wires the rollout to your SLOs — define a window and a minimum sample size (e.g. a 10-minute window, ≥5000 requests so you are not deciding on statistical noise), thresholds like error-rate increase > 0.3% over control or p99 latency up > 5%, and an automatic rollback trigger on breach. One subtlety teams miss: measure the failure rate inside the canary slice, not globally. If the canary holds 5% of traffic, a total meltdown of those users barely moves the global error rate — yet 100% of canaried users are failing. Compare canary-vs-control on the same denominator, or the gate lies to you.

Why this works

Why is the manual canary so dangerous? Because it feels safe — you “only” sent 5% of traffic to the new version — while actually removing the very thing that makes canarying work: a fast, objective, always-watching decision. The whole value proposition is “catch a bad release on 5% of users in minutes and auto-revert.” A human checking a dashboard every so often turns that into “maybe catch it, maybe in an hour, maybe after the dashboard already scrolled past the spike.” If you cannot wire a metric gate, a blue-green flip you can revert in seconds is often the more honest choice than a canary you are eyeballing.

Rolling and feature-flagged: the cheap one and the decoupled one

Rolling deployment replaces instances batch by batch — take down a few old pods, bring up new ones, repeat — until the whole fleet is on the new version. It is the cheapest strategy and the Kubernetes default: no second environment, no 2× infra. The costs are two. First, mixed versions run simultaneously during the roll, so old and new instances serve requests side by side — which means a non-backward-compatible API or schema change is a live bug: an old pod and a new pod must both be able to handle any request and read the same database, or users hit errors mid-rollout. Second, rollback is slow — there is no flip; you have to roll forward (or back) batch by batch all over again, taking as long as the deploy did.

Feature-flag-gated release attacks a different problem: it decouples deploy from release. You ship the new code to production dark — deployed but gated behind a runtime flag that is off — then enable it per cohort (internal users, then 1%, then a region) by flipping the flag, with no redeploy. Turning a feature off is a config change that takes effect in seconds, which is the fastest “rollback” of all because nothing redeploys. Flags also enable dark launches: run the new code path against real traffic but discard its output, comparing performance and error rates before any user sees it. The cost is carrying-cost and complexity — every live flag is a branch in your code and a combinatorial test surface, and stale flags rot into landmines, so flags need an owner and an expiry.

StrategyRollback speedExtra infraMixed versions live?Signature failure mode
Blue-greenSeconds (flip back)~2× (two full envs)No (all-at-once flip)Destructive migration breaks the rollback flip
CanarySeconds–minutes (re-route slice)Small (run one extra version)Yes (during ramp)No automated metric gate → manual babysitting
RollingSlow (roll back batch by batch)NoneYes (whole roll)Non-backward-compatible API/schema mid-roll
Feature-flagInstant (flip flag, no redeploy)None (flag service)N/A (deploy ≠ release)Stale-flag rot, combinatorial test surface

A canary gate in CI/CD is usually a metric query against your monitoring system, with the rollout paused until it passes. Conceptually:

# argo-rollouts-style canary with an SLO gate (sketch)
strategy:
  canary:
    steps:
      - setWeight: 5        # 5% of traffic to the new version
      - pause: { duration: 10m }   # observe a real window
      - analysis:           # automated gate, not a human
          templates: [error-rate]
          # error-rate template: fail if 5xx(canary) - 5xx(stable) > 0.3%
          # over >=5000 requests -> auto-rollback, abort the ramp
      - setWeight: 25
      - pause: { duration: 10m }
      - setWeight: 100      # promote
# the manual-babysitting anti-pattern this replaces:
kubectl set image deploy/api api=api:v2   # 5% canary by replica count
# ...then an engineer stares at a dashboard and hopes to notice a spike
Pick the best fit

A payments API on Kubernetes must ship a risky rewrite. You have solid SLO dashboards and automated metric analysis, the change is backward-compatible at the DB and API level, and you need any bad release caught on a tiny fraction of real traffic and auto-reverted. Pick the release strategy.

Quiz

A blue-green deploy went out, then a critical bug appeared. The team flips back to Blue — but the app now throws database errors on every request. What most likely happened?

Recall before you leave
  1. 01
    Distinguish continuous delivery from continuous deployment, and explain why the rollout strategy is a separate choice.
  2. 02
    Give the signature failure mode of blue-green, canary, and rolling, and the single fix for each.
Recap

Continuous delivery and continuous deployment differ by one step: delivery keeps an always-release-ready artifact and a human clicks deploy, while deployment auto-ships every green commit — the same pipeline, with deployment earned by test coverage, monitoring, and fast rollback. That choice is independent of the rollout strategy, which you pick on its own merits. Blue-green runs two identical environments and flips traffic at the load balancer for seconds-fast rollback, at ~2× infra and a hard rule that the shared schema must suit both versions — a destructive migration kills the rollback unless you use expand/contract. Canary routes a small slice (say 1%→5%→25%→100%) and compares it against the stable version, but only works with an automated metric gate tied to SLOs and measured inside the canary slice; without that gate it is a manual deploy you are babysitting. Rolling replaces instances batch by batch for zero extra infra, accepting mixed versions live (so changes must be backward-compatible) and slow rollback. Feature flags decouple deploy from release entirely: ship dark, enable per cohort at runtime, revert in seconds by flipping a flag — paid for in carrying cost and stale-flag rot. Match the strategy to your rollback needs, infra budget, and — above all — your database compatibility. Now when you see a post-mortem that says “instant rollback failed,” ask first whether a migration ran: that answer tells you which strategy was the wrong fit and what to reach for next time.

Practice

Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.

recallapplystretch0 of 5 done
Connected lessons

Something unclear?

Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.

shortcuts expand
search
K
prev piece
k
next piece
j
cycle tier
t
this menu
?
sources4
expand
  1. 01
  2. 02
  3. 03
  4. 04

Trademarks belong to their respective owners. Editorial reference only.