Capstone: a secure, resilient three-tier architecture end to end

By the end of this lesson you will be able to walk the full request path, name every trust boundary along it, and spot the two structural gaps that made the startup’s “three-tier” an incident waiting to happen.

The shape: one VPC, two AZs, three tiers

Everything lives in one VPC spanning at least two Availability Zones — that “at least two” is the whole reliability story, because an AZ is the blast radius AWS lets fail independently. Inside the VPC, subnets split into two trust zones. Public subnets (one per AZ) hold only the internet-facing edge: the Application Load Balancer and the NAT Gateways. Private subnets hold everything that should never be directly reachable: the Fargate app tasks and the RDS database. The rule is brutally simple — if a resource has a route to an internet gateway, it is exposed; the database must not.

The request path tells the story. A user hits your apex domain; Route 53 resolves it (an ALIAS record at the zone apex, which is how you point a bare domain at an AWS resource since you cannot CNAME an apex). The request lands on CloudFront, which terminates TLS at the edge using an ACM certificate, serves cached static assets, and forwards dynamic requests to the Application Load Balancer. The ALB lives in the public subnets, listens on HTTPS, and routes to a target group of Fargate tasks in the private subnets. A task runs your app, and to do its job it talks to two backends: RDS (Postgres, Multi-AZ) for relational data and S3 for objects. The reply walks back out the same chain. Every arrow in that path is also a boundary you get to lock down.

Each tier maps to a managed service for a reason — you are not running the muscle, AWS is, and you spend your attention on the boundaries instead.

Security: boundaries that reference each other

The security model is defense in depth, the spine of the Well-Architected Security Pillar: many layers, each useless to bypass alone. The first layer is the subnet split — RDS in a private subnet simply has no route to the internet, so “open port to the world” stops being possible. The second is security groups that reference each other by ID, not by CIDR. You write three: the ALB SG allows 443 from the internet; the app SG allows traffic only from the ALB SG; the DB SG allows 5432 only from the app SG. Nobody types an IP range. The chain is internet to ALB SG to app SG to DB SG, and each link is least privilege by construction — exactly the failure the startup hit, except now structurally impossible.

The rest of the layers stack on top. Secrets (the DB password, API keys) live in Secrets Manager and are injected into the task at runtime, never baked into an image or an env file in git. Encryption at rest comes from KMS on RDS and S3; encryption in transit is TLS end to end — ACM at CloudFront, and the app-to-RDS hop using the RDS CA. The Fargate task role is a least-privilege IAM role that grants exactly the S3 bucket and Secrets Manager paths this service needs and nothing more, so a compromised container cannot pivot across the account. Nothing in the entire system is publicly reachable except CloudFront and the ALB.

# Trimmed CloudFormation: the security boundaries are the whole point.
Resources:
  AlbSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      VpcId: !Ref Vpc
      SecurityGroupIngress:
        - { IpProtocol: tcp, FromPort: 443, ToPort: 443, CidrIp: 0.0.0.0/0 }  # ALB = only public ingress

  AppSG:                       # Fargate tasks — private subnets only
    Type: AWS::EC2::SecurityGroup
    Properties:
      VpcId: !Ref Vpc
      SecurityGroupIngress:
        - { IpProtocol: tcp, FromPort: 8080, ToPort: 8080, SourceSecurityGroupId: !Ref AlbSG }

  DbSG:                        # RDS — reachable by the app tier and nothing else
    Type: AWS::EC2::SecurityGroup
    Properties:
      VpcId: !Ref Vpc
      SecurityGroupIngress:
        - { IpProtocol: tcp, FromPort: 5432, ToPort: 5432, SourceSecurityGroupId: !Ref AppSG }

  Db:
    Type: AWS::RDS::DBInstance
    Properties:
      Engine: postgres
      MultiAZ: true                          # synchronous standby in a 2nd AZ, auto failover
      StorageEncrypted: true                 # KMS at rest
      PubliclyAccessible: false              # never a public IP
      DBSubnetGroupName: !Ref PrivateDbSubnets
      VPCSecurityGroups: [ !Ref DbSG ]
      ManageMasterUserPassword: true         # password lives in Secrets Manager, not the template

This is infrastructure as code on purpose. The whole VPC, every subnet, every SG rule, the ALB, the Fargate service, and the RDS instance are declared in one reviewable template (CloudFormation, CDK, or Terraform). The open-DB incident happened because a human ran a one-off console change that nobody reviewed and nobody could see later. When the architecture is code, that drift shows up in a diff and gets caught in review — the SG rule above cannot silently become 0.0.0.0/0 without someone approving the pull request.

Resilience and observability: surviving an AZ and seeing it happen

The single-AZ fault from the hook is fixed structurally. The ALB spans both public subnets; Fargate tasks run across both private subnets; and RDS Multi-AZ keeps a synchronous standby in a second AZ. Per the RDS docs, replication to that standby is synchronous, so a committed write is on both AZs before the client gets an ack, and on primary failure AWS performs an automatic failover by flipping the DB’s DNS endpoint to the standby — your app reconnects to the same hostname. The cost of that safety is real: synchronous replication adds write/commit latency versus single-AZ, and you pay for the standby. That is the central resilience tradeoff, and for a production data tier it is almost always worth paying. (Note the standby does not serve reads — it exists for failover; if you need read scaling you add read replicas or a Multi-AZ cluster.)

You cannot operate what you cannot see, so observability is wired in from the start, the way the Well-Architected detection guidance prescribes. CloudWatch collects metrics and logs from every tier; alarms watch the signals that actually mean “users are hurting”: ALB HTTPCode_ELB_5XX_Count, Fargate CPU/memory utilization (which also drives autoscaling), and RDS DatabaseConnections and free storage. Alarms fire to an SNS topic that pages a human. X-Ray traces a request across CloudFront to ALB to Fargate to RDS so that when latency spikes you can see which tier owns it instead of guessing. The point of tracing the request path earlier is exactly this: each hop is a place a metric, a log, and a trace segment must exist.

Cost: the same design, priced

A correct architecture you cannot afford gets rewritten under pressure into an incorrect one, so cost is part of the design, not an afterthought. The Fargate app tier runs a steady baseline plus autoscaling headroom; you cover the baseline with a Compute Savings Plan (a 1- or 3-year commit that discounts Fargate and Lambda alike) and let on-demand absorb the spikes. The data tier’s Multi-AZ standby roughly doubles RDS compute cost — that is the availability premium, and it is the one place you do not cut. NAT Gateway hours and per-GB egress are the sneaky line item; the per-AZ NATs are for resilience, but you cut their bill by sending AWS-bound traffic (S3) through a free gateway endpoint and keeping chatty cross-AZ traffic in check. S3 gets lifecycle rules to tier cold objects to cheaper storage classes. Right-size Fargate task vCPU/memory and RDS instance class against real CloudWatch utilization rather than guessing — the observability you built for reliability also tells you where you are overprovisioned.