Security groups and network ACLs: two filters, two state models
AWS gives you two packet filters: security groups (instance-level, stateful, allow-only, SG-referencing) and network ACLs (subnet-level, stateless, ordered, deny-capable). Stateful vs stateless is the distinction that bites — open ephemeral return ports or watch responses vanish.
A team locks down a subnet with a custom network ACL: inbound rule 100 allows 443 from the internet, everything else denied. The web servers in that subnet stop responding. Not slowly — completely. curl from outside hangs and times out. The security group is wide open on 443, the route table is fine, the servers are healthy. Hours of packet captures later, someone notices the SYN arrives and the SYN-ACK leaves the instance but never reaches the client. The NACL allowed the inbound request on 443 — but it never allowed the outbound response, which leaves on an ephemeral port like 54321, not 443. A NACL is stateless: it does not remember the inbound connection, so the return packet is a brand-new flow that hit the implicit deny. One outbound rule for ports 1024-65535 fixed it. The bug wasn’t a typo. It was forgetting that one of your two filters has no memory.
Two layers, two completely different state models
Why does a correctly-configured security group sometimes fail to deliver traffic? Because the second filter — the one at the subnet edge — follows completely different rules.
A packet headed for your EC2 instance crosses two filters, and they behave nothing alike. The network ACL guards the subnet edge — it inspects traffic entering and leaving the subnet. The security group guards the elastic network interface (ENI) attached to the instance itself. A packet from the internet hits the NACL first, then the SG; a response leaves through the SG first, then the NACL. Both must permit the traffic or it dies.
The defining difference is state. A security group is stateful: if it allows an inbound request, the response is automatically allowed back out regardless of outbound rules — and if it allows an outbound request (the default SG allows all egress), the response is allowed back in regardless of inbound rules. The SG remembers the connection. A network ACL is stateless: it has no memory of prior packets. If you allow inbound 443, you have separately allowed the outbound return traffic, and that return traffic does not leave on 443 — it leaves on an ephemeral port. The client opened the connection from a high random port, so the server’s reply is destined for that port. AWS recommends opening 1024-65535 for ephemeral return traffic, because different clients pick from different ranges: a NAT gateway and many ELBs use 1024-65535, Linux kernels typically use 32768-60999, and Windows clients use 49152-65535. Open the union, or some clients silently break.
How rules are evaluated, and what each can express
The two filters even decide differently. A security group is allow-only and evaluated as a whole. You cannot write a deny rule in an SG — its only verb is “allow this.” Every rule across every SG attached to the ENI is unioned: if any rule permits the packet, it passes. There is no order and no first-match; it is one big OR of allows, with an implicit deny for anything not matched. The default SG that ships with a VPC allows no inbound traffic and all outbound traffic, and as a convenience contains a self-referencing rule so members can talk to each other.
A network ACL is an ordered list and supports explicit deny. Each rule has a number from 1 to 32766; AWS evaluates lowest number first, and the first match wins — no further rules are checked. Because it can say deny, a NACL can do something an SG fundamentally cannot: block a specific bad IP or CIDR. Put a DENY for 198.51.100.0/24 at rule 90, above your ALLOW 0.0.0.0/0 at rule 100, and that range is shut out while everyone else gets in. Every NACL ends with an unremovable * rule that denies anything unmatched. The default NACL allows all inbound and outbound; a custom NACL starts by denying everything until you add rules.
# Custom NACL — INBOUND rules (evaluated low to high, first match wins)
Rule Type Port Source Allow/Deny
90 HTTPS 443 198.51.100.0/24 DENY # block one bad CIDR first
100 HTTPS 443 0.0.0.0/0 ALLOW # everyone else may reach 443
130 Ephemeral 1024-65535 0.0.0.0/0 ALLOW # return traffic for outbound conns
* ALL ALL 0.0.0.0/0 DENY # implicit, cannot be removed
# Custom NACL — OUTBOUND rules (stateless: you MUST open the return path)
Rule Type Port Destination Allow/Deny
100 Ephemeral 1024-65535 0.0.0.0/0 ALLOW # responses to inbound 443 leave here
110 HTTPS 443 0.0.0.0/0 ALLOW # if the instance initiates outbound 443
* ALL ALL 0.0.0.0/0 DENYWhen you find yourself hardcoding an IP range into a security group rule, ask whether you can reference a role instead — the answer usually yes. The idiomatic least-privilege move lives on the SG side: reference another security group as the source instead of a CIDR. Rather than “allow 5432 from 10.0.2.0/24” on your database SG, write “allow 5432 from the app-tier SG.” Now any instance carrying the app-tier SG can reach Postgres and nothing else can — and the rule keeps working as the app tier scales and its IPs churn. You never hardcode an address; you grant access to a role.
# Least-privilege DB access: allow Postgres ONLY from instances in the app-tier SG
aws ec2 authorize-security-group-ingress \
--group-id sg-db0000000000db \
--protocol tcp --port 5432 \
--source-group sg-app00000000app| Property | Security group | Network ACL |
|---|---|---|
| Operates at | Instance / ENI level | Subnet level |
| State | Stateful — return traffic auto-allowed | Stateless — must allow both directions + ephemeral ports |
| Rule verbs | Allow only (no deny) | Allow and deny |
| Evaluation | All rules unioned (any allow passes) | Numbered, lowest first, first match wins |
| Can reference an SG as source | Yes — the least-privilege idiom | No — CIDR only |
| Default (VPC default object) | No inbound, all outbound | Allow all in and out |
| Default (custom object) | No inbound, all outbound | Deny all until you add rules |
▸Why this works
Why give us two filters that overlap? Defense in depth with division of labor. The security group is your scalpel: per-resource least privilege, expressed against roles (other SGs) rather than addresses, and stateful so you never reason about return paths. The NACL is your blunt subnet-wide guardrail: it can deny — so it can blackhole a known-bad CIDR or enforce a coarse “these two tiers may never talk” rule that no single SG can guarantee, because an SG can only grant, never forbid. Most teams run permissive default NACLs and do all real work in SGs, reaching for a NACL only when they need a deny or a subnet-level blast-radius limit. Using a NACL where an SG belongs is how you end up debugging dropped ephemeral return traffic at 2am.
You must block one specific malicious /24 from reaching every instance in a subnet, no exceptions, regardless of any allow rule someone adds later. Which filter does the job?
A subnet's custom network ACL allows inbound TCP 443 from 0.0.0.0/0, and the security group allows 443 too, but external clients still can't complete a request. What's the most likely cause?
On your database security group, which rule best expresses 'only the app tier may connect to Postgres' and keeps working as the app tier autoscales?
- 01Contrast security groups and network ACLs across state, rule verbs, evaluation, scope, and defaults — and explain the ephemeral-port trap.
- 02When do you actually use a NACL instead of just security groups, and why can't an SG do it?
Your AWS resources sit behind two packet filters with opposite personalities. The security group guards the instance’s ENI: it is stateful, so a response to any allowed flow is automatically permitted in the reverse direction; it is allow-only, with every attached rule unioned and unmatched traffic implicitly denied; and it can name another security group as a source, which is the idiomatic least-privilege pattern — “allow 5432 from the app-tier SG” survives autoscaling because it grants a role, not an address. The network ACL guards the subnet edge: it is stateless and remembers nothing, so you must allow both directions and crucially open the ephemeral return ports 1024-65535, or responses to an allowed inbound connection silently drop; its rules are numbered and evaluated lowest-first with first-match-wins; and it supports explicit deny, which lets it block a specific bad CIDR — something a security group can never do because it has no deny verb. Use the two in concert: SGs as the precise, stateful, per-resource scalpel where almost all real access control lives, and NACLs as the coarse, deny-capable, subnet-wide guardrail you reach for only when you need to blackhole a CIDR or limit a blast radius. The senior instinct is remembering which filter has memory and which doesn’t. Now when you see a connection that silently hangs after the SYN-ACK leaves the instance, check the NACL outbound rules for ephemeral ports before assuming the SG is wrong.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.
Apply this
Put this lesson to work on a real build.