Block and file storage: EBS, EFS, and instance store
EBS, EFS, and instance store are three attachment models, not three speeds. EBS is single-AZ block for one instance; EFS is multi-AZ NFS many instances share; instance store is ephemeral local NVMe you can lose. Pick by attachment and durability, not raw IOPS.
A team runs a self-managed Postgres on an i3 instance because “the local NVMe is insanely fast” — and it is, hundreds of thousands of IOPS at microsecond latency. For two years it never misses. Then a routine cost-optimization sweep flags the instance as oversized and an engineer clicks Stop, planning to resize and start it back up. Stop cryptographically erases every block of the instance store. The volume comes back empty. There was no snapshot because instance store volumes cannot be snapshotted, and the nightly logical dump had silently been failing for three weeks. The fast disk was never the problem. The problem was that “fast local disk” and “durable data” are different axes, and the team had bet a production database on a volume the platform is explicitly designed to throw away. Picking storage on AWS is first about attachment and durability, and only then about speed.
Block vs file vs object, then the three services
Before you can pick the right service, you need to know which abstraction you’re actually buying — because attaching the wrong one to your workload is exactly how teams end up with a fast disk that erases itself, or a shared filesystem where a database volume should be.
Three storage abstractions show up on every AWS design. Block storage hands your OS raw fixed-size blocks; the OS lays a filesystem (ext4, xfs) on top and the device looks like a local disk you can boot from and run a database on. File storage exposes a shared hierarchical namespace over a network protocol (NFS, SMB) so many machines mount the same tree and see each other’s writes. Object storage (S3) stores whole immutable blobs behind an HTTP API with no filesystem semantics — great for assets and backups, useless as a database volume because you can’t do partial in-place writes. This lesson is the block-and-file half: the volumes an EC2 instance actually mounts.
AWS gives you three services in that half, and the trap is to rank them by IOPS. Rank them by attachment model and durability instead:
- EBS — network-attached block, lives in one Availability Zone, attaches to one instance at a time (with a narrow Multi-Attach exception). Durable: it survives the instance.
- EFS — managed NFS file system, spans multiple AZs, many instances mount it concurrently, auto-scales capacity. Durable and shared.
- Instance store — ephemeral local NVMe physically attached to the host. Fastest, but the data is gone on stop, hibernate, terminate, or disk failure.
Together these three cover every mounting pattern: one durable node, many sharing nodes, and ultra-fast scratch. The war stories happen when you try to use one for a job that belongs to another — most commonly treating instance store’s speed as a substitute for EBS’s durability.
EBS: network block you tune by the number
EBS is the default disk for a stateful single node — a database, a message broker, a boot volume. It is network-attached, so it survives the instance: terminate the EC2 box and the volume (if not delete-on-termination) is still there to re-attach. It is single-AZ and bound to one instance at a time; the exception is io1/io2 Multi-Attach, which lets up to 16 Nitro instances in the same AZ share one volume, but only if your application brings its own clustering and I/O fencing — it is not “shared storage,” it is a building block for cluster software.
The volume types are the real senior-level lever, because gp3 decoupled performance from size. A gp3 volume includes a baseline of 3,000 IOPS and 125 MiB/s for free at any size, and you provision more independently — up to 80,000 IOPS and 2,000 MiB/s — without growing the disk. That is the headline fix over gp2, where performance was welded to size at 3 IOPS/GiB and small volumes leaned on a burst-credit balance: a gp2 volume under 1 TiB could burst to 3,000 IOPS only while it had I/O credits, and a busy small volume that drained its BurstBalance collapsed back to its tiny baseline (a 100 GiB gp2 baseline is just 300 IOPS) — a classic 2 a.m. latency-cliff incident. For sustained high IOPS with tight latency there is io1/io2 (provisioned IOPS, io2 Block Express up to 256,000 IOPS at sub-500-microsecond average latency and 99.999% durability). For cheap throughput-oriented sequential workloads — big logs, data lakes — the HDD types st1 (throughput-optimized, up to 500 MiB/s) and sc1 (cold, cheapest) win on price.
EBS volumes also resize online: you can grow capacity, change type, and change IOPS/throughput with Elastic Volumes while the instance keeps running. And snapshots are incremental backups to S3; restoring a volume from a snapshot is lazy-loaded — the volume is usable immediately but blocks fault in from S3 on first touch, so a fresh restore feels slow until it’s warmed (initialize it or use Fast Snapshot Restore if you need full performance from block zero).
# gp3 with 3000 IOPS / 125 MiB/s baseline included free at any size
aws ec2 create-volume \
--volume-type gp3 --size 200 --availability-zone us-east-1a
# Bump IOPS and throughput independently of size — online, no detach
aws ec2 modify-volume \
--volume-id vol-0abc123 --iops 9000 --throughput 500
# Snapshots are incremental backups to S3; restore lazy-loads blocks on first read
aws ec2 create-snapshot --volume-id vol-0abc123 \
--description "nightly db volume backup"▸Why this works
Why is a fresh volume restored from a snapshot slow on first access? The snapshot lives in S3 as a set of blocks. When you create a volume from it, EBS does not copy everything up front — it presents the volume immediately and lazy-loads each block from S3 the first time something reads it. So the first pass over a restored database (a full table scan, a reindex) pays an S3 round-trip per cold block and feels far slower than the volume’s provisioned IOPS suggest. Once a block has been touched it’s local and fast. To avoid the cliff during a real recovery, either pre-warm by reading the whole volume, or enable Fast Snapshot Restore so blocks are available at full performance from the start.
EFS and instance store: shared file vs throwaway local
EFS is the answer when many instances need the same files at once — a fleet of web servers serving the same uploads, a content-management cluster, container tasks that need persistent shared state. It is a managed NFS file system that spans multiple AZs, auto-scales capacity elastically as you write (no provisioning a size), and accepts thousands of concurrent connections. The cost is physics: reaching it over NFS across the network means higher latency than a local EBS volume (roughly single-digit-ms first-byte reads on Standard storage, and tens of ms on the Infrequent Access class), and per-GB it is more expensive than EBS. Throughput has modes — Elastic (auto-scales, pay per byte moved, best for spiky/unpredictable load) and Provisioned (you pin a throughput floor independent of size) — and a lifecycle policy can age cold files down to Infrequent Access (IA) and Archive classes to cut cost. The mistake to avoid is reaching for EFS as a single-node database volume: NFS latency and the lack of block semantics make it the wrong tool where EBS belongs.
Instance store is the opposite trade: ephemeral local NVMe physically attached to the host, so it delivers the highest IOPS and lowest latency of anything here — and AWS will erase it without ceremony. Data persists across a reboot, but is cryptographically erased on stop, hibernate, or terminate, and is simply gone if the underlying disk fails. You cannot snapshot it, cannot detach and re-attach it, and cannot add it after launch. That makes it correct for exactly one category: data you can afford to lose — caches, scratch space, spill/temp files, buffers, or data already replicated across the fleet (a Cassandra or Kafka node whose durability comes from replication, not from the disk). Put a primary database or the only copy of anything on it and you have built the war story at the top of this lesson.
| Dimension | EBS | EFS | Instance store |
|---|---|---|---|
| Type | Network block | NFS file (shared) | Local block (ephemeral) |
| Scope | Single AZ | Multi-AZ (Regional) | One host |
| Attachers | One instance (io2 Multi-Attach: up to 16) | Many, concurrent | One, fixed at launch |
| Durability | Survives instance; snapshots to S3 | Survives instance; multi-AZ | Lost on stop/terminate/disk fail |
| Latency / IOPS | Low ms; tunable to 256k IOPS (io2) | Higher (NFS over network) | Lowest latency, highest IOPS |
A fleet of containerized web servers behind a load balancer must all read and write the same user-uploaded files, and an upload from one node must be visible to the others within seconds. Pick the storage.
You need 9,000 sustained IOPS from a 200 GiB volume for a single-node database, with low latency and no surprise cliffs. Which choice fits best?
A Cassandra node stores its data on instance store NVMe and it has worked great for a year. Why can this be a reasonable design rather than the war story in the hook?
- 01Contrast EBS, EFS, and instance store by attachment model and durability, and say when each is correct.
- 02Name three classic failure modes — instance store data loss, EFS-where-EBS-belonged, and gp2 burst exhaustion — and the fix for each.
EBS, EFS, and instance store are three attachment models, not three points on a speed dial — rank them by attachment and durability first. EBS is network-attached block storage in a single AZ, bound to one instance at a time (io2 Multi-Attach is the narrow cluster-software exception), that survives the instance, snapshots incrementally to S3 with lazy-loaded restores, resizes online, and with gp3 lets you tune IOPS to 80,000 and throughput to 2,000 MiB/s independently of size above a free 3,000-IOPS / 125-MiB/s baseline — the fix for gp2’s burst-credit cliff. EFS is a managed multi-AZ NFS file system that many instances mount concurrently and that auto-scales capacity, trading higher latency and per-GB cost for fleet-wide sharing, with Elastic or Provisioned throughput modes and lifecycle to Infrequent Access. Instance store is ephemeral local NVMe with the highest IOPS and lowest latency, but every block is erased on stop, hibernate, terminate, or disk failure and it can’t be snapshotted, so it’s only ever for caches, scratch, or replicated data. The decision rule: a single-node database volume goes on EBS; shared content across a fleet goes on EFS; ultra-low-latency scratch you can afford to lose goes on instance store. The failure modes — losing a database to a Stop on instance store, paying NFS latency where EBS belonged, and a small gp2 collapsing when its burst credits drain — all come from choosing on raw speed instead of on attachment and durability. Prices quoted (gp3 around 0.08 USD per GB-month, snapshots around 0.05 USD per GB-month in US-East-1) are illustrative and region-dependent — check the EBS pricing page. Now when you see a storage decision on a design review, ask attachment and durability first: does it survive a Stop, does it need to be shared across the fleet, and only then — how many IOPS does it need?
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.