Scaling Databases: Replication, Sharding, and Caching in Practice
Most systems don't die from too much code. They die from one overloaded database. Application servers scale horizontally almost for free; the database is where state lives, and state is hard to split. Having watched a single MySQL instance grind a 50,000-user platform to a halt, here is the scaling ladder I follow, in order, because each rung adds complexity you then live with forever.
Rung 0: Make sure you actually have a scaling problem
Before any architecture change, spend a week on the boring work: enable the slow query log, add the
missing indexes, kill the N+1 queries your ORM generates, and right-size the instance. I've seen
"we need sharding" turn into "we needed an index on created_at" more than once. This
rung has no architectural cost, and it routinely buys 10× headroom.
Rung 1: Read replicas
Most workloads are read-heavy, often 90%+ reads. Replication streams changes from the primary to one or more replicas; writes go to the primary, reads spread across replicas. Managed services (RDS, Cloud SQL) make the setup almost trivial, and frameworks like Laravel support read/write connection splitting natively.
The cost is replication lag: a replica is always slightly behind. The classic bug: user saves a form, the next page reads from a replica, their change is "gone". The fix is read-your-own-writes routing: after a write, pin that user's reads to the primary for a few seconds. Decide this policy up front, not after the bug reports arrive.
Rung 2: Caching
A Redis cache in front of the database absorbs repeated reads: cache-aside (check cache, on miss read DB and populate) covers most needs. The design effort goes into invalidation:
- TTL-only for data that tolerates staleness (rankings, counts, catalogue pages).
- Explicit invalidation on write for data that must read fresh: delete the key, let the next read repopulate it. Deleting beats updating: it avoids racing writers.
- Stampede protection for hot keys: when a popular key expires, a thousand requests hit the DB simultaneously. Locking or staggered TTLs prevent it.
Caching is the highest-leverage rung (we cut primary DB load by ~70%), but every cached read is a small consistency decision someone has to own.
Rung 3: Sharding, the last resort
When a single primary can no longer handle the write volume, you split the data itself: each shard holds a subset of rows, keyed by something like user ID. This is the only rung that scales writes horizontally, and the only one that changes your application permanently:
- Choose the shard key carefully. It should appear in nearly every query and distribute load evenly. Shard by user ID and every cross-user query becomes a scatter-gather across all shards.
- You lose cross-shard joins and transactions. Anything spanning shards moves into application logic, with all the partial-failure handling that implies.
- Resharding is painful. Plan for more shards than you need (e.g. many virtual shards mapped onto few physical nodes) so growth is remapping, not re-splitting.
If a managed alternative fits (Aurora for read scale, DynamoDB or similar for naturally partitionable data), seriously consider letting it do the sharding for you.
The ladder, summarised
Indexes and query fixes → read replicas → caching → sharding. Each rung up trades operational simplicity for capacity, and skipping rungs means paying complexity you didn't need yet. The systems that age best are the ones that climbed exactly as high as their traffic forced them to, and no higher.