System design interview prep: the 12 patterns that matter

System design interviews look open-ended, but the answers are built from a small set of recurring patterns. Twelve of them cover most prompts: load balancing, caching, CDNs, replication, sharding, consistent hashing, queues, idempotency, rate limiting, retries with circuit breakers, fan-out, and search indexing. For each one: the problem it solves, how it works, what it costs, a real example, and how to talk about it in the interview. Learn the trade-offs, not the names; the trade-offs are what gets graded.

How to use these patterns

A typical round runs in four stages: clarify requirements, estimate the load, sketch a high-level design, then go deep on one or two parts. Patterns score in the last two stages, and only when they follow from a requirement you stated.

"Add a cache" earns little. "Reads outnumber writes about 100 to 1 and a profile can be a minute stale, so I'll put a cache in front of the profile service" earns a lot. Lead with the when.

The twelve were chosen by one test: each shows up both in common prompts (feeds, chat, URL shorteners, rate limiters, search) and in production systems that publish how they work.

1. Load balancing and stateless services

The problem. One server has a ceiling on CPU, memory and connections, and it is a single point of failure.

How it works. A load balancer sits in front of a pool of identical servers and spreads requests across them. Layer 4 balancers route TCP or UDP connections without reading them; layer 7 balancers read HTTP, so they can route by path, terminate TLS, and retry. An AWS Application Load Balancer defaults to round robin and also offers least outstanding requests. Health checks remove failed servers from rotation.

This only works if servers are stateless: session data lives in a shared store (a database, Redis, a signed cookie), so any server can handle any request.

Trade-offs. The balancer itself must be redundant. Sticky sessions (pinning a user to one server) cope with in-memory state, but they skew load and lose sessions when a server dies. WebSockets stay on one server once established, which slows rebalancing.

Example. An ALB in front of an auto-scaling group. When a server is removed, the ALB drains it, waiting up to 300 seconds by default for in-flight requests to finish.

In the interview. Say "stateless app servers behind a load balancer" early, then say where the state went. For chat or live updates, add that connection-holding servers find each other through a pub/sub channel.

2. Caching and invalidation

The problem. The database is the slowest part of most read paths, and many reads ask for the same data.

How it works. The most common shape is cache-aside (also called look-aside): read from the cache; on a miss, read from the database and store the result. On a write, update the database and then delete the cache key rather than updating it. Facebook's memcache paper gives the reason: deletes are idempotent, so a repeated delete cannot leave a wrong value behind. Alternatives are write-through (write cache and database together) and write-behind (flush to the database later, which risks losing data). Every cached value also gets a TTL, so a missed invalidation heals itself.

Trade-offs. A cache trades freshness for speed, so decide how stale each kind of data may be. Two failure modes come up often:

• Stale sets. Two requests race, and the slower one writes an old value into the cache after a newer write invalidated it.
• Thundering herd (cache stampede). A hot key expires and thousands of requests miss at once and all hit the database.

Facebook addressed both with leases: on a miss, the cache hands out a token, only the token holder may fill the key, and other clients wait briefly. In their paper this cut the peak database query rate for the affected keys from 17,000 per second to 1,300. Simpler mitigations are a per-key lock, serving the stale value while one request refreshes it, and random jitter on TTLs.

Example. A product page reads price and stock from Redis with a 60-second TTL. A price change writes to the database and deletes the key.

In the interview. Name the hit path, the miss path and the invalidation path, then state the staleness you accept. Interviewers often ask "what happens when a hot key expires?" Have the stampede answer ready.

3. CDNs and edge caching

The problem. Users far from your servers pay in latency, and your origin pays for every byte it serves.

How it works. A CDN keeps copies of responses in data centres close to users, controlled by HTTP headers from the origin. Cache-Control: s-maxage=N sets how long shared caches such as a CDN may keep a response, separately from max-age for browsers. The stale-while-revalidate extension (RFC 5861) lets a cache serve a stale copy for a set time while it fetches a fresh one in the background.

Static assets get content-hashed filenames (app.3f9c1a.js) and a long cache lifetime. A new deploy changes the filename, so you never need to invalidate the old one.

Trade-offs. CDNs cache well only what is the same for many users; personalised responses usually bypass the cache. Explicit invalidation exists but is slower and often billed, which is why versioned URLs are preferred.

Example. This site serves its article pages as prerendered HTML from CloudFront, and only the small dynamic parts reach a Lambda function. The setup is described in How Shipworthy runs Next.js 16 on AWS with SST.

In the interview. For any read-heavy prompt with public content, put a CDN in front and say what is cacheable and for how long. Large media comes from object storage behind the CDN, never through app servers.

4. Replication and consistency

The problem. One database copy is a single point of failure, and one machine can only serve so many reads.

How it works. Writes go to a primary, which streams changes to one or more replicas. Replicas serve reads and can be promoted if the primary fails.

The key choice is synchronous or asynchronous. PostgreSQL streaming replication is asynchronous by default: commits return before replicas have the change, lag is typically under a second, and a primary crash can lose the last few commits. Synchronous replication waits for a standby to confirm, which closes that window but adds at least one round trip to every commit and can stall commits if the standby is down.

Leaderless systems take a different route. In Amazon's Dynamo design, each item lives on N replicas, a write must reach W of them and a read must hear from R. If R + W > N, every read set overlaps every write set. The paper reports (N, R, W) = (3, 2, 2) as a common configuration. Dynamo itself relaxes this with a "sloppy quorum" during failures, trading strictness for availability.

Trade-offs. Async replicas give you replication lag: a user updates their profile, the next read hits a replica, and the change seems to have vanished. The fix is to route a user's reads to the primary briefly after they write (read-your-writes), or for the few views that must be fresh. CAP formalises the trade-off during a network partition; PACELC adds that even without one, you trade latency against consistency.

Example. DynamoDB reads are eventually consistent by default and can be made strongly consistent per request on tables and local secondary indexes, at twice the read cost. Reads from global secondary indexes are always eventually consistent.

In the interview. Do not just say "eventually consistent". Say which reads tolerate staleness (a like count) and which do not (an account balance, a just-saved setting), and route them accordingly.

5. Sharding and partitioning

The problem. The data or the write load outgrows one machine. Replicas do not help, because every replica still takes every write.

How it works. Split the data by a partition key so each shard holds a subset. Hash partitioning spreads keys evenly but makes range queries span every shard. Range partitioning (by date, by alphabetical ID) keeps ranges together but concentrates load on whichever range is hot, such as "today".

Trade-offs. The shard key is the decision that is hardest to undo. A good key spreads both data and traffic, and keeps the data for a typical request on one shard. Problems to raise unprompted:

• Hot keys. One celebrity user or one viral item can overload a single shard however even the data is.
• Cross-shard queries and transactions become scatter-gather calls or are designed out.
• Resharding moves data while serving traffic, which is why the next pattern exists.

Example. Each DynamoDB partition supports up to 3,000 read units and 1,000 write units per second. AWS's documented fix for a hot partition key is write sharding: append a suffix (random, or calculated from another attribute) so one logical key spreads over many partitions, at the cost of querying every suffix and merging when you read.

In the interview. Propose the shard key, justify it from the access pattern, then name its hot-key risk and your mitigation. Raising the weakness before the interviewer does scores well.

6. Consistent hashing

The problem. The naive way to map keys to N servers is hash(key) mod N. When N changes, almost every key moves. Going from 10 servers to 11 remaps about 10 out of every 11 keys, roughly 91%. For a cache, that is a near-total miss storm; for a datastore, it is a massive migration.

How it works. Place servers and keys on the same circular hash space, a ring. Each key belongs to the first server clockwise from it. Adding or removing a server moves only the keys in that server's segment, about 1/N of them, and leaves the rest alone. The idea comes from Karger and colleagues (1997), and Amazon's Dynamo paper is the best-known production description.

Trade-offs. With one point per server, random placement leaves segments of uneven size, so load is uneven. Dynamo's fix is virtual nodes: each physical server takes many points on the ring, which evens out load and lets a bigger machine take more points. You pay with a larger routing table and more complex rebalancing.

Example. Dynamo stores each key on the first N distinct physical nodes clockwise from it, so the ring decides replica placement as well as ownership.

In the interview. Explain why modulo fails in one sentence, draw the ring, and mention virtual nodes. Go deeper only if asked.

7. Queues and asynchronous processing

The problem. Some work is slow (sending email, transcoding video, calling a flaky third party), and doing it inside the user's request makes the request slow and fragile.

How it works. The request handler records the intent as a message on a queue and returns. Workers pull messages and process them at their own pace. The queue absorbs bursts. Failed messages are retried, then moved to a dead-letter queue so one poison message does not block the rest.

Two families are worth distinguishing. Work queues such as Amazon SQS hand each message to one consumer; SQS hides a received message for a visibility timeout (30 seconds by default) and makes it visible again if the consumer does not delete it in time. Logs such as Kafka keep an ordered, replayable stream; events with the same key go to the same partition, and order is guaranteed within a partition, not across partitions.

Trade-offs. You give up an immediate result, so the client needs another way to learn the outcome (polling, a webhook, a push). Ordering is weaker than it looks. And SQS standard queues deliver at least once: AWS's documentation tells you to make consumers idempotent, which leads to the next pattern.

A subtle trap is the dual write: saving to the database and publishing to the queue as two steps, where a crash between them loses an event. The usual fix is a transactional outbox (write the event to an outbox table in the same transaction, publish from there) or change data capture from the database log.

Example. A video upload stores the file in object storage, enqueues a transcode job, and returns "processing" to the user. Workers scale with queue depth.

In the interview. Draw the queue, say what happens when a worker crashes mid-message (redelivery), and say why that is safe (idempotent handler). Mention the dead-letter queue.

8. Idempotency

The problem. Clients retry, queues redeliver, users double-click. If the operation is "charge the card", running it twice is a bug.

How it works. An operation is idempotent when running it twice has the same effect as running it once. Reads and absolute sets ("set status to shipped") are naturally idempotent. Increments and creates are not, so you make them idempotent with a key that identifies the intent:

• The client sends an idempotency key with the request.
• The server records the key atomically with the work it does. A repeat with the same key returns the stored result instead of redoing the work.

Stripe's API is the standard reference: the first result for a key is saved and returned to retries, a reused key with different parameters is rejected, and keys may be pruned after 24 hours.

Trade-offs. The check and the write must be one atomic step, or two concurrent retries both pass the check. Keys need storage and an expiry policy. "Exactly once" in messaging usually means deduplication within a window: SQS FIFO queues drop duplicate sends within five minutes, but a handler that crashes before acknowledging still sees the message again.

Example. A conditional transaction in DynamoDB that writes a guard item keyed by the intent together with the real change, so a duplicate cancels the whole transaction. It is walked through step by step in Idempotent writes in DynamoDB with a guard item.

In the interview. A strong line is: "Over a network we get at-least-once delivery; exactly-once effects come from idempotent processing." Then say what the key is made of.

9. Rate limiting

The problem. One buggy or malicious client can use up capacity meant for everyone.

How it works. Count requests per client (API key, user, IP) and reject the excess with HTTP 429 Too Many Requests, ideally with a Retry-After header. The default algorithm is the token bucket: a bucket holds up to B tokens and refills at R per second, each request takes a token, and an empty bucket means reject. It allows bursts up to B while holding the long-run rate to R. Alternatives are the leaky bucket (smooth output, no bursts), the fixed window counter (simple, but allows double the limit across a window boundary) and the sliding window (fixes the boundary at some extra cost).

class TokenBucket {
  private tokens: number;
  private last = Date.now();

  constructor(
    private capacity: number, // B: the largest burst allowed
    private refillPerSecond: number, // R: the long-run rate
  ) {
    this.tokens = capacity;
  }

  tryTake(): boolean {
    const now = Date.now();
    const elapsed = (now - this.last) / 1000;
    this.tokens = Math.min(
      this.capacity,
      this.tokens + elapsed * this.refillPerSecond,
    );
    this.last = now;
    if (this.tokens < 1) return false; // respond 429
    this.tokens -= 1;
    return true;
  }
}

Trade-offs. This sketch lives in one process's memory. Behind a load balancer the limit must be shared, typically in Redis with the refill-and-take step done atomically (a Lua script), or the limiter accepts some inaccuracy for local speed. Decide whether it fails open (allow traffic if Redis is down) or fails closed.

Example. Amazon API Gateway throttles with a token bucket: the configured rate is how fast tokens are added and the burst is the bucket's capacity. Stripe has described running token-bucket limiters on Redis, alongside load shedders that reserve capacity for critical requests.

In the interview. Say where the limiter sits (at the gateway, before expensive work), what the key is, which algorithm, and what the client sees. "Design a rate limiter" is also a full prompt; a token bucket plus a shared store is the core of that answer.

10. Timeouts, retries and circuit breakers

The problem. A dependency gets slow. Callers wait, their connections fill up, and now they are slow too. Naive retries make it worse by multiplying load on the struggling service.

How it works. Four tools, used together:

• Timeouts on every network call, so a slow dependency cannot hold resources forever.
• Retries with exponential backoff and jitter. Without jitter, clients that failed together retry together. Marc Brooker's analysis on the AWS Architecture Blog found plain exponential backoff performed worst and "full jitter" (a random delay between zero and the current cap) among the best.
• Retry budgets. Google's SRE book recommends capping retries per request (three attempts) and per client (retries below 10% of requests), and retrying at only one layer of the stack, because retries at every layer multiply.
• Circuit breakers. After enough failures, the breaker opens and calls fail immediately without touching the dependency. After a cool-down it goes half-open and lets a trial call through; success closes it, failure opens it again. The pattern was popularised by Michael Nygard's Release It! and is described in Martin Fowler's bliki.

Related is back-pressure: a full component says so (a bounded queue, a 429 or 503) instead of accepting work it cannot finish, and load shedding drops the least important traffic first.

Trade-offs. Retries are only safe for idempotent operations. Timeouts that are too short cause failures under normal load. A breaker needs a fallback: a cached value, a default, a degraded page, or an honest error.

Example. When a recommendations service is slow, the breaker opens and the product page renders without that panel instead of timing out.

In the interview. When you add any synchronous call between services, say "with a timeout, retries with jittered backoff, and a breaker that falls back to X". It shows you have thought about failure, not just the happy path.

11. Fan-out on write vs fan-out on read

The problem. A home feed combines posts from everyone a user follows. Building it is expensive either at read time or at write time.

How it works.

• Fan-out on read (pull). Store each post once. When a user opens the feed, fetch recent posts from everyone they follow and merge them. Writes are cheap; reads are expensive and slow for users who follow many accounts.
• Fan-out on write (push). When someone posts, insert the post ID into a precomputed feed for each follower. Reads become a single fetch; writes cost one insert per follower.

Trade-offs. Push breaks down for accounts with millions of followers, because one post triggers millions of writes. Pull breaks down for heavy readers. The usual answer is a hybrid: push for normal accounts, pull for very large ones, merged at read time.

Example. Twitter's "Timelines at Scale" talk by Raffi Krikorian, as summarised by High Scalability, described a write-based fan-out into per-user home timelines stored in a Redis cluster, capped at 800 entries each. For accounts with very large followings, the plan was to stop fanning out and merge their tweets in at read time. The details have changed since; the trade-off has not.

In the interview. Name both approaches, pick push as the default, and raise the celebrity problem yourself with the hybrid fix. Say the feed stores IDs, not full posts, so an edit does not touch millions of copies.

12. Search indexing

The problem. Databases are good at finding rows by key and bad at "find every document that mentions these words, ranked by relevance". A LIKE '%term%' query cannot use an ordinary B-tree index, so it scans.

How it works. A search engine builds an inverted index: for each term, a list of the documents that contain it. Text is tokenised and normalised (lowercased, stemmed) at index time, and queries the same way. Elasticsearch and OpenSearch are the usual examples.

The database stays the source of truth. The index is a derived copy, fed by a pipeline: the application publishes change events, or a change data capture stream (for example DynamoDB Streams, or Debezium reading a Postgres log) feeds an indexer.

Trade-offs. The index lags the database. Elasticsearch is "near real-time": by default it refreshes every second, and only for indices that have been searched in the last 30 seconds, so a new document takes a moment to become searchable. Schema changes mean reindexing, usually into a new index that an alias is switched to.

Example. An e-commerce catalogue writes products to Postgres, a change stream updates the search index, and the search page queries the index for IDs before loading full details from the database or cache.

In the interview. Draw the database, the change stream and the index as three separate boxes. Say the index is eventually consistent and what the user sees during the lag. Treat autocomplete as a separate prefix-lookup problem.

What did not make the list

Back-of-the-envelope estimation is essential, but it is how you choose between patterns, not a pattern. Unique ID generation is usually a sub-question inside a larger design. Consensus and leader election matter, but the interview answer is usually "use a store that already implements Raft or Paxos". Object storage with pre-signed uploads is one box on the diagram, not a discussion.

The patterns in one answer

Take the classic prompt, "design a URL shortener":

• Redirects vastly outnumber creations, so cache them and put a CDN in front. A 301 is cached by browsers, which helps latency but hides repeat clicks from analytics; a 302 keeps every click visible.
• Stateless redirect servers sit behind a load balancer.
• The mapping table is sharded by a hash of the short code.
• Creation is rate limited per user or IP to stop abuse.
• Click events go onto a queue, so a slow analytics store never slows a redirect.
• The create endpoint accepts an idempotency key, so a retried request does not mint two codes.

None of those choices is clever alone. The score comes from tying each to a requirement and naming its cost.

Frequently asked questions

How many patterns do you need to know for a system design interview?

Fewer than most prep lists suggest. The twelve here (load balancing, caching, CDNs, replication, sharding, consistent hashing, queues, idempotency, rate limiting, retries and circuit breakers, fan-out, and search indexing) cover the building blocks of almost every common prompt. Knowing their trade-offs well beats knowing forty patterns by name.

Should I bring up the CAP theorem in a system design interview?

Only when it changes a decision. CAP is about what a replicated system does during a network partition. It is more useful to say which reads must be strongly consistent and which can be stale, and what that costs in latency, than to label the whole design CP or AP.

What is the difference between fan-out on write and fan-out on read?

Fan-out on write copies each new item into every follower's precomputed feed when it is created, so reads are cheap and writes are expensive. Fan-out on read builds the feed at request time by querying everyone the reader follows, so writes are cheap and reads are expensive. Large systems usually mix the two.

Do interviewers still ask about consistent hashing if I would use a managed database?

Often, yes. Managed stores such as DynamoDB partition data for you, but distributed caches and custom routing layers still need a way to map keys to nodes that does not reshuffle everything when a node is added. Explaining why hash modulo N fails is a quick way to show you understand the problem.