03 — A Framework For System Design Interviews

A system design interview is a forty-five minute conversation in which an interviewer names a product in one sentence and you describe the machines, the storage, and the network calls that would make it work at some scale. Nothing is compiled and nothing is run. The artifact is a diagram plus the arithmetic you speak aloud while drawing it.

There is a procedure for this: a minute-by-minute script that turns the one-sentence prompt into numbers, and the numbers into an architecture you can defend against a competing one. It covers all forty-five minutes: restate the ask, ask the five questions whose answers change the design, size the system, draw a diagram whose every arrow carries a rate, choose your own deep-dive topics, and name the first thing that breaks. It also identifies which assumptions are load-bearing — the ones where being wrong does not cost you a correction, it costs you the whole design.

The inputs and outputs are these.

IN     one sentence:  "Design a photo feed."
       plus whatever the interviewer will tell you when you ask

OUT    stated assumptions      10M daily users · 200:1 reads:writes · 2 MB photos
       per-second rates        4,630 reads/s peak · 23.2 writes/s peak
       an API and a data model
       one diagram             every arrow labelled with one of those rates
       two defended tradeoffs  "fan-out on write, and here is what it costs me"
       one named bottleneck    "the fan-out queue, at seven hours"
                               (fan-out = one post becoming many writes; defined below)

The round does not measure whether you know the components. It measures whether you can turn a one-sentence prompt into a set of numbers, and then defend one architecture against another using those numbers. A candidate who draws the correct diagram without saying where the traffic goes scores below a candidate who draws a simpler one and can tell you which box saturates first.

Everything below is worked on a single running example, a photo feed with ten million daily users, so the numbers accumulate instead of resetting each section. Chapters 04 onward apply the same procedure to other systems.

Which round is this

Before you spend forty-five minutes on the wrong scoring sheet, check that you are in the round this chapter describes. “Design X” is asked in three different interviews, and they reward three different things.

Read this chapter’s column top to bottom; the other two are there so you can tell which round you are in. Several cells use terms defined immediately after the table.

The five terms in this chapter’s column

Four cells in the left-hand column are terms of art, and one of those cells holds two. Here are all five in plain words:

• Topology — the set of boxes and the arrows between them. Which services exist, and who calls whom.
• Egress — the cost of bytes leaving your data centre toward users. For anything that serves images or video this is usually the largest line on the bill.
• Fan-out — one incoming event causing many outgoing writes or reads. One post landing in a million followers’ feeds is fan-out, and it is the defining difficulty of this chapter’s example.
• Invariant — a statement that must be true after every operation, no matter what. “A user never sees a post from someone they unfollowed” is an invariant.
• Partition — the situation where the network splits and two halves of your system can still run but cannot talk to each other. This is precisely when invariants break.

The words in the other two columns

You do not need these to run this chapter’s round, only to read the table. One line each:

• Operating point (ML) — the precision/recall tradeoff a model commits to once: the threshold it actually ships at, rather than the curve it could have shipped anywhere on.
• Label latency (ML) — how long after a prediction the ground truth about it arrives. It decides whether the model can be retrained daily or annually.
• Feature fetch (ML) — retrieving a model’s inputs at serving time. It is that round’s dominant cost, the way egress is this one’s.
• Control loop (agent) — an agent’s observe-act-check cycle.
• Calls per task (agent) — how many model invocations one user request costs.

Each is developed in the chapter its column links to.

In short: the classic round is scored on the tradeoff, the ML round on the objective, the agent round on the loop.

Four habits that score in all three rounds

These are not repeated later in the chapter.

1. Narrate the decision you are weighing rather than working in silence.
2. State an assumption out loud the moment you make it, rather than letting it stay implicit.
3. Never bluff. “I have not built one, here is how I would reason about it” scores far above a confident wrong mechanism.
4. Correct your own mistakes immediately and out loud. Catching it yourself is worth more than never having made it.

Chapter 10 develops all four at length if you want the longer treatment.

What this chapter assumes you can look up

Some follow-up questions in this round drill into how a single database works: how an index finds a row, what a transaction actually guarantees, how far behind a replica can fall, how to pick the column you split a table on. Those are derived in sql 03 — Database Internals, and this chapter links to the specific sections when they come up.

Two more chapters sit underneath this one. The step-by-step growth path from one server to many is 01 — Scale From Zero To Millions. The estimation drills that make the arithmetic below automatic are 02 — Back Of The Envelope. Cite them; do not re-derive them on the clock.

The four steps, with the clock

The round has four steps, preceded by a two-minute step 0, each with a fixed number of minutes and a specific thing it is scored on. The minute boundaries are worth memorizing, because every remaining section of this chapter is written against them.

The diagram below is a timeline, not a data flow. Each box is one step; the second line is how many minutes it gets and the clock range it occupies, so at any point in the round you can check which box you are supposed to be in.

flowchart TD
    S0["0 · Restate the ask<br/>2 min · 0-2"] --> S1
    S1["1 · Requirements + envelope<br/>8 min · 2-10<br/>functional · non-functional · numbers"] --> S2
    S2["2 · High-level design<br/>12 min · 10-22<br/>API · data model · boxes · buy-in"] --> S3
    S3["3 · Deep dive<br/>18 min · 22-40<br/>two topics, you choose them"] --> S4
    S4["4 · Wrap<br/>5 min · 40-45<br/>bottleneck · failure · what I would revisit"]

    style S1 fill:#1d3557,color:#fff
    style S3 fill:#2d6a4f,color:#fff
    style S4 fill:#bc6c25,color:#fff

In words: two minutes restating the ask, eight on requirements and the envelope, twelve on the high-level design, eighteen on two deep dives you choose yourself, five wrapping up.

“Envelope” there means the back-of-the-envelope arithmetic that sizes the system — a handful of multiplications you do out loud, on round numbers, to turn the product description into rates and bytes.

The budget mirrors the scoring sheet. The right-hand column of the next table is what each block earns points for, and it differs by block:

2 + 8 + 12 + 18 + 5 = 45

Deep dive is the longest block, so everything before it is setup. A common mistake inverts this: twenty-five minutes of requirements and boxes, then eight rushed minutes on the only part that differentiates candidates. If you are at minute 22 and still drawing, stop drawing.

Inside each block there is a second layer of budgeting that stops one block from eating the next. The table below splits each block’s minutes across what happens inside it. Do not time these sub-blocks on the clock; just notice when one runs long.

Step 0 — Restate the ask (2 min)

Say the problem back in your own words, including the parts you intend not to build. Two minutes here is the cheapest way to catch that you are about to design the wrong system.

Give one sentence back, and make it carry the scope you are excluding.

“So: a photo feed. I am reading that as the read path — a user opens the app and gets a ranked list of recent posts from people they follow — plus the write path for creating a post. I am excluding ranking quality, comments, and direct messages unless you want them. Sound right?”

The read path is everything that happens when a user asks for data, and the write path is everything that happens when a user produces it. Splitting the system into those two halves in your first sentence is a habit worth having, because almost every decision later in the round applies to one half and not the other.

The exclusions are the valuable half. “I am excluding ranking quality” is what lets you spend eighteen minutes on fan-out, and an exclusion the interviewer accepts in minute one cannot be held against you in minute forty.

Step 1 — Requirements and the envelope (8 min)

These eight minutes decide the design, even though nothing has been drawn yet. Four things happen in them:

1. Split the requirements into two kinds, and notice that only one kind shapes the architecture.
2. Ask the five questions whose answers fork the design.
3. Convert those answers into per-second rates and bytes.
4. Write the assumptions down in a form that tells you which ones you cannot afford to get wrong.

The split, and why it is not bookkeeping

Requirements come in two kinds, and the boring-sounding second kind is the one that determines your architecture.

The three pieces of shorthand in that table

All three recur for the rest of the chapter.

p99 is the 99th-percentile latency. Sort every request in a time window by how long it took, then read off the one that beat 99% of the others. “p99 under 200 ms” means 99 out of 100 requests finish inside 200 milliseconds and the slowest 1 may be worse. Use percentiles rather than averages, because an average hides the slow tail that users actually complain about.

Availability is the fraction of time the system answers at all, so 99.9% is a promise about downtime. It is worth knowing what that promise costs, because interviewers ask. Convert the percentage into minutes per year:

minutes in a year        365 x 24 x 60         =  525,600
99.9%   allows           525,600 x 0.001       =      525.6   minutes down/year
99.99%  allows           525,600 x 0.0001      =       52.56

525.6 minutes is a little under nine hours a year. 52.56 minutes is under an hour. Each extra nine you add cuts the allowance by 10x and costs roughly ten times the engineering.

Retention is how long you keep data before deleting it. It is the multiplier that turns a write rate into a storage bill, and it is question 4 below.

Why the second column is the one that matters

Functional requirements decide what you store. Non-functional requirements decide how many machines it takes and where the data lives.

A photo feed with a 200 ms p99 and one with a 2 s p99 are the same feature and different systems. That is the reason to spend your remaining requirements time on the second column.

Writing the functional list

Write functional requirements as verbs the user performs, never as components. Listing “a load balancer, a cache, a queue” is naming your answer before you have the question. Listing “create a post, read a feed, follow a user” is a scope agreement.

The list below is the running example’s, in the compressed form you would actually write on a whiteboard. IN means in scope and OUT means explicitly not. “Reverse chronological” means newest first, “paginated” means the feed arrives in fixed-size pages rather than all at once, and DMs are direct messages:

IN     create post (image + caption)
       read feed (reverse chronological, paginated)
       follow / unfollow a user
OUT    ranking, comments, DMs, stories, search

The five questions that change a design

You have roughly five minutes of question time, and these are the five questions worth spending it on. The test for a good question is whether the answer appears in an expression you write within the next sixty seconds. These five each fork the architecture: two different answers lead to two genuinely different diagrams.

Every term in the table is defined below, alongside the arithmetic its answer unlocks. The right-hand column is what changes on the whiteboard depending on how the interviewer answers.

Each question below is followed immediately by the arithmetic its answer unlocks, on the running example.

Question 1 — Read:write ratio

This is the number of reads the system serves for every write it accepts.

You almost never have to ask for it directly. You get it by asking how many people use the product and how often each one reads and writes, then dividing. DAU is daily active users: the count of distinct people who open the app on a given day.

Here is the division on the running example. The top three lines are the inputs — one you assume, two you ask for — and the rest is arithmetic:

DAU                                          10,000,000
feed opens per user per day                          20
posts per user per day                              0.1

feed reads/day    10,000,000 x 20   =   200,000,000
posts/day         10,000,000 x 0.1  =     1,000,000

200,000,000 / 1,000,000 = 200

200:1. That single number decides the next twenty minutes. The read path gets the cache, the replicas, the denormalization and the fan-out. The write path can be a synchronous insert into one primary.

Three of those words are jargon and all three recur below:

• Replica — a second copy of the database that stays in sync with the authoritative copy (the primary) and can answer reads but not writes.
• Denormalization — storing the same fact in more than one place so a read does not have to join several tables to reassemble it. You pay for it with extra writes and the risk of the copies disagreeing.
• Synchronous — the user’s request waits for the work to finish. The opposite is asynchronous, where the request returns immediately and the work happens later.

Why DAU is the weak half of that question: look at what cancelled in the division. Reads are DAU x 20 and writes are DAU x 0.1, so the ratio is (DAU x 20) / (DAU x 0.1), which is 20 / 0.1. The DAU divides out, so the ratio is a fact about one user’s behaviour and carries no information about scale.

That is why “how many DAU?” is a weak question and “how often does one person read, and how often do they post?” is a strong one. The first sizes the fleet. The second forks the design.

Where the ~10:1 and ~100:1 thresholds come from

They are one decade either side of a crossover you can compute in four steps. Two architectures are competing, and both are developed at length in Step 3:

1. Fan-out on write — copying each new post into every follower’s stored feed — costs writes/s x followers.
2. Read-assembly — merging a user’s followees’ recent posts when the feed is opened — costs reads/s x followees.
3. The average user’s followers and followees are the same number, because every follow edge is counted once from each end. So that number cancels when you divide one cost by the other, and what is left is reads/s / writes/s — the read:write ratio itself.
4. Therefore the raw crossover sits at 1:1, with fan-out the cheaper side above it.

The decades on either side are margin, not arithmetic. A queued cache write and a latency-budgeted index seek are within about 10x of each other in cost, so a 10x gap in raw counts is the smallest gap that survives any reasonable weighting of one against the other.

Below 10:1 the write path is within an order of magnitude of the read path and stops being cheap. Above 100:1 the read path outnumbers it so heavily that a cache is not an optimization you add but the thing that makes the numbers work at all.

Inside the band both designs are defensible and the tiebreaker is the latency budget rather than the ratio. The band is wide, and it is where most real problems land.

Why chapter 01 uses a different ratio for the same product

The two chapters that use this example disagree on purpose, and quoting the wrong one is a real way to lose the round.

Ch 01 sizes a social feed at 2 writes per user per day — 20 requests of which 10% are writes, so 9:1. This chapter uses 0.1 posts against 20 opens, so 200:1.

Both are defensible and each is load-bearing in its own chapter. 9:1 and 200:1 sit on opposite sides of the ~10:1 line in the threshold card below, and the two chapters build the architectures that line predicts: ch 01 caches computed timelines and never fans out on write; this chapter fans out on write and treats the read path as the system. Ch 01’s higher write rate is what forces its memory ladder — the hot set, the post corpus, the replica-apply wall — and 0.1 posts/day would collapse all three.

The mistake is not picking one; it is quoting one chapter’s ratio while drawing the other’s diagram.

Convert to per-second rates immediately

A per-day number sizes nothing. Machines are bought against a rate. There are 86,400 seconds in a day (60 x 60 x 24), so divide by that, then apply a peak multiplier:

reads/s avg    200,000,000 / 86,400  =  2,315
writes/s avg     1,000,000 / 86,400  =  11.6

peak multiplier (diurnal, single-region consumer app)   2x
reads/s peak         2,315 x 2       =  4,630
writes/s peak         11.6 x 2       =  23.2

The peak multiplier accounts for traffic not being flat. A consumer app in one time zone is busy in the evening and quiet at 4 a.m., which is what diurnal means. Doubling the daily average is the standard rough allowance for that. Size machines against the peak, never the average.

At 23.2 writes per second at peak and 11.6 writes/s on average, the write path is a single unsharded relational database — one PostgreSQL primary, meaning one machine holding the whole table rather than the table split across many — and it stays that way for the whole two-year horizon and well past it. Rejecting complexity on a number is worth more than adding it.

Label every rate: offered load or capacity

Every one of those numbers is offered load — what the workload demands. Not one of them is capacity, which is what a given set of machines can supply. That distinction is cheap to make here and expensive to recover later, so make it in the same breath as the arithmetic:

State whether a rate is offered load or service capacity at the moment you derive it, and never divide one by the other without saying which is which.

The error this rule prevents is deriving a rate as demand and then spending it as spare throughput. It always flatters the design, because a fleet sized at exactly its peak offered load has zero spare. “How long does this extra work take” with a demand figure in the denominator is a question about a fleet that does not exist. It comes up twice in the fan-out deep dive below, and it is the most common arithmetic mistake in this round.

Question 2 — Consistency, per field

Consistency here means: after a write succeeds, when is every reader guaranteed to see it? Strong consistency means immediately. Eventual consistency means after some delay. The design question is how much delay each piece of data can tolerate, and that tolerated delay is called staleness.

The useful version of this question is never “is the system CP or AP.” (That phrasing refers to the CAP theorem, which says that when the network splits a system in two you must choose between staying consistent and staying available. CP picks consistency, AP picks availability.) Asking it at the whole-system level produces a slogan. Asking it per field produces a design.

Here is the answer for the running example. Each row is one piece of data; the middle column is how out of date it is allowed to be, and the right-hand column is the machinery that number buys:

One row needs unpacking. Read-your-writes is the guarantee that a user always sees their own most recent write even if everyone else sees a slightly older state, and the cheap way to provide it is to route that one user’s next few reads to the primary instead of a replica. It is the single most valuable consistency guarantee per unit of cost, because the person most likely to notice staleness is the person who just caused it.

A system is not one consistency level; it is one per field, and the expensive one is the privacy check. If unfollowing someone leaves their posts visible to you for 30 seconds, that is a product bug. If a like count is 30 seconds stale, nobody notices.

The design consequence is concrete and splits in two. The follow graph gets synchronous invalidation — deleting the cached copy at the same moment the underlying data changes, so the next read is forced to go to the source. The counters get an asynchronous aggregator that sweeps up recent likes on a timer.

If you want the database-level machinery underneath this, transaction guarantees are Transactions acid precisely and the specific ways a lagging replica turns into a correctness bug are sql 03 — Replica lag is a correctness problem.

Question 3 — Latency, against the speed of light

A round trip time (RTT) is how long a packet takes to get to the far end and back. Between continents it is set by the speed of light in fibre plus routing overhead, and no amount of engineering removes it.

The standing constant is a cross-region RTT of 70-150 ms. Put the worst case against the target, and count how many trips a single page needs:

p99 budget, end to end                        200 ms
cross-region RTT, worst case                  150 ms
sequential round trips on a feed page              2   (feed ids, then hydrate)

2 x 150 = 300

“Sequential” is doing the work in that expression: two round trips that must happen one after the other, because the second depends on the first. Fetching the list of post identifiers tells you which posts to hydrate — to fetch the actual bodies and images for — so you cannot start the second call until the first returns, and the two latencies add. Round trips that can run in parallel do not add; they cost whatever the slowest one costs.

That is 300 ms of network against a 200 ms budget, before a single byte is read off a disk. It leaves three ways out, and you pick one out loud:

• The target is regional, in which case state it: “p99 200 ms for users in the same region as their data.”
• You put read replicas near the user, so the trip is no longer cross-region.
• The feed is precomputed and served from an edge cache — a copy of the response held in a data centre physically close to the user.

That is an architectural decision derived in fifteen seconds from two numbers, and it is among the highest-value moves in step 1.

Question 4 — Retention

How long you keep each thing. This is the question that turns a write rate into a storage system, because bytes per write x writes per day x days kept is the entire storage estimate.

Run it first on the photos. The three inputs are on top; note that 2 MB is written out as 2,000,000 bytes so the multiplication stays visible, and 730 is two years of days:

photos/day                                          1,000,000
bytes per photo                                     2,000,000
days in 2 years                                           730

media bytes/day    1,000,000 x 2,000,000  =  2,000,000,000,000  B
media TB/day       2,000,000,000,000 / 1e12          =        2  TB
media PB in 2 y    2 TB x 730 = 1,460 TB / 1,000     =     1.46  PB

That is 1.46 petabytes of media over two years.

Now run the identical three lines on the metadata — the small structured record that describes each post: who wrote it, when, the caption, and a pointer to where the image lives. Call it 500 bytes per post:

metadata bytes/day     1,000,000 x 500     =  500,000,000  B
metadata GB/day        500,000,000 / 1e9   =          0.5  GB
metadata GB in 2 y     0.5 GB x 730        =          365  GB

Compare the two results. 1.46 PB is 1,460,000 GB, so 1,460,000 / 365 = 4,000.

The media is 4,000x the metadata, so they are two different storage systems; say so before you draw anything. Each half goes somewhere different:

• Metadata: one database. 365 GB over two years fits on one machine’s solid-state disk with room to spare. This is why the photo feed’s database is not the hard part.
• Media: an object store behind a CDN. An object store is a service like Amazon S3 that holds whole files addressed by a key, cheaply and in unlimited quantity, but with no indexes, no joins and no transactions. A CDN — content delivery network — is a fleet of caches spread around the world that serve those files from wherever the user is. The only thing the database then holds is a URL pointing into the object store.

One more table is worth sizing, because it is by far the database’s largest by row count and candidates rarely mention it: the follow graph itself. At 200 follows per user,

follow edges       10,000,000 x 200          =  2,000,000,000  rows
payload at 24 B    2,000,000,000 x 24 / 1e9  =             48  GB
with both indexes (roughly 2x the payload)   =            ~96  GB

The 24 bytes is three 8-byte columns per row — follower id, followee id, timestamp — matching the follows table in the data model below. The doubling accounts for the two indexes that table carries: an index is a second sorted copy of the columns it covers, so it costs storage as well as write time.

Two billion rows sounds like a sharding problem and is not. About 100 GB alongside the 365 GB of post metadata is 365 + 96 = 461 GB, under half a terabyte, which is still one machine. (The 10M there is DAU, so read the row count as a floor — registered accounts outnumber daily actives.)

Say the row count out loud anyway. “Two billion rows on one box” is the claim an interviewer will push on, and having the byte figure ready is the difference between a defended answer and a lucky one.

Question 5 — Now versus two years

Are you designing for today’s load, or for the load at the end of a growth curve? The answer changes which parts you are allowed to keep simple.

Growing 3x per year for two years is 3 x 3 = 9x, so apply that to today’s figures:

DAU today                                   10,000,000
growth                                              3x per year
DAU in 2 years        10,000,000 x 9    =   90,000,000
reads/s peak in 2 y        4,630 x 9    =   41,670

Ten million daily users becomes ninety million, and the peak read rate goes with it.

Now test that endpoint against a physical constant of the machines themselves. A NIC is a network interface card, the port through which a server sends bytes, and a common one moves 1 gigabit per second. A feed response is about 20 KB, written below as 20,000 bytes. Three steps: bytes per second, then bits per second (8 bits to a byte), then divide by what one NIC supplies.

bytes/s      41,670 x 20,000            =  833,400,000
bits/s      833,400,000 x 8             =  6,667,200,000
NICs at 1 Gbps
6,667,200,000 / 1,000,000,000 = 6.67

That is seven machines of pure NIC just to push feed bytes, before any compute. The number forces two properties on the feed service from day one:

• Horizontally scalable — you add capacity by adding more identical machines, rather than buying a bigger one.
• Stateless — no machine remembers anything between requests, so any request can go to any machine and losing one costs nothing.

The metadata database needs neither: at 208.8 writes/s peak in two years (23.2 x 9) it can stay single-primary for the whole window. Design the part that has to change; do not design the part that does not.

What interviewers probe: whether a question’s answer moves your next sentence. “How many DAU?” is only a good question if the number appears in an expression within sixty seconds. If you ask it and then draw a load balancer — the box that spreads incoming requests across a fleet of identical servers — without using it, you asked for the ritual.

The five assumptions every design rests on

A design is not a diagram; it is a set of assumptions with a diagram attached, and the diagram is only correct relative to them. There are five, each with a default so you are never stuck, and they sort into the ones you state and move past and the ones you stop and ask about.

The five are the same in every classic system design round, whatever the product. The last column is what you say when the interviewer will not give you a number — you are allowed to invent it, as long as you say you are inventing it.

The five letters map onto the five questions one for one, and the ask-order table below is written in letters, so learn the mapping now rather than reconstructing it on the clock.

• Question 1 (read:write) sets C, and through the two usage numbers behind it, half of A.
• Question 2 (what may never be stale) sets E.
• Question 3 (p99 and where the users are) sets D.
• Question 4 (retention) sets B.
• Question 5 (now versus two years) sets the growth half of A.

E is the one candidates skip, so unpack it into its three separate promises.

• Availability — how often the system answers at all. 99.9% buys you the 525.6 minutes a year computed above.
• Staleness — how out of date an answer may be. Per field, not per system: the like count may lag ten seconds, the follower list may lag zero.
• Durability — whether an accepted write can ever be lost.

Durability is a different promise from availability, and mixing them up is common. A system that cannot serve your post for a minute is an outage. A system that loses your post is a scandal. Being unable to read for sixty seconds and having the data gone forever are answered by completely different mechanisms, so name which one you are promising.

Which assumptions are load-bearing

An assumption is load-bearing when being wrong about it does not cost you a correction — it costs you the design. A one-line test tells the two apart, so that under time pressure you know which assumptions are worth a question and which you can declare.

First, the threshold card

The test needs something to compare against, so write this card first. Every number in it has already appeared above, or is cited from a chapter you have read. None of it requires knowing the alternative architecture in advance, which is the whole point — you can run the test cold on a product you have never seen.

Read the card as three columns: the quantity, the value at which it changes character, and what crosses over when it does. You are looking your own numbers up in the middle column.

THRESHOLDS — the published ones. A number that stays on one side of all of
these moves the bill. A number that crosses one moves something you can name.

  read:write ratio     ~10:1  ·  ~100:1     below 10:1 the write path is the
                                            system; above 100:1 caching is the
                                            design rather than an addition
  one cross-region     150 ms worst case    a p99 budget under 2 x 150 ms forces
    round trip                              regional replicas or edge precompute
  one upload in one    ~19 MB               = 5 Mbps mobile uplink x 30 s; past
    HTTP request                            it, chunked resumable upload and an
                                            upload-session store
  one NIC              1 Gbps               = 6,250 x 20 KB feed responses/s;
                                            past it, more identical machines
  one machine's RAM    128 GB               past it, the cache is a cluster
  one primary          ~2,000 writes/s      past it, a shard key or another store
  one machine's SSD    a few TB             past it, the bytes leave the database
                                            for an object store

The last four rows are machine constants rather than facts about this product. The commodity box is Numbers worth memorizing cold’s 64-256 GB of RAM and 1-10 Gbps of network, and the primary’s ceiling is usually single-threaded replica apply, which ch 01 puts near 2,000 writes/s.

This chapter takes the pessimistic 1 Gbps end of that NIC range and says so out loud. Silently picking an end of a range is how two people reach opposite fleet sizes from the same traffic figure.

The test itself

Move the assumption an order of magnitude in each direction and ask whether the set of boxes changes, or only the number of machines inside them. If only the count changes, state it and keep going. If a box appears or disappears, stop and ask.

Answering that question is mechanical, in two steps:

1. Re-run the one expression the assumption feeds, at 10x and at 0.1x, and write down both new numbers. Nothing else in the design moves; you are redoing arithmetic you already did once.
2. Look each new number up in the threshold card. Same side of every line: the assumption is not load-bearing, so state it and go. Crosses a line: read off what that line forces. More identical machines is still only a count, so state it and go. A new component, a shard key, or a different store is a box, and that is the one you stop and ask about.

Two footnotes make the test cover the cases the bare version does not.

Not every assumption has an order of magnitude. A promise like “the visibility check may never be stale” is on or off, and one tenth of zero is still zero. For those, the move is relax it completely ↔ tighten it to zero, and step 1 is unchanged: run the expression at both ends.

A move can cross a threshold at one end only. That is a real third verdict, not a fudge. Say which end, because it tells the interviewer exactly which correction would force a redraw and which would not.

The test run on all five

The table below is the two steps applied to each assumption in turn, using numbers already derived above. Column 2 is the 10x move, column 3 is step 1 and step 2 together — the re-run expression and what the card says about it — and column 4 is the verdict.

The followers row, worked

Run this one out loud even though nobody asks you to: it is the number the entire fan-out dive divides by, and it turns out not to be load-bearing.

Follower count multiplies both sides of the comparison — writes/s x followers against reads/s x followees — and the average user’s followers and followees are the same number, so it cancels. The table below moves it over two decades. The last column does not move:

followers    fan-out writes/s peak    read-assembly seeks/s peak    ratio
     20      23.2 x    20 =    464    4,630 x    20 =     92,600    200x
    200      23.2 x   200 =  4,640    4,630 x   200 =    926,000    200x
  2,000      23.2 x 2,000 = 46,400    4,630 x 2,000 =  9,260,000    200x

The fan-out-versus-read-assembly ratio is not an independent number: it is the read:write ratio again. That is why row C above is the load-bearing one and this one is not. Getting the follower count wrong by 10x costs you a fleet size and no boxes.

What follower count does decide is the burst — 2,000 writes cannot be spread out and 30 million certainly cannot — and that is a tail-latency problem taken up in beat 5 of the deep dive, not a choice between architectures.

The verdict on this shape, and why not to memorize it

Three of the five are load-bearing here, and they are the three that decide topology rather than machine count: the read:write ratio, the latency budget against geography, and the strictest staleness requirement. Traffic volume, retention and follower count move the bill and the fleet size; they do not move the boxes.

That is the ranking for a fan-out shape. Running the test, rather than remembering the ranking, is what makes the next subsection work when you have ninety seconds of question time and five things you would like to know.

Do not carry the ranking to the next product; carry the test.

Take a large-object shape instead: a video service, same 10M daily users, one 20 MB clip uploaded per user per ten days, each clip watched 200 times, so the same 200:1 ratio. Below, the same two steps run cold against the same card, on the two assumptions that swapped places. Each block reads move, then expression, then what the card says, then verdict:

B  data size    2 MB  <-> 200 MB per clip, around a 20 MB centre
   expression   upload seconds = bytes x 8 / 5 Mbps  ->  3.2 s <-> 320 s
   card         the 2 MB end falls back under ~19 MB; the 20 MB centre and
                the 200 MB end are both past it
   verdict      LOAD-BEARING. Chunked resumable upload, an upload-session
                store and a resume endpoint exist at 20 MB and do not at 2 MB

C  read:write   20:1  <-> 2,000:1
   expression   egress = views/day x 20 MB  ->  0.4 PB/day <-> 40 PB/day
   card         crosses the NIC line at every point, in both directions
   verdict      NOT load-bearing. The bytes go client -> object store -> CDN
                without touching your services at any ratio; crossing the NIC
                line buys edge machines, which is a count. A bigger invoice,
                the same diagram

The two shapes invert. On the photo feed, B is load-bearing only at its top end and C decides everything. On video, B decides everything and C decides only the bill.

The five assumptions are the same in every classic round; which of them are load-bearing is a property of the product, not of the list. Memorizing “read:write is the number that decides everything” is how you spend your single question badly on a product where it is not.

The failure mode the test prevents is spending your entire question budget on “how many users?” — the least load-bearing question on the list — and then discovering at minute thirty that the interviewer wanted a globally distributed system, which is a fork you could have found by asking where the users are.

The assumption move

You will not get answers to all five, and you should not try. The executable version is: what to ask in the order that matters, what to declare without asking, and the exact sentence that buys you permission to proceed.

Ask in this order, and stop when the clock says stop. You never ask for C, because C is not a number anyone has; it is derived from A. Ask instead for the two usage numbers, opens and posts per user per day, and divide them yourself. If you get a second question, add D. A third, add E. Declare the rest: they are yours to set, and nobody will fault a stated default.

Each row of the table below is one budget. Find the row matching how many questions you think you will get, and read across: what you ask, what you work out from the answer, and what you simply assert.

That order is the fan-out shape’s. Run the test above before you commit to it — on a large-object shape B moves to the front, and the question you cannot afford to skip is “how big is one of these things.”

Then write the card in the corner of the whiteboard and leave it there. Every later decision points back at a row, and when the interviewer corrects one you change that row visibly instead of quietly re-deriving.

Here is the finished card for the running example. The right-hand tag on each line records where the number came from — STATED means you invented it, ASKED means the interviewer gave it to you, DERIVED means you computed it — and the arrows mark the three rows the test above found to be load-bearing:

A  traffic   10M DAU · 3x/yr growth · 2x peak                        STATED
   usage     20 reads · 0.1 writes per user/day                      ASKED
B  data      2 MB photo · 500 B row · 2-year retention               STATED
C  r:w       20 / 0.1 = 200:1                        DERIVED  <- LOAD-BEARING
D  latency   p99 200 ms, same region                   ASKED  <- LOAD-BEARING
E  failure   99.9% · seconds stale, except visibility  ASKED  <- LOAD-BEARING

Say it as one sentence and move:

“Assuming 10M DAU growing 3x a year, 200:1 read:write, p99 200 ms same-region, media in an object store with a 2-year retention, and everything except the follow-graph visibility check may be seconds stale. Tell me if any of those are wrong; otherwise I am building to them and I will flag where each one is load-bearing.”

“I will flag where each one is load-bearing” is the part that scores, because it promises the interviewer that a wrong assumption produces a local fix rather than a restart. Deliver on it later with one clause at the moment each assumption is spent — “this is where the 200:1 is doing the work; at 5:1 I would assemble on read instead” — which takes three seconds and converts a guess into a bounded guess.

When a correction lands, say which row moved and which boxes move with it. If the interviewer says the ratio is 5:1, the response is not a nod: it is “then the write path is the system, the fan-out cache comes out, and reads assemble from the follow graph — give me thirty seconds to redraw.” A load-bearing assumption corrected out loud is a recovery. The same correction absorbed silently is a design that no longer matches its own arithmetic.

Step 2 — High-level design (12 min)

Draw in a fixed order: the interface first, then the data it moves, then the boxes, then a checkpoint.

Four artifacts, in that order, because each one constrains the next. The API tells you what queries exist. The queries tell you what the tables and indexes must be. Only then do you know which services and stores have to appear on the whiteboard.

API sketch (2 min)

Write the endpoints — the individual operations a client can call over the network — with only the parameters that matter. This is a sketch, not a specification.

The three endpoints below match the functional list from step 1. The arrow separates what the client sends from what it gets back:

POST /v1/posts          {image_upload_id, caption}     -> {post_id}
GET  /v1/feed?cursor=&limit=20                          -> {items[], next_cursor}
POST /v1/follows        {target_user_id}                -> 204

POST, GET and the 204 are HTTP conventions: POST creates something, GET retrieves it, PUT (which appears in the diagram below) writes an object to a known location, and 204 is the response code meaning “it worked, and there is nothing to send back.”

Three details in that sketch are where an interviewer probes, and each is worth a sentence out loud.

Cursor, not offset. Paging through a feed with OFFSET 10000 makes the database walk past and throw away ten thousand rows before returning the ones you wanted, and that cost grows with how deep the user has scrolled. A cursor is instead a marker saying “resume after this exact post,” so the database jumps straight to the position in its index. That is the difference between a page fetch whose cost grows linearly with the offset and one that stays effectively constant — O(n) against O(log n) in the standard notation, where n is the number of rows skipped. Why an index makes that jump cheap is B trees why a lookup is four page reads.

Uploads do not go through your API. A presigned URL is a temporary, signed link that lets the client write one specific object directly into the object store without your servers ever touching the bytes. The client uploads 2 MB straight there. Otherwise 11.6 writes/s x 2 MB = 23 MB/s of upload traffic transits your API tier for no reason, and every one of those machines is now sized by photo bytes instead of by requests.

Idempotency key on every write. Idempotent means doing the operation twice has the same effect as doing it once. Networks make retries unavoidable — a client that never sees your response cannot tell “the post failed” from “the reply was lost” — so a retried POST must not create a second post. The fix is one header carrying a unique key the server remembers, and it removes an entire class of failure for essentially nothing.

Data model (2 min)

Write the tables, their keys, and — next to each one — the actual read query it exists to serve. The access pattern is the point; a schema with no read query next to it is a guess.

Two terms before the block. A primary key (PK) is the column or columns that uniquely identify a row. An index is an extra sorted structure that makes lookups on some other column fast, at the cost of extra work on every write.

In the sketch below, the arrow comments are the point: each one is the question that index exists to answer. Read those first, then the columns.

users(user_id PK, handle, created_at)
posts(post_id PK, author_id, media_url, caption, created_at)
    index (author_id, created_at DESC)      <- "posts by an author, newest first"
follows(follower_id, followee_id, created_at)
    PK (follower_id, followee_id)           <- "who do I follow"
    index (followee_id, follower_id)        <- "who follows me" (the fan-out query)
feed_cache(user_id, post_id, created_at)    <- materialized, if fan-out on write

The last line says materialized, which means the feed is computed in advance and stored, rather than assembled from the follow graph at read time. Writing it into the model now, with the “if” attached, tells the interviewer you know it is a choice and that you are about to defend it in step 3.

Name the choice you are making and the number behind it. If the follow-up is about sharding — splitting one table across many machines, where the shard key is the column that decides which machine a row lands on — the selection rules and the cost of changing your mind later are Choosing a shard key.

The diagram (6 min)

Now the boxes, and every arrow between them carries a rate. An unlabelled arrow is a decoration; a labelled one is a claim you can defend. Six minutes is enough for roughly ten boxes, so draw the ones the arithmetic forced and nothing else.

In the diagram below, follow the solid arrows left to right for the read path, and notice the one dotted arrow — that is the 2 MB image bypassing your services entirely. Every rate on an arrow was computed in step 1. A walkthrough in words follows the diagram, so read it once for shape and then read the prose.

flowchart LR
    C(["Client"]) -->|"4,630 reads/s peak"| CDN["CDN / edge"]
    CDN -->|"cache miss"| LB["Load balancer"]
    LB --> FS["Feed service<br/>stateless"]
    LB --> WS["Post service<br/>11.6 writes/s avg · 23.2 peak"]
    FS -->|"95% hit"| CA[("Feed cache<br/>Redis")]
    FS -->|"5% miss = 232/s"| DB[("Metadata DB<br/>primary + replicas")]
    WS --> DB
    WS --> Q[["Fan-out queue"]]
    Q --> FW["Fan-out worker"]
    FW --> CA
    C -.->|"presigned PUT<br/>2 MB"| OS[("Object store")]
    OS --> CDN

    style CA fill:#2d6a4f,color:#fff
    style Q fill:#bc6c25,color:#fff
    style DB fill:#1d3557,color:#fff

Walk the diagram out loud, because every box in it was forced by a number from step 1. Take it as three paths: the read path, the write path, and the image path.

The read path. Client traffic arrives at 4,630 reads/s peak and hits the CDN / edge first, whose whole job is to answer from a machine near the user so most requests never reach you at all. Anything it cannot answer is a cache miss, and those go on to the load balancer, which spreads them evenly across the fleet behind it. The load balancer sends reads to the stateless feed service, which answers 95% of them from the feed cache — an in-memory key-value store (Redis) holding each user’s precomputed list of post identifiers. The remaining 5%, which is 232 reads/s, fall through to the metadata DB.

That database is drawn as one box playing two roles rather than one machine. The primary is the single authoritative copy that takes every write. Its replicas are synchronized copies that absorb reads.

The write path. The load balancer sends writes to the post service, labelled 11.6 writes/s avg · 23.2 peak because that is the entire write load, and that number is the argument for keeping this whole path boring. The post service writes the metadata row into the database and drops a job onto the fan-out queue. Fan-out workers pull from that queue and push the new post into each follower’s cached feed.

The image path. The dotted arrow is the presigned PUT of 2 MB. Image bytes go from the client straight into the object store, never through your services, and the CDN serves them to readers from there.

Two more pieces of vocabulary from that walkthrough. The hit rate is the fraction of reads a cache answers itself, so a 95% hit rate means one read in twenty continues past it. The origin is whatever the cache falls back to — here, the metadata database.

Where the 95% hit rate comes from

The 95% is derived, not assumed, which matters in a chapter whose rule is that a hit rate you cannot justify is a claim you cannot make. There are two reasons a read misses a cache — the entry was evicted to make room, or it was never there — and the derivation rules out the first, then counts the second.

Start with the size. The cache holds one entry per daily active user, and each entry is 200 post identifiers of 8 bytes each:

feed cache    10,000,000 users x 200 ids x 8 B  =  16,000,000,000 B  =  16 GB

16 GB against the threshold card’s 128 GB machine, so nothing is ever evicted for want of room. Capacity eviction is not the source of misses.

That leaves cold misses only: the first open after an entry’s 24-hour expiry. At 20 opens per user per day, exactly one open in twenty is that first one, so the miss rate is 1 / 20:

miss rate      1 first open in 20  =    0.05
hit rate                                0.95
origin reads   4,630 x 0.05        =  231.5

232 origin reads per second is a number a single replica serves easily, and that is the whole argument for the cache, stated as arithmetic instead of as “caching helps.”

The buy-in checkpoint (2 min)

Stop drawing and ask whether this is the design the interviewer wants you to go deep on. This is a scored moment in its own right, and skipping it is why candidates spend eighteen minutes on something the interviewer did not care about.

“That is the skeleton. The two things I think are actually hard here are fan-out — whether the feed is materialized on write or assembled on read, and what a celebrity account does to that — and the cache invalidation on unfollow, because that one is a privacy bug rather than a staleness bug. I am going to start with fan-out. Stop me if you would rather see something else.”

Step 3 — Deep dive (18 min), and how to choose it yourself

A deep dive is nine uninterrupted minutes on one hard sub-problem, taken to the level of arithmetic and failure cases. This is the longest block and most of the score, and it takes three things: picking the two sub-problems worth diving into, the five-beat shape every dive follows, and — worked in full below — what a dive sounds like at speaking pace.

Do not wait to be asked “what would you like to go deeper on?” That question is a rescue, and being rescued costs you the point you would have earned by naming the hard part first. Announce the two topics at the buy-in checkpoint, with a one-clause reason each.

Picking the right two

Every system in this interview belongs to one of a handful of shapes, and each shape has a known hard part. Find your problem in the middle column, and the right-hand column is your deep dive. Shorthand in the rows is defined immediately after the table.

The shorthand in those rows, in the order it appears:

• Hot set — the small fraction of data that gets most of the traffic. When it is tiny, a cache is unreasonably effective.
• Cache stampede — what happens when a popular key expires and every request that wanted it goes to the origin at the same instant, which can take the origin down.
• Eviction — the policy for deciding what to throw out when a cache is full.
• Contended counter — one number many clients want to change at the same instant: a request count, a remaining-inventory figure, an account balance.
• Atomicity — making read-modify-write happen as one indivisible step, so two simultaneous increments do not lose one of themselves. This is the difficulty in a contended counter.
• Exactly-once — a message is processed precisely once. Expensive, and across a network not strictly purchasable.
• At-least-once with an idempotent consumer — you accept duplicate deliveries and make reprocessing harmless. This is the practical equivalent of exactly-once.
• CDC — change data capture: streaming a database’s changes out as an ordered event log.
• Resumability — continuing an interrupted upload from where it stopped rather than from the start.
• Geospatial — indexing positions on the earth, usually by chopping the world into cells and asking which cell you are in.

Two rules for the choice:

1. Pick the one where the right answer depends on a number you computed in step 1. Fan-out is a deep dive because 200:1 makes read-assembly expensive; at 2:1 it would not be worth eight minutes.
2. Do not pick the one with a library answer. “How do I store images” resolves to “object store plus CDN” in one sentence. Anything you can finish in a sentence is not a deep dive; it is step 2.

The shape of nine minutes

Every deep dive, on every system, follows the same five beats. Learn the shape rather than any particular answer, because the shape is what stops you from drifting into a component tour: walking through what each box does, which is description rather than design.

The five boxes below run in order, top to bottom. The two highlighted ones are where the score is: beat 2 is the arithmetic, and beat 4 is where you attack your own answer.

flowchart TD
    A["1 · State the tension<br/>two options, both defensible"] --> B["2 · Do the arithmetic<br/>the number that separates them"]
    B --> C["3 · Choose, out loud<br/>and name the cost you accepted"]
    C --> D["4 · Break your own choice<br/>the case where it fails"]
    D --> E["5 · Patch it<br/>hybrid, or a named escape hatch"]

    style B fill:#1d3557,color:#fff
    style D fill:#9d0208,color:#fff
    style E fill:#2d6a4f,color:#fff

In words:

1. State the tension. Two options, both defensible, so the choice you are about to make is a real one rather than a formality.
2. Do the arithmetic that separates them. This is the number the whole dive turns on.
3. Choose out loud, naming the cost you accepted rather than pretending the choice was free.
4. Break your own choice by finding the case where it fails. This is the beat that separates candidates.
5. Patch it, with a hybrid or a named escape hatch, rather than a claim that the failure does not matter.

Here is the whole sequence worked on fan-out, written at the pace you would actually speak it. Quoted text is what you say; everything else is commentary for you, the reader.

Beat 1 — the tension

“Either I materialize each user’s feed when a post is written, or I assemble it from the follow graph when the feed is read. Both are correct; they trade write amplification against read latency.”

The two options have names worth knowing, and the rest of the dive uses them:

• Fan-out on write does the work when a post is created. Copy the post’s identifier into the stored feed of every follower, so a read is a single lookup.
• Read-assembly (sometimes called fan-out on read) does the work when the feed is opened. Look up everyone this user follows, fetch each one’s recent posts, and merge them.
• Write amplification — the ratio of writes the storage layer performs to writes the user performed. One post becoming 200 cache writes is 200x amplification. This is the price of the first option; slow reads are the price of the second.
• Followee — someone you follow, the mirror of a follower.
• Index seek — one jump into a sorted index to find a row.

Beat 2 — the arithmetic

Cost each option at peak, in the units it actually consumes. Fan-out spends cache writes; read-assembly spends index seeks.

average followers per user                    200
posts/s avg                                  11.6
posts/s peak                                 23.2

fan-out writes/s — OFFERED LOAD
  avg                 11.6 x 200          =  2,320
  peak                23.2 x 200          =  4,640

read-assembly, per feed open — OFFERED LOAD
  followees                                   200
  index seeks to merge                        200
  at 4,630 feed opens/s peak
  4,630 x 200                            =  926,000

Both sides are demand, both at peak, so the comparison is apples to apples — which is the only way this comparison means anything. 4,640 against 926,000 is a factor of 200, and it is the read:write ratio again, exactly as the crossover derivation predicted.

Beat 3 — the choice, and the cost you accepted

“4,640 cache writes per second at peak is nothing. 926,000 index seeks per second is a fleet. So: fan-out on write. The cost I am accepting is that the feed cache is now derived state that can drift, and that a delete has to fan out too.”

Derived state is data computed from other data rather than written directly by a user. It can drift — silently stop matching the source it was derived from, because some update failed or arrived out of order.

Naming that cost is the whole point of beat 3. You are not claiming the choice was free; you are saying what you bought and what you paid.

Size the fleet before you go on

The next two beats divide by the fan-out fleet, and demand is not a denominator, so turn the offered load into a capacity figure now. Provisioning means deciding how much capacity to buy; the 25% below is deliberate headroom above the worst load you expect.

fan-out fleet — SERVICE CAPACITY
  provision at peak load + 25%    4,640 x 1.25   =  5,800  writes/s

spare at peak                     5,800 - 4,640  =  1,160  writes/s
spare at average load             5,800 - 2,320  =  3,480  writes/s

Three numbers there, and keeping them apart is the whole trick. 5,800 is what the fleet supplies. 4,640 is what the ordinary workload demands at peak. 1,160 is what is left over.

A celebrity post is paid for out of that 1,160 writes/s, not out of 2,320, which is work the fleet is already doing.

Beat 4 — break your own choice

“It breaks on a celebrity. One account with 30 million followers — registered accounts, not the 10M daily actives — posting once produces 30 million cache writes, and they come out of the spare:”

30,000,000 / 1,160 = 25,862      spare at peak
30,000,000 / 3,480 =  8,621      spare at the daily average

“That is 25,862 seconds — seven hours — if it lands at peak. Even at the daily average, where the spare is 3,480/s, it is 8,621 seconds, nearly two and a half hours, consumed by one post. And it is not a throughput problem I can solve by adding workers, because the last follower still gets the post hours after the first.”

(25,862 seconds is 25,862 / 3,600 = 7.2 hours; 8,621 seconds is 2.4 hours.)

That last clause is the important one, so make sure it lands. Two terms are doing the work:

• Throughput — how much work the system gets through per second.
• Tail latency — how long the unluckiest individual unit of work waits.

Adding workers raises throughput, which shortens the total. But the queue is still drained in order, so somebody is always last. The complaint is not that the fleet was slow; it is that one follower saw the post hours after another. Distinguishing those two is worth more than the arithmetic that produced them.

The division that flatters you

The naive version of that division is 30,000,000 / 2,320 = 12,931 s, about three and a half hours. It is flattering because 2,320/s is the fan-out fleet’s steady-state load, so spending it on a celebrity assumes the fleet is idle at steady state, the opposite of what the number says.

The fleet’s steady-state load and the fleet’s capacity are two numbers, and only one of them is a denominator.

Here the wrong version happens to be too flattering by exactly a factor of two, and that two is a coincidence rather than a law. It is worth knowing so you do not quote it as a rule:

spare at peak      = 0.25 x peak     (because headroom is 25%)
average load       = 0.50 x peak     (because the peak multiplier is 2x)
ratio              = 0.25 / 0.50     =  1/2

Provision at 40% headroom, or take a 3x peak instead of 2x, and the factor moves. The direction of the error is always the same; the size of it is not.

Beat 5 — patch it

“Hybrid: do not fan out above a follower threshold, and let readers merge those accounts in.”

Two things are worth being precise about here, because the usual justification for this patch is wrong.

It is the burst, not the average, that forces the special case

Per follower per day, fan-out costs posts_per_day writes and read-merge costs opens_per_day seeks. The follower count multiplies both sides and cancels — the same invariance the load-bearing test found when it moved followers 20 ↔ 2,000 and the 200x ratio did not budge.

So a celebrity posting 10 times a day, against followers who open 20 times a day, is still cheaper to fan out on the mean. The reason to special-case the celebrity is the burst: 30 million writes cannot be spread out, so the last follower waits hours.

What the patch buys back on the read side

The extra read cost is bounded by how many above-threshold accounts one user follows, not by 200. At 5 such follows per user:

merge seeks/s     4,630 x 5     =  23,150
full read-assembly                 926,000
23,150 / 926,000 = 0.025

That is 2.5% of what full read-assembly would have cost — which is why the hybrid is cheap.

Where the threshold number comes from

The threshold comes from a delivery SLO, not from taste. An SLO is a service level objective: a target you commit to and can be measured against. Here it is “every follower has the post within 60 seconds.”

The ceiling is spare capacity times the delivery target, never load times the delivery target:

1,160 x 60  =   69,600      against spare at peak
3,480 x 60  =  208,800      against spare at the daily average

69,600 followers is the ceiling — the peak one, because you do not get to choose when a celebrity posts, and a threshold that only holds at 4 a.m. is not a threshold.

The gap between those two lines matters more than either number. Against the average-spare ceiling of 208,800, a 100,000 threshold is comfortably safe. Against the peak-spare ceiling of 69,600, the same 100,000 is 44% too high (100,000 / 69,600 = 1.44). Same threshold, opposite verdict, and the only thing that changed was which spare went into the multiplication.

“I would set it at 50,000 for headroom, and log the distribution of follower counts to check that few real accounts sit near the line.”

The tempting version of that calculation multiplies the wrong number, and it is worth saying out loud why you did not. Spending the 2,320/s of average offered load gives 2,320 x 60 = 139,200, and a comfortable round-down to 100,000. But 2,320/s is work the fleet is already doing, so that ceiling silently assumes an idle fleet. It is twice as generous as anything the fleet can pay for, which makes a 100,000 threshold that looks safely conservative actually sit above the real ceiling.

The form of the expression is threshold = (capacity - load) x SLO_seconds, never load x SLO_seconds. The shape of the argument is the same either way — a hybrid, thresholded on followers, justified by a delivery SLO — but only one of the two ceilings is real.

Beat 4 is the one that separates candidates. Anyone can pick fan-out on write; volunteering the celebrity case before the interviewer raises it says you have run one.

The second deep dive

Your second dive gets the other nine minutes and the same five beats, but it should be a different kind of hard. If the first dive was a throughput tradeoff — how much work per second, and can the fleet carry it — make the second a correctness one: the cache invalidation on unfollow, where being wrong is a privacy bug rather than a slow page, or the idempotency of a retried write. Two throughput deep dives read as one deep dive done twice, and cover half the ground with the interviewer.

Step 4 — Wrap (5 min)

The last five minutes are three deliverables — the bottleneck, the failure modes, and one self-critique — at two minutes, two minutes and one minute.

Do not summarize the design. The interviewer watched you build it, and a recap spends your most valuable minutes repeating what they already have.

1 — The bottleneck, with the number that makes it one

“The first thing to fall over is the fan-out queue at a celebrity post: 30M writes against the 1,160 writes/s that a 5,800 writes/s fleet has spare at peak, which is seven hours. Second is feed-cache memory: 90M users x 200 cached post ids x 8 bytes is 144 GB, which is a Redis cluster rather than a Redis.”

The 90M there is the two-year DAU figure, and the rest of that second claim is one multiplication:

90,000,000 x 200 x 8 = 144,000,000,000

144 billion bytes is 144 GB, past the 128 GB commodity box on the threshold card — more memory than one machine holds. So the feed cache becomes a cluster, the data split across several Redis machines, rather than a single one.

Notice the form of both bottleneck claims: a component, then the number that makes it fail before any other component does. A bottleneck without a number is a guess, and every candidate guesses the same one.

2 — Failure modes, with the detection signal

Give three rows, not ten, and use the same frame every time: what breaks, how you would know it broke, and what guards against it.

The middle column is the one candidates omit and the one interviewers care most about, because anyone can list mitigations and only someone who has operated a system knows what the alarm actually looks like.

Four terms in that table, in plain words:

• Queue depth — how many jobs are waiting. Oldest-message age is how long the most patient of them has been waiting, and that second one is the right alarm, because a queue can be draining at full throughput and still be falling further behind.
• QPS — queries per second, the request rate arriving at a component.
• Request coalescing — when a hundred requests for the same missing key arrive at once, one of them goes to the origin and the other ninety-nine wait for its answer.
• TTL — time to live, the expiry clock on a cached entry. Jittered means each entry gets a slightly randomized expiry, so a batch written together does not all expire in the same instant and cause the stampede you were trying to avoid.

3 — The self-critique

One sentence, unprompted, naming the thing you would revisit and the trigger that would make you revisit it:

“The part I am least sure about is the 50,000-follower threshold. I derived it from a 60-second delivery SLO against the 1,160 writes/s of spare capacity in a 5,800 writes/s fan-out fleet, which assumes one celebrity posts at a time and that the fleet is at peak; if two post together the ceiling halves, and the whole thing is only as good as my 25% provisioning headroom, so the honest version is a dynamic threshold driven by current queue depth.”

Volunteering a weakness with a trigger reads as calibration, not doubt. Volunteering nothing reads as not having thought past the diagram.

Phrases that signal seniority

The rest of this chapter is reference material you can drill: the sentences that score, the sentences that cost you, the four standard ways candidates lose, a recovery plan for when you are behind, the rubric you are being marked against, and a one-page cheat sheet.

These are specific to distributed systems. Read the right-hand column as the inference the interviewer draws — the phrase is only worth anything because of what it implies you have done. (The equivalents that work in any design round are in ch 10.)

Two of those rows lean on ideas worth stating plainly:

• Hot key — a single item so much more popular than the rest that the one machine holding it saturates while the others idle. A celebrity’s row in a table sharded by user is the standard example, and it is the standard failure of an otherwise sensible shard key.
• Two generals problem — the classic proof that two parties communicating over a lossy network can never both be certain the other received a message. That is why exactly-once delivery cannot be bought outright, and why the practical answer is at-least-once delivery with a consumer that tolerates duplicates.

Phrases that hurt

The same list from the other side. The middle column says what is wrong with the phrase, and the right column is a replacement that makes the same point with something falsifiable attached.

Four of those replacements assume vocabulary:

• NoSQL — a loose family of non-relational stores. A KV store is the simplest member: a key-value store, which retrieves a value by its exact key and offers no joins and usually no multi-row transactions. It is fast precisely because it does less.
• Kafka — a durable, ordered log used as a queue. Calling it a durability boundary means the write is safely persisted there, so the write path survives even if everything downstream is down.
• Consistent hashing — a scheme for assigning keys to machines so that adding a machine moves only a small share of keys. The arithmetic in that row is the whole argument: with plain key mod N across 16 machines, going to 17 remaps 16/17 of all keys, about 94%, whereas a consistent-hashing ring moves about 1/(N+1), roughly 5.9%.
• Brownout — a partial degradation, where the system is slow but alive. Naive retries turn one into a full outage by adding load exactly when the system has least to spare. The fix is retries with full jitter — randomized backoff, so retrying clients spread out instead of synchronizing — and a retry budget, capping retries as a fraction of total traffic.

The four ways candidates lose this round

Four failure modes account for most rejections in this round, and none of them is about not knowing a component. Each one below gets the mechanism of how it costs you, plus the tell — the observable thing an interviewer sees when it is happening, which is also how you catch yourself doing it.

1. Jumping to architecture

You put boxes on the whiteboard at minute three, before a single number has been said. The cost is not aesthetic: at minute twenty-five the interviewer mentions the write rate is comparable to the read rate, and the entire fan-out design you spent fifteen minutes on was the answer to a different question. The five step-1 questions exist so that the diagram is a consequence rather than a guess.

The tell: you drew a load balancer before you said a number.

2. No numbers

Every claim becomes unfalsifiable, and unfalsifiable claims cannot be scored. “Add a cache” and “add a cache, and at 95% hit rate the origin sees 232 QPS which is one replica” are the same idea; only the second one is evidence.

The tell: the words “a lot of traffic.”

Rough numbers stated confidently beat exact numbers you do not have. If you are unsure whether a photo is 500 KB or 5 MB, say “call it 2 MB, and note that 5x on this number turns 1.46 PB into 7.3 PB, which changes the storage tier but not the topology.” That last move is a sensitivity bound: instead of defending your estimate, you say how much the conclusion would move if the estimate were wrong. Bounding the sensitivity is worth more than the estimate, and it is the load-bearing test from step 1 applied on the fly.

3. Silent whiteboarding

Ninety seconds of quiet while you work something out is ninety seconds the interviewer cannot score, and the default assumption when someone goes quiet is that they are stuck. Narrate the decision, not the arithmetic:

“Give me ten seconds — I am deciding whether the follow graph lives in the same database as the posts, and it turns on whether the visibility check has to be transactional.”

4. Gold-plating

You reach for multi-region deployment, a service mesh, Kafka, and a CQRS split, on a system doing 11.6 writes per second.

• Multi-region means running the whole system in several geographies at once.
• A service mesh is an infrastructure layer that routes and secures traffic between services.
• CQRS, command query responsibility segregation, means splitting the write model and the read model into separate systems.

Each is a real technique with a real problem it solves, and none of those problems exists at 11.6 writes per second. Every component you add is a component you must defend, and an interviewer who asks “why is Kafka there” and gets no number has found a hole you dug yourself.

The strong move is rejecting complexity on a number, out loud, at least once per round:

“I am not sharding the metadata database. It is 365 GB of post metadata plus about 100 GB of follow edges over two years, at 11.6 writes per second — under half a terabyte on one machine, with an order of magnitude of headroom, and a shard key I do not need is a migration I will regret.”

The tell: you cannot name what each box would look like if it failed.

When you are behind

You will sometimes reach minute 22 with the diagram half-drawn. The recovery plan is a checkpoint at minute 22, a fixed order in which to cut material, and three things you may never cut. Decide all of this now, because the one thing you cannot do at minute 22 is deliberate about triage.

The diamond at the top of the diagram below is the checkpoint question. The three branches out of it are the three places you might be, and they converge on one cut order and one untouchable minimum.

flowchart TD
    T{"Minute 22:<br/>where are you?"} -->|"still in requirements"| P1["Emergency:<br/>state all five assumptions,<br/>skip the API, draw now"]
    T -->|"drawing boxes"| P2["Compress: name the data model<br/>in one line, go to deep dive"]
    T -->|"starting deep dive"| P3["On track"]

    P1 --> CUT["Cut order:<br/>1 API sketch<br/>2 data model detail<br/>3 second deep dive<br/>4 wrap summary"]
    P2 --> CUT
    CUT --> KEEP["Never cut:<br/>the read:write number<br/>one full deep dive with arithmetic<br/>one named bottleneck"]

    style KEEP fill:#2d6a4f,color:#fff
    style P1 fill:#9d0208,color:#fff

Ask yourself the question once, at minute 22, and act on the answer:

• Still in requirements. Emergency. Declare all five assumptions from the card rather than asking anything further, skip the API sketch, and draw immediately.
• Drawing boxes. Merely late. Compress the data model to one spoken line and go straight to the deep dive.
• Starting the deep dive. On track. No triage needed.

The last two boxes in the diagram are the cut order and the untouchable minimum, and they are the same regardless of which branch you took. The cut order in full:

Never cut: the read:write ratio, one complete deep dive with arithmetic in it, and one named bottleneck. Those three are the round.

At minute 30, offering the choice costs nothing and scores well: “I have the unfollow-invalidation dive and the failure modes left — which is more useful to you?”

Scoring rubric

This is what the interviewer is filling in while you talk; read it as a self-assessment checklist after a practice round.

The dimension that decides the loop is “Deep dive.” It is 18 of the 45 minutes and it is the only block where the strong answer requires having built something.

Cheat sheet

The whole chapter compressed to one page, ordered by when in the round each move happens. Everything in it is derived above, and the right-hand column is the number that makes the move defensible.

Next: 04 — Design A Rate Limiter — the framework applied to a contended-counter system, where four algorithms differ only in what they do at a boundary and the arithmetic picks the winner.