10 — Design Interview Playbook

The agent design round is 45 minutes, one vague sentence from the interviewer, and a whiteboard. It needs a seven-step method for turning the sentence into an architecture you can defend, and a sense of which step to be on at any given minute.

The method covers six decisions:

• decide whether the problem needs an agent at all
• draw the control loop
• name the tools
• lay out what goes in the model’s window
• state the ways the design breaks, and how you would detect each one
• estimate the cost of a single task

The round does not measure whether you can build the system, because nobody builds anything in 45 minutes. It measures two things: whether you have a repeatable procedure for turning an ambiguous request into a defensible architecture, and whether you know what breaks.

What goes in and what comes out

The input is a single ambiguous sentence, delivered in the first minute and never elaborated unless you ask:

“Design an agent that handles customer support tickets.”

There is no requirements document, no sample data, and no acceptance criteria. How many tickets arrive per day, what the agent is allowed to change, and what “handled” even means are all missing on purpose — noticing what is missing is the first thing being scored.

The output is not code and not a running system. It is five artifacts on the whiteboard, and the interviewer’s scoring sheet has a row for each of them:

1. A tier decision — one model call, a fixed pipeline, or an agent — with the reason you chose it and the evidence that would change your mind.
2. A loop diagram — what calls the model, what the model is allowed to do, and how a run stops.
3. A tool table — for every action the agent can take: its name, its arguments, when it should be called, and whether it can be undone.
4. A failure list — what breaks, the signal that tells you it broke, and the guard that contains it.
5. One cost number — roughly what a single task costs, and the lever you would pull first to cut it.

Produce those five artifacts and defend each one and you pass the round, even when a specific detail turns out to be wrong. A polished architecture with none of the five does not.

The words this chapter uses

The method uses ten terms, defined here before it starts.

Two consequences of that table drive most of what follows.

First: the whole window is resent on every call. So a conversation’s token bill grows far faster than its number of steps. That single fact is behind every cost argument in this chapter, and step 7 turns it into dollars.

Second: the model only ever sees text. Anything you write in the prompt is advice. Anything you enforce in the harness is a rule. Advice can be argued with; a harness check cannot.

Which round is this, and what this chapter owns

Four different interviews are all called “design.” They are scored on different artifacts, and the first thing worth knowing is which one you are sitting in.

Two of those column entries need unpacking. A topology is which services and stores exist and how a request flows between them. The read:write ratio is how many reads the system serves for each write. A label is the known-correct answer attached to each training example.

The generative-AI round shares this chapter’s framing, cost and failure-mode stages, but it is scored on the build ladder and the eval strategy rather than on a loop.

Nothing below covers topology, data models, or label design. Cite those chapters, don’t re-derive them on the clock.

Three sections here are round-agnostic, and the other three framework chapters delegate to them rather than repeating them: Handling “I don’t know”, Time management, and Phrases that hurt.

The seven steps

The method is seven steps on a 40-minute clock with five minutes held in reserve. Each box carries a step number, the minutes it gets, and what those minutes produce; the two green boxes are the two that carry the most credit.

flowchart TD
    S1["1. Clarify · 5 min<br/>scope, scale, constraints"] --> S2
    S2["2. Decide the tier · 2 min<br/>call / workflow / agent"] --> S3
    S3["3. Draw the loop · 5 min<br/>context · tools · stop · guards"] --> S4
    S4["4. Tools · 8 min<br/>names, schemas, boundaries"] --> S5
    S5["5. Context & memory · 6 min<br/>what's in the window, what persists"] --> S6
    S6["6. Failure modes · 8 min<br/>and the guard for each"] --> S7
    S7["7. Evals & cost · 6 min<br/>how you'd know it works"] --> Q["Buffer · 5 min"]

    style S3 fill:#2d6a4f,color:#fff
    style S6 fill:#2d6a4f,color:#fff

What each step produces:

1. Clarify — pin down the scope of the problem, the scale it runs at, and the constraints on it.
2. Decide the tier — choose between a single call, a workflow, and an agent.
3. Draw the loop — put the control loop on the board and label its four parts.
4. Tools — fix the tool names, their schemas, and their boundaries. A schema is the machine-readable description of a tool’s arguments.
5. Context & memory — decide what sits in the window and what survives between sessions.
6. Failure modes — list what breaks and the guard for each.
7. Evals & cost — show how you would know it works, and what it costs.

The final box, Buffer · 5 min, is not a step at all. It is slack for the interviewer’s questions and for whichever step runs long, which is almost always step 6.

The two green boxes are step 3, draw the loop and step 6, failure modes. They are the two the time-management section refuses to let you cut.

Why this order and no other

The order is not a preference. Each step consumes the previous step’s output, so doing them out of order forces rework in front of the interviewer.

The Consumes column is the point: no step’s input exists before the step above it has run.

The durations add up to the round:

5 + 2 + 5 + 8 + 6 + 8 + 6 = 40,  + 5 buffer = 45

The “action space” in row 4 is simply the set of things the agent is able to do — every tool you gave it, and nothing else.

Memorize the boundaries, not the durations — 5 / 7 / 12 / 20 / 26 / 34 / 40 — because in the room you can see a clock but you cannot see an elapsed timer.

The checkpoint in time management below asks “minute 25: where are you?” That question only means something once you know that minute 25 is late in step 5: context is half-drawn and you have not started failure modes.

Where the budget goes, and why

Steps 3 and 6 are where the signal is.

Step 4 is where candidates burn 20 minutes writing JSON schemas nobody asked for — JSON, JavaScript Object Notation, being the bracket-and-quote format that programs use to exchange data. Keep step 4 to names, arguments, and one boundary sentence each.

The low-signal step still gets eight minutes, which looks like the budget contradicting the emphasis. It doesn’t. Step 4 is where the action space gets fixed, and steps 5 and 6 both consume it:

• Context layout depends on how large the tool schemas are.
• Every failure mode you can name is a failure of something the agent can do.

Cut step 4 to two minutes and steps 5-6 have nothing concrete to bite on.

Step 3 needs only five minutes because you are drawing a diagram you have drawn a hundred times. The thinking happened in step 2. Step 3 is transcription plus four labels.

Narrate the step you’re on. “Let me pin down scope first, then I’ll draw the loop” tells the interviewer you have a method, and it buys you permission to not answer everything at once. Interviewers score what they hear; a silent design scores nothing.

Step 1 — Clarify (5 min)

Five minutes is enough for a handful of questions, and asking many more than that loses points. Ask fewer, better questions.

These six cover most designs. The right-hand column is the test: if a question’s answer would not change the design, it is not worth the clock.

Six terms in that table are worth unpacking before you use them out loud:

• Reactive — the system waits for a user message.
• Proactive — the system wakes itself on a schedule or a signal.
• Idempotency — the property that doing the same action twice leaves the world exactly as doing it once did. A proactive system needs it because it retries, and a retry must not double-book a meeting or double-issue a refund.
• Batch — an interface where you submit a pile of jobs and collect the answers hours later instead of waiting on each one. Providers charge about half price for it, which is the 2× discount in row 4.
• Human-in-the-loop policy — the rule for which actions pause and wait for a person to approve them. Related: a latency budget is the ceiling on how long one task may take from trigger to answer.
• RAG (retrieval-augmented generation) — instead of hoping the model memorized your documents, you search them at request time and paste the best passages into the window before asking the question.

Question 2 pays twice, and it is worth saying so out loud:

“I’m asking what ‘done’ means because that’s both my stop condition and my grader — if a machine can’t check it, I have neither.”

What to actually say

Open with two questions and an assumption, not with an interrogation.

“Before I design anything — is this triggered by a user message or on a schedule? And when it finishes, is there something a program can check to confirm it succeeded, or is a human reading the output? I’ll assume user-triggered and human-reviewed unless you say otherwise, and I’ll flag where that assumption changes the design.”

Then state your assumptions out loud and move.

“Assuming: user-triggered, ~5k tasks/day, sub-30-second budget, it can write to the ticketing system but not to billing, and a wrong answer costs a support re-contact rather than money. Tell me if any of those are wrong — otherwise I’ll build to them.”

Scoring and failure

What the interviewer records during these five minutes comes down to one thing: whether an answer would have changed your design.

Never ask “what model should I use?” Pick one and justify it. Asking hands the only interesting decision back to the interviewer.

Step 2 — Decide the tier (2 min)

Step 2 decides which of three things you are about to draw — a single model call, a fixed workflow, or an agent. This two-minute decision is the one the rest of the design inherits.

The decision is a tree of three questions, asked in a fixed order.

flowchart TD
    Q1{Can you draw the<br/>flowchart today?} -->|Yes| W["Workflow<br/>+ agent escape hatch"]
    Q1 -->|No| Q2{Does the next step depend<br/>on what the env returns?}
    Q2 -->|Yes| A[Agent]
    Q2 -->|No| Q3{One shot enough?}
    Q3 -->|Yes| L[Single LLM call]
    Q3 -->|No| W

    style W fill:#2d6a4f,color:#fff
    style L fill:#2d6a4f,color:#fff

Question 1: “Can you draw the flowchart today?” If you can sketch the steps and their order on paper right now, the sequence is fixed — and fixed sequences are workflows. The workflow still gets an agent escape hatch: one branch that hands the odd input to a model-driven loop instead of failing.

Question 2: “Does the next step depend on what the env returns?” “Env” is the world outside the model: the tools it calls, the files it reads, and the APIs it talks to. An API is the network interface another system exposes for programs to call. When the answer genuinely comes back and reshapes the plan, you need an agent.

Question 3: “One shot enough?” If a single LLM call with a good prompt finishes the job, take it. Otherwise fall back to a workflow.

The green boxes are the two non-agent endings — Workflow and Single LLM call. They are green because this tree lands on one of them far more often than the interview’s framing suggests.

The most common trap in this round is a disguised workflow. They describe a fixed four-step pipeline and say “design an agent.”

What to actually say

“Most of this is a fixed pipeline — extract, validate, route, notify. I’d build it as a workflow with one LLM call per stage, because that’s cheaper, testable, and debuggable: I can unit-test each stage, and a failure tells me which stage. I’d add an agent escape hatch for the ~5% of inputs that don’t fit the pipeline, and I’d log how often that hatch fires — if it’s 30%, my pipeline is wrong and I should revisit.”

That answer scores higher than an agent design, because it shows you optimize for the product rather than for the interesting technology. The last sentence matters: you named the metric that would falsify your own decision.

The cost half of the argument

The tier decision is also a cost decision, so have the number ready.

Start with the two rules of thumb:

• A workflow with N stages costs about N× a single call, because each stage sends a fresh, flat context.
• A ReAct agent costs 5-20× a single call. ReAct is the standard loop in which the model alternates reasoning with an action — the name is short for “reason + act.”

The agent is worse than N× because its context growth is quadratic. Every call resends the whole conversation, so the tenth call carries nine turns of history, and the total grows with the square of the number of calls rather than in step with it (ch 02).

Now the arithmetic, on the shape step 7 costs out below: a 9,000-token stable prefix, plus roughly 1,700 tokens added per turn.

one standalone call   9,000 prefix + 1,700 of task text     =  10,700 input tokens
8-stage workflow      8 x 10,700   (each stage starts fresh) =  85,600
8-call ReAct agent    8 x 9,000 + 1,700 x (0+1+...+7)
                    = 72,000    + 47,600                     = 119,600

The ReAct line charges the first call no history at all, which is why it is not simply 8 × 10,700. The 1,700 is what a turn adds, and the first turn has nothing before it to carry. By call 8 the history is seven increments deep, so the increments sum to 0+1+…+7 = 28.

Two ratios come out of those three lines:

vs. one plain call    119,600 / 10,700 = 11.2  ->  ~11x
vs. the workflow      119,600 / 85,600 =  1.4  ->  ~1.4x

So the 10× figure has a denominator, and you should know which one. The 5-20× band above is quoted against a single call. Set the agent against the eight-stage workflow instead and the gap collapses to 1.4×. Quote the 10× and be ready to say which pair it compares.

Either way the workflow is the one you can debug, and that is the argument that actually wins the point.

Step 3 — Draw the loop (5 min)

Step 3 puts the control loop on the board — the artifact the interviewer takes notes against. It is the same diagram every time, with the same four labels.

Everything hangs off one field, stop_reason, and there are two exits: green Done is the success path, red Halt is a guard tripping.

flowchart TD
    G([Trigger]) --> M[Model call]
    M --> D{stop_reason}
    D -->|tool_use| E[Execute tools]
    E --> A[Append results]
    A --> GD{Guards}
    GD -->|pass| M
    GD -->|trip| H([Halt + report])
    D -->|end_turn| V{Verify}
    V -->|pass| F([Done])
    V -->|fail| M

    style F fill:#2d6a4f,color:#fff
    style H fill:#9d0208,color:#fff

The entry. A trigger — a user message or a schedule — starts a model call. The response carries a field called stop_reason that says why the model stopped generating, and the whole loop branches on it.

The tool_use branch. This value means the model has asked for one or more tools to run. The harness executes them, appends the results to the window, and hands control to the guards. If the guards pass, the loop goes round again. If one trips, the run ends at Halt + report, which stops the loop and hands back what was accomplished plus the reason it stopped.

The end_turn branch. This value means the model simply finished writing. It is not a claim that the task succeeded, so the run goes to a verification step: pass and you are done, fail and you re-enter the loop with the failure in context.

Then write the four labels beside the diagram, because they are the four things the interviewer is listening for:

• Context — what’s in the window, in what order.
• Tools — the action space.
• Stop — verifiable if at all possible, not just end_turn.
• Guards — step cap, budget, loop detector.

What to actually say

“The loop is: call the model, branch on stop_reason. On tool_use I execute, append results, run guards, and iterate. On end_turn I don’t trust it — end_turn means the model stopped generating, not that the task succeeded, so I run a verification step and re-enter the loop if it fails. Guards are a step cap, a dollar ledger, and a loop detector on the hash of (tool, args).”

Three of those terms should be defined if you are asked:

• A step cap is a hard limit on iterations.
• A dollar ledger is a running total of spend for the run.
• A loop detector watches for repetition by hashing the pair (tool name, arguments). A hash is a short fingerprint of a value, so identical calls produce identical fingerprints and repetition is cheap to spot.

Name the pattern, then name why

Name the pattern from chapter 02 explicitly, with the reason.

The orchestrator-worker pattern means one lead model call splits the task into pieces, each piece runs in its own fresh context window, and each worker — also called a subagent — hands back a short summary instead of everything it read.

“This is orchestrator-worker, because the number of subtasks isn’t known until we read the repo — and because each worker burns a big window and hands back a small summary, so the orchestrator never holds the noise.”

Naming the pattern earns little on its own; naming why earns more, because the reason is the part you cannot memorize in advance.

Step 4 — Tools (8 min)

Step 4 fixes the action space, and the whole thing fits in a four-column table. A table, not JSON, because the columns are where the judgement lives — especially the last one.

Two notations in that table: the question mark on product? marks an optional argument, and “gated” means the harness refuses to run the call until a separate condition is satisfied.

Then say the four things that matter:

1. “The description is a prompt.” It states when not to call, not just what it does. It is the highest-leverage text in the system, because it is the only place the model learns your boundaries (ch 03).
2. Which tools are irreversible, and what gates them (ch 07).
3. Whether you’d give it bash. Handing the agent a shell — one tool that runs arbitrary commands on a machine — buys enormous breadth at the cost of control. Say which side you’re choosing and why.
4. What happens past ~20 tools. Every schema sits in the window on every call, so at that point you need either a tool-search step that retrieves the handful of relevant descriptions per turn, or a router — a cheap first call that picks the subset (Tool selection at scale).

What to actually say

“Four tools. Two are reads and I’d let the model call them freely. issue_refund is irreversible, so it doesn’t get gated by a prompt instruction — it gets gated in the harness: a hard cap on amount_cents, and above that threshold it returns a pending-approval object instead of executing. The model can’t route around a check it never sees.”

That last sentence — safety in the harness, not the prompt — is worth saying once, deliberately.

Do not write full JSON schemas unless asked. It eats the clock and demonstrates typing, not judgment. If they ask for a schema, write one, and mention that strict schemas guarantee shape, not sense: the model can still pass a well-typed wrong order_id.

Step 5 — Context & memory (6 min)

Step 5 is about the one block of text the model sees on every call: what goes in it, in what order, what gets thrown away as it grows, and what survives to the next session.

The diagram lays the window out left to right, in the order the text is actually assembled. It is sorted from the part that never changes to the part that changes every time.

flowchart LR
    A["Tool schemas<br/>renders FIRST · stable"] --> B["System prompt<br/>stable"] --> BP{{cache breakpoint}} --> C["Retrieved docs<br/>semi-stable"] --> D["History<br/>grows each turn"] --> E["Current turn<br/>volatile"]

    style BP fill:#2d6a4f,color:#fff
    style E fill:#9d0208,color:#fff

The green box is the cache breakpoint: a marker telling the provider “everything before this point is reusable.” It is green because it is the only element in this diagram you actually place — the rest of the order is the API’s.

The red box is Current turn, and here red carries no warning at all. That block is new on every single call, which is what “volatile” means. Volatile is neither bad nor avoidable; it is just the one block that can never be cached. (The loop diagram used the same red for Halt, where it did mean something bad.)

Say four things about this diagram.

1. Layout and the cache breakpoint — “stable first, volatile last, so caching works.” Be precise about the order, because it is the API’s and not yours to choose: render order is tools -> system -> messages (Prompt caching derived).

Caching is a prefix match: the provider reuses the work it already did on the longest run of tokens at the front of the window that is byte-identical to last time. So the only thing you control is where the breakpoint goes. Everything before it must be stable, and one edited token anywhere before it invalidates the whole prefix.

2. Growth strategy — offload big tool outputs to disk and keep pointers; compact at the threshold. Offloading means writing the 40,000-token file dump to a file and putting a one-line reference in the window instead. Compaction means summarizing the older half of the conversation and continuing from the summary.

3. What persists across sessions — semantic facts, episodic outcomes, procedural lessons. Those are the three kinds of long-term memory:

• semantic — what is true (“this customer is on the enterprise plan”)
• episodic — what happened (“ticket 41 was resolved by a refund”)
• procedural — what was learned (“check the shipping table before quoting a date”)

And say what deliberately doesn’t persist.

4. The one number — “I’d watch cache_read_input_tokens; if it’s zero we have a silent invalidator.” That is a field the API returns on every response, counting how many tokens were served from cache instead of recomputed. Zero means nothing was reused.

What to actually say, with the mechanism attached

“Layout is stable-first: tool schemas, then system prompt, then a cache breakpoint, then retrieved docs and history. That order isn’t cosmetic — caching is a prefix match because attention is causal, so a token’s K and V depend only on what precedes it. Anything volatile in front invalidates everything behind it. A datetime.now() at position 30 of the system prompt costs you the entire 40k history.”

Unpacking “because attention is causal”

If they push on that middle clause, start one level below the word causal, because the chapter has been leaning on attention without saying what it is.

Attention is the operation that lets one token in the window use the others. For every token, the model computes a match score against each token it is allowed to look at, turns those scores into weights, and mixes the contents of those tokens together in proportion to the weights. That is the whole mechanism by which a sentence two thousand tokens back changes what gets predicted here.

Causal is the restriction on which tokens it may look at: only the ones before it. Backwards, never forwards.

For every token the model computes two vectors: a key (K), which is what a later token matches against, and a value (V), which is the content that gets mixed in. Because of causality, those vectors depend only on the tokens before it.

So the work done on a prefix is reusable exactly as long as the prefix is unchanged, and useless the moment one earlier token differs. That derivation (Prompt caching derived) takes fifteen seconds and separates you from everyone who memorized “caching is a prefix match.”

The growth answer

“For growth: tool outputs get written to disk and the agent keeps a one-line pointer, so context stays flat instead of growing linearly — that turns an O(n²) bill into O(n). Compaction is the fallback when the narrative itself gets long, and I’d budget one cold cache turn after each compaction because it rewrites the prefix.”

O(n²) and O(n) are the standard shorthand for how a quantity scales with n turns:

• an O(n²) bill grows with the square of the turn count — double the turns and you roughly quadruple the tokens
• an O(n) bill merely doubles

Keeping each turn’s contribution flat is what moves you from the first to the second.

A cold turn is one where nothing in the window matches what was cached, so the whole prompt is processed again at full price. Compaction guarantees exactly one of those, because it rewrites the text at the front.

The metric line is a strong signal. Very few candidates name a specific field here, and it’s the field a real operator checks first.

The weak answer named in the table below is “use a vector database” — a store that indexes text by meaning so you can fetch passages similar to a query. It is a retrieval component, not a context strategy, which is why offering it alone does not answer the question this step asks.

Step 6 — Failure modes (8 min)

Step 6 is the segment that decides the round: what breaks, the signal that would tell you it broke, and the guard that contains it. Volunteer these. Do not wait to be asked. It is the highest-signal segment, because it is the one part that cannot be answered from memorized material.

The full list is an index of what to volunteer, not an answer you deliver.

Frame every row you deliver the same way: “Here’s what breaks, here’s how I’d know, here’s the guard.”

That is exactly what a table cannot do at three words per cell. “Egress allowlist” and “two-phase commit” are things to say, and a candidate who can only say them fails the follow-up — which is always some version of why does that work?

You will not deliver all nine in eight minutes and you should not try; the rule for choosing which ones is at the end of this section.

What to actually say

“The one I’d worry about most here is the loop, because the mechanism makes it self-reinforcing: once the context contains three near-identical call/result pairs, a fourth identical call becomes more likely, not less — it’s a next-token predictor and you just showed it a pattern. So I hash (tool, args), trip at three, and before halting I feed the trip message back into the conversation. Told explicitly that it’s repeating, the model usually changes strategy, so I recover instead of failing.”

That paragraph is the shape to copy: one mechanism, one detection signal, one guard, one recovery. The blocks below give the same treatment to the remaining rows.

The mechanism under each row

Runaway cost. A step cap is not a budget, because cost is not linear in steps. Every call resends the whole history, so call t carries t-1 increments of it. Across N calls you pay 0+1+…+(N-1) increments, not N.

Put numbers on that. A run that escapes to 30 steps instead of 8 sounds like 30 / 8 = 3.75×. It is not:

8 steps    0+1+...+7   =  28 increments
30 steps   0+1+...+29  = 435 increments
                          435 / 28 = 15.5x

Detect with a dollar ledger that sums all four usage fields — input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens, the four token counts every response reports. The first one excludes cached tokens, so a ledger that sums input and output alone sees roughly half the real spend on a well-cached agent, and a $1 cap fires near $2 (Budget enforcement).

Guard in two layers. At 80%, inject “budget nearly spent, wrap up” into the context, which shifts the distribution toward concluding. At 100%, halt in the harness and return partial results labelled as partial.

Irreversible mistake. The — in the Detect column is the lesson, not an omission. Detection is a signal that arrives after the action, and after is worthless when there is no undo.

So the whole budget goes to prevention, ordered by whether the guarantee decays with context length (Irreversible actions):

1. Don’t expose the capability. A read-only credential does not deny DROP TABLE — the SQL command that destroys a database table — it cannot express it. There is nothing to persuade.
2. Convert the class of the action. A soft delete, which marks a record hidden rather than erasing it, makes the action reversible.
3. Two-phase commit. Split the write in two: propose_refund returns a diff plus an approval token, and issue_refund will not execute without that token. An approval token is a short unguessable string that authorizes exactly this one action. The model cannot construct one it was never given, because the token is minted outside its context — the same reason the harness beats the prompt.

Prompt injection — why the prompt cannot fix it. This is the failure where text the agent reads — a support email, a web page, a code comment — contains instructions, and the model follows them.

The reason no prompt closes it is that there is exactly one token stream. The system role, the user role, and your <untrusted_content> tags are all just tokens in the same sequence, weighed against each other by the same softmax — the step inside attention that turns raw match scores into the weights every token gets — and carrying no record of who wrote them. The model’s deference to the system role is a learned tendency, not memory protection.

Why the injected text tends to win. Competing on equal terms would already be bad enough, but injected text also holds two structural advantages over your instructions:

• It is more recent. Retrieved content lands at the high-recall end of the window while your rules sit far behind it.
• It is more specific to the task in progress, because it was written about the very ticket or page the agent is looking at. Your system prompt had to be written in advance for every ticket.

The lethal trifecta, and the leg you can actually remove. The framing that makes this designable is that the failure needs three things at once:

1. private data access
2. untrusted content
3. an external egress channel — where egress means any path by which bytes leave your system: an outbound web request, an email, a written file

Any two legs is survivable. An agent that reads secrets and reads hostile web pages but has no wire out can be fully hijacked and still leak nothing.

You can rarely drop data access (it is why the agent exists) or untrusted content (reading the ticket is the task), so egress is the removable leg. Legitimate destinations are a small, enumerable, slow-changing set, so you keep an allowlist — an explicit roster of permitted destinations, with everything not on it refused — checked in a socket-layer branch with no model in the decision path.

One channel people forget: rendered markdown is egress. An image URL in the agent’s reply fires a web request from the user’s browser the moment the reply is displayed (Prompt injection).

Wrong tool. This is a text bug, not a model bug. The model selects by matching the task against the descriptions it was given, so when two siblings’ descriptions overlap, the winner is decided by wording rather than by intent. search_web gets called where search_docs was meant because nothing in search_web’s text says don’t.

Detect with tool-call precision on a labelled set — correct calls over total calls, measured on a set of cases where you have written down which tool was the right one (Metrics that matter). It doubles as a leading indicator of cost, since every wrong call costs a call plus the resent history.

Guard by writing the negative condition into each description. The boundary sentence (“never use this for X”) is what stops overlap; prompt-level nudging is the fourth fix, not the first (ch 03).

Hallucinated answer. A hallucination is a confident claim with nothing behind it, and nothing in the way text is generated prevents one. The model samples a likely continuation, and a fluent fabrication is a likelier continuation than an admission of ignorance — unless you have made abstention, the model saying “I don’t know”, likely.

Detect by resolving every citation ID against the actual retrieved set. That is what a grounding check is: proof that each claim traces back to a passage that really was retrieved. Checking that the answer contains citation-shaped strings checks formatting, not grounding — a model that fabricates a claim will fabricate a plausible ID beside it.

Guard with required citations, an explicit instruction to abstain, and — the part people forget — a give_up branch in the retrieval loop. Without one the agent rewrites the query eleven times, drifting further from the question with each rewrite, and finally answers from prior knowledge, which is the exact failure RAG existed to prevent (ch 05).

Silent truncation. max_tokens is the ceiling you set on how much text one call may produce. It is enforced outside the model, and the model cannot see it coming.

Hitting it is not an error. You get HTTP 200 — the status code that means “this request succeeded” — and a sentence that ends mid-word. So pulling the text straight out of the response with response.content[0].text ships a truncated analysis as a complete one. Every downstream consumer then treats it as finished, including an LLM judge in your eval suite — a second model call whose job is to score the output — which will happily score it.

Detect by checking stop_reason on every single response, not just when something looks wrong.

Guard three ways: stream the output, meaning you receive it token by token as it is produced (which also avoids network timeouts on long outputs); raise the cap; and pair the invisible hard cap with a visible token budget stated in the context. You need both caps because they fail in opposite directions — the hard cap always holds and always produces garbage at the boundary, while the in-context budget lands gracefully and sometimes doesn’t hold (Hard cap vs task budget two mechanisms both required).

Degraded after a deploy. Prefix caching is an exact match on tokens, so this failure is a step function, not a drift. It does not decline gradually; it falls off a cliff. A reordered tool definition or one new byte in the system prompt takes the hit rate from 0.92 to 0.00 on every request, instantly, with no error and no exception (Why cache hit rate is the highest signal derived metric).

The bill moves the same way. Prefix tokens that were billing at 0.10× the normal input rate — the discount for a cache hit — now bill at 1.25×, the surcharge for writing a fresh cache entry. That is a 1.25 / 0.10 = 12.5× swing on the largest block of tokens in the request, and latency moves with it, because you are paying to reprocess the whole prompt again.

Detect with cache_read_input_tokens on the first response after any deploy; zero is the alarm. Guard by hashing the rendered prefix at startup and alerting when the hash changes without an intentional prompt change, which catches it before it bills for a day.

Late-session drift. This is the run that ends somewhere sensible-looking and nowhere near what was asked. Two mechanisms compound to produce it:

• Positional: how reliably the model uses a piece of text depends on where it sits, and that reliability is U-shaped — strong at the very beginning and the very end of the window, weakest in the middle. So by turn 30 the original instructions have been pushed into the worst position there is (Why quality degrades in long contexts).
• Generative: each self-chosen next step is locally reasonable given the last few steps, so every arrow is defensible and only the chain is wrong.

The trap is that a drifting agent restates its goal in its own words, and that restatement is produced by the same drifting process. Score the plan against it and it always passes, because restating is how drift launders itself (Autonomous loop).

So the comparison has to be against the immutable original goal text, stored outside the context where the model cannot edit it, and re-injected verbatim at the end of the window — the other high-recall position — so the anchor is refreshed rather than buried.

You have eight minutes, so choose two

Nine mechanisms is your preparation, not your answer. In the room, pick the two or three that this specific system actually has, and say why you picked them. The choice is itself scored, because it shows you read the design rather than a list.

“Three of these apply here and the rest don’t, and I’d rather go deep on three than list nine. First, it has an irreversible write, so: a refund against the wrong order. There’s no detection for that one — the signal arrives after the money moves — so it’s two-phase commit plus a hard cap in the harness. Second, it reads customer-pasted email, which puts untrusted content next to order data next to an outbound HTTP tool. That’s all three legs of the trifecta, and egress is the leg I can actually remove, so: allowlist. Third, silent truncation on the summarization step, because it fails as a success and nothing downstream would notice. What I’d skip is runaway cost — this loop is capped at six turns and can’t spawn subagents, so the ledger is a formality here rather than a guard.”

Naming the failure modes that do not apply, and why they don’t, proves the list is derived from your design rather than recited. It also buys back two minutes, which is the only place in this step you will find them.

Detection is the part candidates skip, and it’s the part that separates someone who has operated a system from someone who has designed one on paper. If you only have time for one thing in this step, give the detection signal.

Step 7 — Evals & cost (6 min)

Step 7 answers the two questions every design gets asked at the end: how you would know it works, and what it costs. Give two to three minutes to each, then stop.

Evals

Four decisions make the difference between an eval suite a team trusts and one it ignores.

1. Start with roughly 20 hand-written cases, and grow the set from production failures rather than from imagination. Imagined failures cluster around what you already thought of.

2. Assert the outcome strictly and the trajectory loosely. The outcome is the final result; the trajectory is the sequence of steps taken to reach it. Assert that the forbidden tool was never called — not that the agent took exactly these five steps. Trajectory assertions break on every prompt change and teach the team to ignore the suite.

3. Prefer code assertions, which are exact and free. Reserve a calibrated LLM judge for genuinely subjective dimensions. Calibrate it against about 50 human labels and report the agreement rate, the fraction of cases where judge and human reached the same verdict, before trusting it.

4. Put a gate in CI — continuous integration, the automated checks that run on every code change — on aggregate success plus zero safety violations. Run each case three times and take the majority verdict rather than demanding an exact match, because the output is not reproducible even at fixed sampling settings (Sampling and why temperature0 isnt deterministic).

“I’d gate on two things: aggregate success above the current baseline, and zero safety violations — that second one is a hard gate, not a percentage, because one refund to the wrong account is not offset by 99 correct ones.”

Cost — do the arithmetic out loud

This is the part almost nobody does, and it takes ninety seconds. Say every step.

First, the prices. All of them are per million tokens:

Step A — state the shape. Before any arithmetic, put four numbers on the board: calls per task, the size of the stable prefix, what each turn adds, and the output length.

“Say 8 model calls per task. Stable prefix — system plus tool schemas — around 9,000 tokens, and each turn adds roughly 1,700 tokens of assistant text plus tool result. So the average call carries about 15k input, and outputs are around 800 tokens.”

Step B — the uncached number. This is the simple version: multiply the average call by the number of calls, and ignore caching entirely.

8 x (15,000 x $5/1M  +  800 x $25/1M)
= 8 x ($0.075 + $0.020)
= 8 x $0.095
= $0.76 per task

Step C — the caching adjustment, derived rather than waved at. Now split the same token volume into three buckets, because each is billed at a different rate: tokens written into the cache, tokens read back out of it, and output.

total input  = 8 x 9,000 + 1,700 x 28   = 72,000 + 47,600 = 119,600 tokens
   writes    = 9,000 + 7 x 1,700        =  20,900 tokens @ 1.25x
   reads     = 119,600 - 20,900         =  98,700 tokens @ 0.10x

writes  20,900 x $5/1M x 1.25 = $0.1306
reads   98,700 x $5/1M x 0.10 = $0.0494
output   6,400 x $25/1M       = $0.1600
                                -------
                                $0.34 per task

Four of those numbers deserve a sentence each, because they are where the block stops being obvious.

Where 28 comes from. It is the same triangle as in the runaway-cost paragraph above: the first call carries no history increments and the eighth carries seven, so 0+1+…+7 = 28.

Why the total input is 119,600 and not 8 × 15,000 = 120,000. The 15k average in step A is that same total, divided by 8 and rounded. The two blocks are describing the same tokens.

Where 20,900 of writes comes from. Each block of text is written to the cache exactly once, the first time it appears: the 9,000-token prefix once, then each of the seven 1,700-token increments once. 9,000 + 11,900 = 20,900.

Everything else is a read, and it checks out from the other direction: the prefix is re-read on calls 2 through 8, and the increments are re-read 28 − 7 = 21 times between them.

7 x 9,000 + 21 x 1,700 = 63,000 + 35,700 = 98,700   <- matches the subtraction above

The 6,400 output tokens is just 8 × 800.

Now read the result out loud, and read the decomposition, not only the total:

“Caching takes it from $0.76 to about $0.34 — and notice what that does to the shape of the bill: output is now 47% of it. Caching can’t touch output, so my next lever is shorter outputs and fewer turns, not more caching.”

That 47% is $0.16 / $0.34. The observation shows you are reading the decomposition rather than reciting a lever.

Step D — routing, at task boundaries. Routing means sending easy tasks to a small cheap model and hard ones to a large expensive model. Haiku and Opus are the small and large tiers of the same model family; Haiku prices at $1/1M input and $5/1M output, a fifth of Opus on both sides.

“At 10k tasks/day that’s $3.4k/day. About 60% of these are simple lookups that finish in 3 calls, so I’d put a Haiku router in front and run that whole class on Haiku — about $0.03 a task. Blended: 0.6 x $0.03 + 0.4 x $0.34 ≈ $0.15, so roughly $1.5k/day. That is a 5× improvement on the $7.6k/day the uncached $0.76 would have cost — not on the $3.4k I just quoted, which is only 2.2× away.”

Two ratios are being juggled there, so keep them straight:

$7.6k / $1.5k = 5.1x   <- against the uncached baseline
$3.4k / $1.5k = 2.3x   <- against the already-cached number

The $0.03 is not a guess. It is the same writes/reads/output decomposition as step C, run on a 3-call task at Haiku’s prices — worth having done once so you can say it rather than assert it:

3 calls, same shape, Haiku at $1/1M in and $5/1M out
total input  = 3 x 9,000 + 1,700 x 3    = 27,000 + 5,100 = 32,100 tokens
   writes    = 9,000 + 2 x 1,700        = 12,400 tokens @ 1.25x
   reads     = 32,100 - 12,400          = 19,700 tokens @ 0.10x

writes  12,400 x $1/1M x 1.25 = $0.0155
reads   19,700 x $1/1M x 0.10 = $0.0020
output   2,400 x $5/1M        = $0.0120
                                -------
                                $0.0295 per task

The x 3 on the second line is the same triangle again: 0+1+2 = 3 for three calls. That it also equals the call count is a coincidence at n = 3 and does not hold in general.

Step E — name the caveat. This is the one that gets remembered, and it turns on caches being model-scoped: a cache entry belongs to the model that created it, so switching models mid-conversation means re-reading the entire history at full price.

“I route at the task boundary, not per turn, because caches are model-scoped. A warm Opus cache read is $0.50 per million input tokens; a cold Haiku prefill is $1.00. On a long history, downgrading mid-conversation can actually cost more, before you even discuss quality.”

Those two numbers come straight off the price table: a warm Opus read is $5 × 0.10 = $0.50 per million, while Haiku’s ordinary input price is $1.00 per million with no cache to hit. Prefill is the one-time pass the model makes over the whole prompt before it writes its first output token — the work a cache hit lets you skip.

Rough numbers stated confidently beat exact numbers you don’t have. Doing the arithmetic at all puts you in a small minority; doing it and then reading the decomposition puts you in a much smaller one. Full derivations in chapter 09.

Handling “I don’t know”

The seven steps assume you can answer what you are asked. In essentially every round there comes a moment when you cannot. It is a scored moment, and the score depends entirely on what you do next. Bluffing is the only answer that fails outright — interviewers ask follow-ups, and a fabricated foundation collapses on the second one.

Two questions, asked in order, land you on one of three answers — and none of the three endings is red: the worst outcome here is neutral.

flowchart TD
    Q["Question you can't answer"] --> A{Do you know the<br/>adjacent mechanism?}
    A -->|Yes| R1["Reason from it out loud,<br/>label it as reasoning"]
    A -->|No| B{Can you name how<br/>you'd find out?}
    B -->|Yes| R2["Name the experiment<br/>and the metric"]
    B -->|No| R3["Say so plainly,<br/>then bound the impact"]
    R1 --> G([Scores well])
    R2 --> G
    R3 --> OK([Scores neutral])

    style G fill:#2d6a4f,color:#fff
    style OK fill:#40916c,color:#fff

The adjacent mechanism is the physics or economics next door to the fact you are missing. If you know it, reason from it out loud and label it as reasoning. If you don’t, fall through to naming the experiment. If you can’t do that either, say so plainly and bound the impact.

Here are those three answers, in descending order of preference.

1. Derive from the adjacent mechanism, labelled as derivation.

“I haven’t measured that specific number. But I can reason it: output tokens price at 5× input on every tier, and that ratio comes from decode being memory-bandwidth-bound, so I’d expect the output half to dominate once caching is on. I’d verify before quoting it.”

Decode is the phase where the model writes its answer one token at a time. It is memory-bandwidth-bound because each of those tokens requires reading the model’s entire weights out of memory — so the hardware is waiting on memory, not on arithmetic, and no amount of batching makes a single token cheaper.

2. Name the experiment.

“I don’t know. Here’s how I’d find out in an afternoon: 50 labelled cases, run the two configurations, compare tool-precision and cost per completed task. If the gap is under 3 points I’d take the cheaper one.”

3. Say it plainly and bound the blast radius.

“I don’t know that, and I’d rather not guess. It’s contained though — it only affects the retrieval tier, so if I’m wrong the fix is swapping one component, not redesigning the loop.”

The thing that scores is showing your reasoning is still sound without the fact. Then move — don’t apologize twice, and don’t let it change your posture for the rest of the round.

One more habit belongs here: if you realize mid-answer that you said something wrong, correct it immediately and out loud.

“Actually, let me take that back — I said caching would help latency, and it mostly won’t; it cuts prefill, and decode is the wall clock.”

Self-correction reads as rigor. Letting it stand and hoping reads as either sloppy or dishonest.

Time management

Unknowns cost you seconds; the clock costs you whole steps, because you will fall behind. The question is never whether you run out of time; it is which minutes you take it from.

The checkpoint comes at minute 25: find where you are, and that branch leads to the cut order and the never-cut list.

flowchart TD
    T{"Minute 25:<br/>where are you?"} -->|"Still on step 1-2"| P1["Emergency: state assumptions,<br/>draw the loop NOW"]
    T -->|"On step 3-4"| P2["Compress tools to 4 rows,<br/>skip schemas entirely"]
    T -->|"On step 5-6"| P3["On track. Protect step 6."]

    P1 --> CUT["Cut order:<br/>1. tool schemas<br/>2. memory detail<br/>3. eval mechanics<br/>4. cost precision"]
    P2 --> CUT
    CUT --> KEEP["NEVER cut:<br/>the loop diagram<br/>failure modes + detection<br/>one cost number"]

    style KEEP fill:#2d6a4f,color:#fff
    style P1 fill:#9d0208,color:#fff

The three branches, in words:

• Still on step 1-2 — emergency. State your assumptions in one breath and draw the loop now, because a round with no loop diagram has no artifact.
• On step 3-4 — merely late. Compress tools to four rows and skip schemas entirely.
• On step 5-6 — on track. The job is to protect step 6 from everything else.

The two late branches feed the same cut order, and the cut order feeds the list of three things that are never cut.

The cut order, and why it’s this order:

Never cut:

• The loop diagram. It is the artifact the interviewer writes their notes against.
• Failure modes with detection. Highest signal per minute in the entire round.
• At least one cost number. Even “roughly a dollar a task uncached, fifteen cents with caching and routing” scores, because most candidates say nothing.

Two mechanical habits buy time back.

Announce a timebox and hold it: “I’ll give tools three minutes.” The interviewer then stops worrying about pacing and starts listening to content.

Ask which half they want, at around minute 30: “I have failure modes and cost left — which is more useful to you?” That is not a concession. It reads as prioritization, and it guarantees the remaining minutes land on something they care about.

If you finish early, do not fill the silence with more architecture. Offer the trade-off you skipped:

“The thing I’d want to revisit is whether the orchestrator is worth it at this scale — at 5k tasks/day a single agent with good offloading might beat it on both cost and debuggability.”

Phrases that signal seniority

These are the sentences that move a score, each paired with the thing it proves about you. Every one of them is a claim you should be able to defend for thirty seconds, which is why the second column names the mechanism rather than the vibe.

Four of those rows rest on ideas that need spelling out.

Multi-agent means one lead agent farming work out to subagents. Its point is context isolation, not speed: each subagent reads a lot and returns a little, so a system can explore hundreds of thousands of tokens of material while the lead holds only a few thousand tokens of summaries. You pay 4-15× the tokens of a single agent for that compression.

Both ends of that band are derived rather than quoted, from the same worked table in ch 06. At the low end, 4 workers explore 40,000 tokens each, and the whole system’s 178,400 tokens divide by a single agent’s 40,000 to give 4.5×. At the high end, 6 workers explore 60,000 each — 360,000 tokens of material against roughly 6 × 800 = 4,800 tokens of summaries the lead actually holds — and 381,600 / 25,000 = 15.3×.

Shadow mode means running the new system on real traffic while discarding its output and comparing it against whatever handles production today, so you learn its failure rate without anyone being harmed by it.

The false cache hit row is about semantic caching, which reuses a previous answer when a new question merely looks similar. A miss costs you one ordinary model call. A false hit serves a confidently wrong answer to a different question. So the similarity threshold belongs high.

Resist putting a multiplier on that asymmetry, because the two sides are not denominated in the same unit. A miss is priced in fractions of a cent; a false hit is priced in whatever a confidently wrong answer costs downstream — a refund against the wrong order, or a reply that contradicts policy. Quote the asymmetry and refuse the number.

ReAct is just the reason-then-act loop from step 3. Saying that native tool calling is ReAct means the pattern the original paper achieved by prompting is now the API’s built-in behaviour.

Phrases that hurt

These are the reflexes that cost points, each with the sentence to say instead. The pattern across all nine is the same: the left column names a tool or a hope, the right column names a mechanism.

Three terms in that table are worth one line each:

• LangChain and LangGraph are open-source libraries for wiring model calls together. Naming one answers “what would you install,” not “what would you build.”
• A state graph is that answer: an explicit set of states and the transitions between them, which is a design you can defend on a whiteboard regardless of what runs it.
• Fine-tuning means further training the model’s weights on your own examples — the slowest and most expensive lever available, which is why it belongs last.

Scoring rubric

This is what the interviewer is actually filling in while you talk. The last column is what “strong” concretely sounds like, in one sentence per row.

The column that decides the loop is “Failure modes.” It is the only dimension where the strong answer requires having operated something, so it’s the one interviewers weight when they’re deciding between two candidates who both drew a reasonable diagram.

Practice list

Eight prompts, each chosen because it forces one specific decision the others don’t. Whiteboard each on a timer; all eight are worked end to end in 11-case-studies/.

Four rows bring their own vocabulary.

Row 5. Hybrid search means running an exact keyword search and a meaning-based search together and merging the results. The meaning-based half is called dense retrieval because it compares numeric vectors rather than words — which is exactly why it misses a literal error code like ERR_4021, a string that appears in no similar-meaning sentence. Rerank economics is the trade in paying a second, slower model to re-sort the top few dozen hits.

Row 6. False-positive cost asymmetry is the fact that waking someone at 3am for nothing and missing a real outage are both errors, with wildly different prices. So the alerting threshold cannot sit in the middle.

Row 7. SQL is the query language databases speak. The lesson is that a read-only credential beats trying to parse a generated query for dangerous statements — a query that is merely expensive still passes the parser, and can scan a warehouse for an hour before anyone notices.

Row 8. Structured output is the API mode that forces a reply to match a schema, so you get a parseable record instead of prose. Its confidence threshold is the score above which the agent files the invoice itself instead of routing it to a person.

For each prompt, force yourself to answer in order: tier → pattern → tools → context → failures → evals → cost.

Three habits make the reps count.

Talk out loud, on a timer, to a wall. The bottleneck in this round is verbal fluency under time pressure, and silent practice trains none of it.

Record one and listen back. You will hear filler, hedging, and the places you skipped the “why,” which is your actual gap list.

After each rep, write down the one number you quoted. If you cannot produce a cost figure from memory for all eight prompts, you do not have step 7 yet.

After five reps the structure becomes automatic and you can spend the clock on the interesting parts instead of on remembering what comes next.

Cheat sheet

One row per moment that comes up in the round: the mechanism you should be able to state, and the sentence that states it.

Next: 11 — Case Studies — eight worked designs in exactly this format.