intermediate 2 min answer

eBay's architects published four principles in the late 2000s - partition everything, asynchrony everywhere, automate everything, everything fails. A 25-engineer company with one PostgreSQL primary copies the list verbatim into its architecture handbook. Which of the four survive the copy, and what do the others cost?

ebayprinciplesscalerationalehandbook
Show the full answer Hide the answer

The situation they were in

Randy Shoup, then a Distinguished Architect at eBay, presented these four principles in a run of talks and an SE Radio interview around 2008 to 2010, later extended to a ten-item list that added embrace inconsistency, expect ®evolution and dependencies matter. The slides are specific about mechanism: functional segmentation plus horizontal sharding of data along the primary access path, no session state in application servers, communication through event queues with at-least-once delivery and no ordering guarantee, consumers written to be idempotent, each consumer carrying its own service-level target.

Each principle is a compressed statement of a constraint eBay actually had: data and traffic that no single machine could hold, and a fan-out of consumers owned by many teams.

Which two transfer

"Everything fails" and "automate everything" are cost-of-operations principles, and they pay from the first server onward. Designing for a dependency being down, and automating the deploy and the restore, cost roughly the same at 25 engineers as at 2,500 and save work immediately. Copy them.

What the other two cost at 25 engineers

"Partition everything" and "asynchrony everywhere" are capacity principles. Their rationale is a workload that exceeds one host, and a single PostgreSQL primary is the evidence that this company's workload does not.

Adopting them anyway imports a standing tax: no cross-partition joins, so every query that spans tenants becomes an application-side merge; at-least-once delivery, so every consumer needs an idempotency key and a reconciliation job; and no ordering guarantee, so each read-after-write in the product needs its own answer. That last one is the expensive part, because it is paid in every feature by the product team, not once by the platform team. A team of 25 will spend months of cumulative effort avoiding a problem it does not have, and will still have a single primary.

The test for whether a principle transfers

Write the sentence "we do X because Y" and check whether Y is true here, with a number. We partition because one host cannot hold the working set is checkable: measure the working set. If Y names a scale you do not have, the principle is decoration, and decoration is not free because reviewers cite it. A principle with no rationale attached cannot be applied to a case its authors did not foresee, which is the only situation in which principles earn their keep.

When not to copy it

The list most likely to be dropped by a small team is embrace inconsistency, and it is the one that makes the other three affordable: partitioning and asynchrony are only cheap if the business accepts eventual consistency and reconciliation. A handbook that copies "partition everything" and quietly keeps the expectation of immediate global consistency has bought the cost of both models.