Scale-Bound Principle

A handbook at a 25-engineer company opens with four principles copied from a famous talk: partition everything, asynchrony everywhere, automate everything, everything fails. The company runs one PostgreSQL primary at 8% CPU. A year later every new service carries an idempotency key, a reconciliation job and a review argument about ordering, and the primary is still at 8%.

Two of those principles are about operations and pay from the first server onward. Two are about capacity, and their rationale is a workload no single machine can hold. Copying all four looked like adopting good practice and was a decision to pay a recurring tax against a problem the company did not have.

Why it matters

Principles are compressed constraints: statement, rationale, implications, exceptions. The rationale is the part that makes one usable in a case its authors did not foresee, so a principle copied without it cannot be applied, only cited, which is how a handbook starts making decisions nobody is accountable for.

The bill arrives in design reviews, where a reviewer blocks a simpler design with a quotation, and in features, because asynchronous writes move a cost into every feature: each read-after-write in the product now needs its own answer.

Implementation patterns

• Write the rationale as a testable sentence. "We partition because no single host can hold the working set" is checkable: measure it, and record the number with its date.
• Attach a trigger to each aspirational principle. "This becomes binding when the primary passes 60% sustained write utilisation" turns a slogan into a decision that fires on evidence.
• Separate operations principles from capacity principles, because the first set applies at any size and the second only above a threshold.
• Review the set annually and delete the ones whose rationale is still false, because a handbook that only grows is one nobody trusts, and a principle nobody can quote from memory constrains no decision in the room.

Industry example

Randy Shoup, then a Distinguished Architect at eBay, presented these four principles in talks and an SE Radio interview around 2008 to 2010, later extended to a ten-item list. What makes the eBay material a good example rather than a cautionary tale is that the mechanism was published with the principles: functional segmentation plus sharding along the primary access path, no session state in application servers, event queues with at-least-once delivery and no ordering guarantee, idempotent consumers, each with its own service-level target.

The later list added embrace inconsistency, which imitators drop most often and which makes the other three affordable, since partitioning and asynchrony are only cheap where the business accepts eventual consistency and reconciliation. Copying "partition everything" while keeping an expectation of immediate global consistency buys the costs of both models and the benefits of neither.

Failure scenarios

• The review veto. A simple design is rejected by citation, the reviewer cannot say what the principle protects here, and the team ships something more complex and less understood.
• The idempotency tax. Every consumer gets a key table and a reconciliation job that nothing needed.
• Consistency confusion. Asynchronous writes are mandated while requirements still assume read-after-write, so undocumented synchronous exceptions accumulate.
• Principle inflation. Past the point of recall, reviewers quote whichever sentence supports the position they already hold.

Trade-offs

Adopting a principle before its rationale is true buys consistency and avoids a future migration: a system built asynchronous from the start never has to be converted. That is a real gain, and it is why the practice is tempting. It pays cognitive load and delivery speed now, for a scale that may never arrive.

The honest version of that argument names the migration it avoids and estimates it. If nobody can estimate the migration, the argument is being asserted rather than made.

When not to use it

This is a test to apply to borrowed principles, not a reason to reject them. Operations principles (design for dependency failure, automate the deploy and the restore, make the system observable) are not scale-bound and should be adopted on day one, because they cost roughly the same at any size. Do not use "that was written for a bigger company" as a general defence against rigour: use it only where the rationale names a scale, and show the measurement proving you do not have it yet.

Interview question

Q: A new architecture handbook at a 30-engineer company lists ten principles taken from large technology companies. How do you decide which ones to keep?

What a strong answer covers: separating operations principles that pay immediately from capacity principles that need a scale threshold; rewriting each kept principle as "we do X because Y" and checking Y against a measurement; attaching a trigger and an owner to the aspirational ones; deleting the rest rather than leaving them as review ammunition; and naming the consistency expectation that makes partitioning affordable.

Quick check

Quiz: Which of eBay's four published principles transfer unchanged to a company on a single database primary, and why? Answer: "everything fails" and "automate everything", because they are about the cost of operating a system rather than about exceeding one machine.

Flashcard: One-line test for a borrowed principle? Write "we do X because Y" and check Y here with a number; if Y names a scale you cannot measure, the principle is decoration reviewers will still cite.