Adapter Conformance Suite
also called Port Conformance Tests, Shared Adapter Test Suite
One test suite written against a port and run against every implementation including the in-memory fake, so the fake cannot promise behaviour the real adapter does not deliver.
A repository port has two implementations: Postgres in production and an in-memory fake used by 400 unit tests. The fake's save overwrites by key. Postgres has a unique index and raises a violation. Nothing in the port said which was correct.
For a year the suite is green and the double-submit path looks tested. In production, two clicks on a payment form create two rows, and the handler meant to catch a duplicate never sees an error, because in every test it ran against a store that silently overwrote.
Nobody wrote a wrong test. Two implementations of one interface disagreed about behaviour the interface never specified, and every test ran against the one that does not ship.
Why it matters
A port is a promise, and in most codebases that promise exists only in the head of whoever wrote the first adapter. The fake is the dangerous implementation precisely because it is the one almost all tests exercise, so the gap between fake and real is untested by construction, and it widens each time someone adds a convenience to the fake to make a test pass.
The failures share a signature: a defect class only ever reproducible in production, with a green pipeline. That attacks what the suite was bought for, the belief that green means safe to ship.
Implementation patterns
- Write the suite against the port, parameterised over every adapter, including the fake. Modern frameworks do this with parameterised fixtures.
- Specify the behaviours your application actually depends on, one test each: uniqueness and the error raised, ordering without an explicit sort, case sensitivity, whether a pagination token survives an insert, empty result versus absent, delete of a missing row, and timestamp resolution.
- Make the fake refuse what it cannot imitate. An in-memory store cannot reproduce a serialisation conflict, so it should raise "unsupported" rather than succeed and teach the test a falsehood.
- Run the fake-backed variant on every commit and the real-backed variants on every merge; if the real run exceeds roughly 5 minutes, move it to nightly rather than dropping it.
- Every production bug from a difference becomes a new conformance test, not a one-adapter fix.
Industry example
The pattern predates service contract testing by a decade: xUnit Test Patterns (2007) catalogues the abstract test case, whose subclasses supply the object under test, so one specification runs against many implementations.
The archetype is a team running Postgres in production and a lighter embedded store in CI for speed. The embedded store has looser typing and no fixed-precision decimal type, so money arithmetic rounds differently, and the difference surfaces as a one-cent discrepancy in a settlement file weeks later. The conformance test that catches it stores a decimal amount and reads it back.
Failure scenarios
- Fake drift. The fake accumulates helpful behaviour until it is a different database, and the suite certifies that different database.
- Happy-path-only conformance. A suite testing save and load but no error or conflict certifies almost nothing while looking thorough.
- The fake taught the bug. A failing test is fixed by changing the fake instead of the specification, turning a caught defect into a permanent blind spot.
Trade-offs
| Choose | Gains | Pays |
|---|---|---|
| One conformance suite over all adapters | A fake you can trust, an executable specification, new adapters verified on arrival | Slower merges, containerised dependencies in CI, and writing down behaviour you currently assume |
| Per-adapter tests | Fast and easy to start | The fake becomes fiction and the bugs land in production |
When not to use it
One adapter, no fake, and a real dependency fast enough to use in tests is the case where this suite is pure cost. A local Postgres answering in 40 ms needs no in-memory substitute, and without a second implementation there is nothing to compare. A port with no behaviour, a wrapper that forwards one call, has nothing to specify either.
What flips it is the second implementation, or the fake becoming load-bearing, which in practice means more than roughly 50 tests depend on it. At that point the fake is production code with no tests of its own.
Interview question
Q: Your repository port has a Postgres adapter and an in-memory fake used across the unit suite. A duplicate-insert bug reaches production that the suite could not have caught. What do you change, and how do you stop the fix becoming one more special case in the fake?
What a strong answer covers: identifying the fake as an untested implementation of an under-specified interface; moving the specification into one suite run against both; naming the behaviours worth specifying rather than gesturing at coverage; the rule that a production difference becomes a conformance test; and the CI cost, with a real-backed run per merge as the compromise.
Quick check
Quiz: Why is an in-memory fake used by most of the suite a production risk rather than a convenience? — It is an unverified implementation of the port, so tests prove things about the fake while production runs the real adapter, and the gap is tested by nothing.
Flashcard: What should a fake do with behaviour it cannot imitate, such as a serialisation conflict? — Raise an unsupported error rather than succeed, so no test relies on a guarantee the real adapter lacks.