A multi-tenant platform's tests depend on shared test data that drifts and breaks. What test data strategy works?

Why shared test data breaks

Tests mutate it. One test's changes affect another's expectations, order matters, and parallel execution becomes impossible. Over time nobody knows which data any test depends on, so nothing can be cleaned up and the dataset grows into an unmaintainable artefact everyone is afraid to touch.

The strategy

1. Each test creates the data it needs and cleans up after itself, or runs in an isolated tenant or schema that is discarded. This is the single most important change — it makes tests independent, parallelisable and order-independent.

2. Data built through the application's own interfaces, not inserted directly. Direct insertion creates states the application cannot produce, so tests pass against data that could not exist — and miss defects in the creation path.

3. Builders with sensible defaults, so a test specifies only what matters to it. A test that must construct forty fields to exercise one is unreadable and breaks whenever the model changes.

4. Deterministic values. Random data produces tests that fail occasionally with an unreproducible input. Where variation is wanted, seed it and log the seed.

5. A small, curated reference dataset for genuinely shared reference data — currencies, countries, product categories — which is read-only and therefore safe to share.

The multi-tenant advantage

Tenancy is already an isolation boundary. Creating a tenant per test gives complete isolation using a mechanism the system already has — which is cheaper and more faithful than any test-specific isolation scheme, and it also exercises the tenant provisioning path.

The production-data question

Do not copy production data into test environments. It carries personal data into systems with weaker controls and broader access, it cannot be reconciled with deletion obligations, and it creates a compliance exposure that is difficult to bound.

Where realistic data is genuinely required — performance testing, data migration rehearsal — use generated data preserving the statistical shape (volume, distribution, skew, cardinality) or rigorously masked data with the masking verified rather than assumed.

The shape is what matters for those tests, and the shape can be generated.