Which non-functional properties can be tested pre-release, and which can only be verified in production?

Testable pre-release

• Performance under a modelled load, provided the model reproduces the real access distribution — uniform data is what makes a load test pass at five times peak while the real peak fails.
• Resource behaviour under sustained load — memory growth, connection leaks, fragmentation — via soak testing, since these are proportional to elapsed time and no short test finds them.
• Resilience to injected dependency failure, particularly slow-not-down, which is the mode that causes most outages and is invisible to error-rate controls.
• Security properties — dependency vulnerabilities, common weaknesses, policy compliance.
• Accessibility, for the substantial share that is mechanically checkable.
• Migration and cutover procedures, rehearsed and timed.

Verifiable only in production

• Behaviour under real traffic distribution, including the skew, correlated bursts and pathological cases no generator reproduces.
• Real network conditions — mobile variability, distant clients, packet loss, path MTU issues — which are invisible from the server side entirely.
• Third-party behaviour under real load, including their rate limits and their worst day coinciding with yours.
• Interaction with real data, including the records that should not exist and do.
• Capacity limits, since the true ceiling depends on the real workload mix.
• User-perceived quality, which is measured at the client and correlates imperfectly with any server metric.

What follows

A strategy that treats production verification as part of testing, not as its absence. Feature flags, progressive rollout with health gates comparing canary against control, shadow traffic, and continuous fault injection at low traffic share.

And observability designed in, since production verification is only possible if the properties can be measured — which requires instrumentation decided at design time rather than added after an incident.

The specific case for real-time systems

For a communication platform, almost everything that matters — media quality, join latency, behaviour under network degradation, regional performance — is only measurable at the client, under real conditions.

Server-side testing verifies that the server is correct, which is necessary and is not the product.