You are building webhook delivery for a platform. What must exist beyond sending an HTTP request?
Show the full answer Hide the answer
What is being tested
Whether you treat outbound delivery to endpoints you do not control as a resilience problem.
What must exist
1. Signing, with a timestamp. The receiver must verify the event came from you and has not been altered, and the timestamp prevents replay. Without it, anyone who learns the URL can inject events.
2. Retries on a published schedule — exponential backoff over hours or days, then a dead-letter state the integrator can see. Published, so integrators can reason about it; ended, because infinite retries accumulate forever.
3. Per-endpoint isolation. Circuit breaking after consecutive failures, bounded concurrency per endpoint, and separate queues by endpoint health. Without it, workers block on timeouts to one dead endpoint and delivery degrades for every integrator.
4. Aggressive timeouts — two seconds, not thirty. The consumer's job is to acknowledge, not to process, and the documentation must say so.
5. Notification through a different channel when an endpoint is failing, and disabling of persistently dead endpoints with a re-enable step.
6. Visibility — a dashboard showing delivery state, failures with response codes, and manual replay.
7. Rate-limited replay. Delivering two days of backlog at full speed to a just-recovered endpoint knocks it over again.
The mechanism most often missing
A reconciliation API. Webhooks are an optimisation over polling, not a guarantee. Some events will be missed — an endpoint down beyond the retry window, a deployment that lost them, a bug that dropped them silently.
The platform must support "what changed since timestamp X", and the documentation must tell integrators to use it periodically. A design that treats webhook delivery as authoritative will diverge with no way to detect it.
What to tell consumers
Acknowledge immediately and process asynchronously; deduplicate on event ID because delivery is at-least-once; do not assume ordering; and verify signatures.