Network API Gateways

Definition

At the network level, an API gateway is where external traffic stops being external: TLS terminates, identity is established, limits are enforced, and the request is routed to an internal destination over an internal protocol.

The network responsibilities that belong here

• TLS termination and certificate lifecycle, including rotation and modern cipher policy in one place rather than on every service.
• Protocol translation. HTTP/3 or HTTP/2 outside, whatever is convenient inside — commonly gRPC. This lets you adopt new external protocols without touching services.
• Connection management. Absorbing many slow client connections and multiplexing onto a small number of efficient internal ones. This protects backends from slow-client resource exhaustion in the same way a reverse proxy does.
• Ingress-side limits — request size, header size, timeouts, connection counts — enforced before anything expensive happens.
• Correlation ID injection, so every downstream log and trace can be tied back to one external request.

The critical-path consequence

The gateway must be more available than anything behind it, because everything behind it is unreachable if the gateway is down. Three implications that are frequently under-planned:

• Configuration changes are as dangerous as code deployments. A wrong routing rule is a total outage in one second, so gateway configuration needs staging, review, canary and instant rollback.
• The gateway must not be a shared bottleneck for organisational change. If every team needs a platform team's help to add a route, the gateway becomes the release constraint for the company. Self-service configuration with automated validation is the answer.
• Capacity is measured in connections as well as requests. Long-lived and slow connections consume resources disproportionate to their request rate.

Failure scenarios

• A regex or path change routing all traffic to the wrong backend, deployed globally without a canary.
• Timeouts longer than the client's, so the gateway holds resources for requests nobody is waiting for.
• Authorisation performed only at the gateway, so anything reachable internally is unprotected — the assumption zero-trust designs exist to remove.
• A single shared gateway across tenants, so one tenant's spike degrades everyone.
• No request size limits, allowing a large-body request to exhaust memory before any application logic runs.

Interview question

"Your gateway is healthy but every request returns 503. Where do you look, in order?"