A team wants to build a new internal API on serverless functions. It will serve steady traffic of about 200 requests per second during business hours. What do you advise?

What the interviewer is testing

Whether you can apply the serverless trade-off to a specific workload rather than treating it as a default good or a default bad.

The reasoning

The workload described is the one serverless is worst at: sustained, predictable throughput.

Cost. Per-request pricing wins when utilisation is low, because you pay nothing when idle. At 200 requests per second sustained through business hours, utilisation is not low, and per-request pricing crosses over per-hour pricing — commonly by a factor of two to five. Do the arithmetic rather than asserting it, but the direction is reliable.

Cold starts. Steady traffic keeps functions warm, so this is not the main issue here, though scale-out at the morning ramp will still show it.

Connection management. This is the concrete technical problem. An internal API almost certainly talks to a database. Functions scale to hundreds of concurrent instances, each wanting a connection, against a database with a connection limit in the low hundreds. You end up needing a connection proxy — real infrastructure with its own cost and failure mode — to solve a problem containers do not have, because a container holds a pool.

Execution model. Long-running requests, streaming responses, background work after the response, and in-process caching are all natural in a container and awkward or impossible in a function.

Where serverless would be right

Change any of these and the answer flips:

• Traffic is spiky, or near zero most of the day.
• The workload is genuinely event-driven — a queue consumer, an S3 trigger, a scheduled job.
• The team has no platform capability and operating containers is a real cost.
• The service is small, isolated, and rarely changed — glue rather than product.

What a strong answer adds

Naming the middle option: a managed container service with scale-to-zero (Cloud Run, Container Apps) gives most of the operational simplicity of serverless with the container execution model, and is frequently the right answer for exactly this workload.

Also worth saying: this is a two-way door for a new internal API. If the team is enthusiastic and the cost delta is small in absolute terms, the learning may be worth more than the saving. Say what you would measure to decide in three months.