intermediate 3 min answer

Fastly's 2019 Lucet announcement said it instantiates a WebAssembly module in under 50 microseconds with a few kilobytes of memory overhead, against roughly 5 milliseconds for V8, and Fastly's own later material cites 35.4 microseconds as the startup time. A team reads that, moves a product-detail API to the platform, and finds it slower than the regional service it replaced. What did the instantiation number buy them, and what did it quietly take away?

fastlywebassemblycold startconnection poolingedge
Show the full answer Hide the answer

What is gained, quantified

An instantiation cost measured in tens of microseconds makes a fresh instance per request affordable. That changes two things structurally rather than incrementally.

There is no warm pool, so there is no cold start. Conventional serverless has a bimodal latency distribution because a request either finds a warm instance or pays initialisation; the tail is the miss rate times the init cost. At 35 microseconds the init cost is below the noise floor of a network hop, so the distribution is unimodal. For an edge deployment this matters more than it does centrally, because traffic is spread over hundreds of locations and most of them are idle most of the time - a miss at the edge is a common case, not a tail case.

And nothing survives between requests, so nothing can leak between them. No shared heap, no reused globals, no state carried from one tenant's request into another's.

What is paid

The same sentence read the other way: nothing survives between requests. Every habit that made the regional service fast depended on something surviving.

  • No connection pool. The regional service held warm TLS connections to the catalogue database. Each request here establishes its own path to whatever it needs.
  • No in-process cache. A memoised config lookup, a compiled template, a parsed schema, a hot LRU - all of it is rebuilt or refetched.
  • No amortised warm-up. There is no JIT to reach steady state and no lazily-built index to keep.

The arithmetic is brutal and simple. You saved 35 microseconds and your handler now makes a 40 ms fetch it did not make before - three orders of magnitude the wrong way. The instantiation number is real and it is irrelevant to the latency the user feels, because the user's latency is dominated by data access, not by code start.

When the bill arrives

The day someone adds a second dependency to the handler. One fetch at 30 ms is tolerable. Two sequential fetches is 60 ms, and the edge response is now slower than the regional service that held both behind a warm pool. The regression arrives through ordinary feature work, not through a bad decision, which is why it is rarely attributed correctly.

A second bill arrives with runtime churn: Fastly moved Compute from Lucet to Wasmtime in 2022. Pinning an architecture to one runtime's published startup figure is a bet on an implementation detail the vendor is free to change.

How to keep the option to reverse

  • Write the handler as a pure function of request plus injected configuration, so the same code runs in a container unchanged. The moment it calls the platform's own storage API directly, it stops being portable.
  • Put the data the handler needs in the runtime's local store rather than fetching it from a region. The rule is: move the compute only after the data it reads is already there.
  • Measure fetch count per request as a first-class metric, with an alert when it rises above one. That single number predicts the regression months before the latency graph does.

When this is the wrong answer

If the handler makes even one call back to a central database, keep it in a region. You have moved the compute and left the data, which adds a hop without removing one. A regional service with a warm connection pool wins on every percentile.

The genuine fits for a per-request-instance runtime are handlers whose entire input is the request plus something already replicated locally: request rewriting, header and cookie logic, A/B assignment, signed-URL validation, bot scoring on request features, and rendering from a locally cached payload. Everything that needs to ask a question of a database belongs where the database is.