Data Key Caching

Envelope encryption done literally means a call to the key service per object: generate a data key, encrypt the object, store the wrapped key alongside. At 200 million objects a day plus reads, that is on the order of 600 million requests a day, roughly 7000 per second average and 20000 to 35000 at peak.

Two things break at once. The cost is several hundred thousand dollars a year for requests that carry no data, and the peak sits at or above default account quotas, so the pipeline throttles and retries into its own throttle.

Why it matters

Caching data keys moves the request count by two orders of magnitude, which is the difference between a design that ships and one that is sent back by the platform team. Reuse across roughly 1000 objects takes 600 million calls a day down to about 600000.

More importantly, it forces a question the security review usually skipped: what is the key boundary actually isolating? Per-object keys are only meaningful if something depends on one object's key being independent of another's. Often nothing does, and the per-object design was chosen because it sounded stronger.

Implementation patterns

• Three simultaneous limits per cached key: maximum messages, maximum bytes, and maximum age. Whichever is reached first forces a new key. Ages are usually minutes.
• Cache scoped to the isolation boundary. A per-tenant cache keeps tenant data under tenant keys while still reusing within a tenant, which preserves the property that usually matters.
• Local, in-process caches, because a shared cache of plaintext keys across hosts is a new and worse asset to protect.
• Wipe on eviction and avoid persisting the plaintext key anywhere, including logs, crash dumps and heap snapshots.
• Decrypt-side caching too, since reads typically outnumber writes; the same limits apply.
• Metrics on cache hit rate and on requests per second to the key service, because the reuse factor is the number that determines both cost and exposure and it drifts as traffic changes.

Industry example

The pattern is standard in cloud encryption toolkits, which ship caching implementations with exactly these three limits, and the reason it exists is the arithmetic above rather than any subtlety of cryptography. The related failure it prevents is visible in any system that has hit a managed service's request quota in production: throttling arrives at peak, retries amplify it, and the encryption layer becomes the availability constraint for a pipeline that was never thinking about encryption as a dependency.

Failure scenarios

• An unbounded cache, so one data key ends up protecting a day's data and a single memory disclosure exposes all of it.
• Cache shared across tenants, quietly removing the isolation the design claimed.
• Plaintext keys in a heap dump collected for a performance investigation and stored in a ticket.
• Throttling storms when the cache is cold after a deployment and every worker requests keys at once.
• Keys cached longer than the rotation policy claims, so the documented rotation interval is not the real one.
• No decrypt-side limits, so a long-running reader accumulates a large set of plaintext keys in memory.

Trade-offs

Reuse is a direct dial between cost and blast radius. One compromised cached key exposes every object encrypted under it, so the limits are the security property, not an implementation detail. Caching also means plaintext keys live in process memory for their lifetime, which changes the threat model: memory disclosure now yields keys, not only data, and that is the reason lifetimes are minutes rather than days.

When not to use it

When the requirement is genuinely per-record: cryptographic erasure of a single record by destroying its key, or per-tenant isolation that must survive another tenant's key being compromised. Then the request cost is the requirement, and the right move is to make the unit coarser - per tenant per day rather than per object - rather than to cache across the boundary.

And if the realistic threat is a lost disk or a stray snapshot, skip envelope encryption entirely: volume encryption with one managed key meets that threat at zero request cost, and the envelope design adds a bill without adding a control.

Interview question

Q: A review mandates envelope encryption per object for a pipeline doing 200 million writes a day. Size the key-service load, then tell me what you would propose instead and what security property you would be trading.

What a strong answer covers: the arithmetic to roughly 600 million calls a day including reads, and that peak exceeds typical quotas · the cost in the low hundreds of thousands of dollars a year · caching with message, byte and age limits as the fix, worth about 100x · the explicit trade, which is blast radius per key · scoping the cache to the tenant boundary so the isolation that matters survives · and pushing back on the requirement itself by asking what the key boundary is meant to isolate.

Quick check

Quiz: What is the dominant variable in an envelope-encryption cost estimate? The reuse factor - how many objects share a data key - not the object count or the per-request price.

Flashcard: Which three limits bound a cached data key? Messages encrypted, bytes encrypted and age, whichever is reached first.