Confidential Inference: What a TEE Actually Proves

At WWDC in June 2026, Apple said that Private Cloud Compute, the infrastructure that handles Apple Intelligence requests too large for a phone, would begin running on hardware Apple does not own. The new nodes sit in Google Cloud, on NVIDIA Blackwell GPUs, behind Intel TDX trust domains, on machines whose boot chain is anchored by Google's Titan security chip. The privacy claim did not change: stateless computation, enforceable guarantees, no privileged runtime access, non-targetability, verifiable transparency (Apple, 2026, Expanding Private Cloud Compute). A company whose entire marketing position rests on not reading your data moved that data onto a competitor's servers and told its users the guarantee was unchanged.

That is either a remarkable piece of engineering or a remarkable piece of nerve, and the difference comes down to a question most teams evaluating "confidential AI" never ask precisely: what does the hardware actually attest to? A trusted execution environment proves the identity of the code that is running. It proves nothing about that code's behaviour. Every interesting property in a system like PCC comes from what gets built on top of that one narrow proof.

Why this matters: The performance objection to confidential inference has largely collapsed; Hopper was already under 9% on average and tuned Blackwell measures in low single digits. What remains is a semantics problem: teams buy "the provider cannot read my prompts" and receive "the provider cannot read your prompts unless the provider chose an image that reads them, and the hardware will cheerfully certify that image." Knowing which of those you bought decides whether the architecture is worth its operational cost.

TL;DR

• Confidential inference spans two enclaves. A CPU trust domain (AMD SEV-SNP, Intel TDX, Arm CCA) protects host memory; an H100 or B200 in confidential mode walls its HBM into a Compute Protected Region. The link between them is the expensive part, because on Hopper PCIe is untrusted and every crossing byte is staged through a shared bounce buffer and encrypted with AES-GCM-256.
• The overhead is therefore a transfer tax, and shrinks as models grow: average under 9%, most LLM queries under 5%, near zero for long sequences (arXiv:2409.03992).
• End-to-end serving is worse than microbenchmarks. Under Intel TDX at a fixed request rate, confidential mode raised average TTFT by 21.8% for Mistral-7B and 27.8% for Qwen3-30B-A3B, with token throughput down 17.7% and 21.1% (arXiv:2607.19353). Blackwell, tuned, measures at 1% to 3%; with a stock stack, 30% to 40% (arXiv:2608.26575).
• Attestation signs a measurement: a hash chain over what was loaded. It is an identity, so an image that logs every prompt attests perfectly. And two valid tokens, one from the CPU and one from the GPU, do not prove the GPU is attached to that CPU; the binding is application work.
• The controls that make attestation mean something are external: key release gated on appraisal policy, a transparency log of approved measurements, and non-targetability so no single user can be steered to a watched node.
• Memory encryption does not cover timing, interrupts, power, or interconnect traffic. Heckler and WeSee break CVM integrity by injecting interrupts; Energon recovers transformer architecture from GPU power and thermal traces.

At a Glance

The system is a chain of refusals. The device refuses to send until the node proves what it is running; the key manager refuses to release until that proof satisfies policy; the enclave refuses to hand plaintext to anything outside its boundary. Break one link and the rest is theatre.

flowchart LR
  C[Client device] -->|nonce challenge| N[Confidential VM]
  N -->|CPU quote| V[Verifier]
  G[GPU in CC mode] -->|device evidence| V
  V -->|signed token| K[Key manager]
  K -->|weight key, if policy holds| N
  N -->|encrypted DMA| G
  G -->|result in enclave| C
  L[(Transparency log)] -.checked by.-> C

  class C,L blue
  class N,G purple
  class V,K teal
  classDef blue fill:#1e40af,stroke:#3b82f6,stroke-width:1px,color:#fff
  classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
  classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff

[IMAGE: Side-by-side memory maps of the same inference server, non-confidential and confidential. Left: host RAM and GPU HBM both shaded "readable by hypervisor", with labelled regions for weights, KV cache and the current prompt. Right: the same regions shaded "encrypted, key in silicon", with a narrow shared bounce buffer highlighted in amber as the only crossing point. Caption: "Confidential mode does not add a wall around the workload; it moves the wall to the silicon and leaves one guarded door."]

Before an Enclave Could Hold a Model

Enclaves predate the problem they are now used to solve. Intel SGX arrived in 2015 built for small, carefully written code: a few megabytes of protected memory and an expectation that developers would split their application into trusted and untrusted halves by hand. A 14 GB weight file does not fit in an SGX enclave page cache, and nobody partitions a PyTorch process manually. What made confidential AI plausible was moving the boundary up from the process to the whole virtual machine. AMD's Secure Encrypted Virtualization, announced in 2016, encrypted guest memory with a per-VM key held in the platform security processor, so an unmodified OS and application could run inside; Intel's Trust Domain Extensions took the same shape. Both began as confidentiality without integrity, since the hypervisor could still remap, replay and corrupt pages it could not read. SEV-SNP and TDX closed that with page-ownership structures, and an empirical comparison finds the two converge on four mechanisms: memory encryption, a page ownership table, a protected guest-to-hypervisor channel, and intra-guest privilege separation (Misono et al., 2024, ACM SIGMETRICS).

None of that helps when the interesting computation happens on an accelerator outside the boundary. That piece landed in 2023, when NVIDIA shipped Hopper with a confidential mode: HBM partitioned into a Compute Protected Region, compute engines confined to it, and the DMA engines the sole path across, with hardware forcing AES-GCM-256 on everything crossing (Dhanuskodi et al., 2023, Creating the First Confidential GPUs, ACM Queue 21(4)).

timeline
    title From process enclaves to confidential AI factories
    2015 : Intel SGX ships enclaves sized for small trusted code
         : Too small and too manual for ML workloads
    2016 : AMD SEV encrypts whole-VM memory with a per-VM key
    2019 : Confidential Computing Consortium forms at the Linux Foundation
         : Alibaba, Arm, Google Cloud, Huawei, Intel, Microsoft, Red Hat
    2020 : AMD details SEV-SNP, adding integrity and page ownership
    2021 : CipherLeaks breaks constant-time crypto via the ciphertext side channel
    2023 : NVIDIA Hopper ships the first confidential GPU
         : Intel TDX trust domains reach cloud availability
    2024 : Azure makes confidential H100 VMs generally available
         : Apple publishes Private Cloud Compute
         : Heckler, WeSee and TDXdown break CVM integrity and timing defences
    2026 : Apple extends PCC to Google Cloud on Blackwell GPUs
         : Tuned Blackwell confidential inference measures in low single digits

The Confidential Computing Consortium, formed on 17 October 2019 with Alibaba, Arm, Google Cloud, Huawei, Intel, Microsoft and Red Hat as founding premiere members, marks when this stopped being three vendors' proprietary features and became a category (Linux Foundation, 2019).

How Confidential Inference Actually Works

The boundary has two halves and one seam

A confidential inference node is two enclaves joined by a link that is trusted or not, depending on which generation of hardware you bought.

On the CPU side the inference server runs inside a confidential VM, its memory encrypted with a key generated inside the security processor, never exported, different for every VM. The hypervisor keeps scheduling the guest and servicing its I/O, but reads only ciphertext, and page ownership stops it swapping page A for page B without the guest noticing.

On the GPU side, an H100 in confidential mode is not simply a GPU with encrypted memory. The firmware disables the debug and performance-counter paths that would let the host observe internals, blocks the usual MMIO windows into device memory, and establishes the CPR. Inside it, kernels execute exactly as they always did: no encryption on the arithmetic, no overhead per FLOP, no slowdown that scales with model size.

The seam is PCIe. On Hopper the link is not part of the trusted boundary, so data cannot move directly from encrypted guest memory into protected HBM. It is staged: the CVM encrypts a chunk under a session key negotiated with the GPU at startup, copies the ciphertext into a shared bounce buffer the hypervisor can see and cannot use, and the DMA engine pulls it inside the CPR and decrypts and authenticates it there. Results travel back the same way. Every byte crossing is encrypted, copied an extra time, and authenticated.

Overhead as a fraction of runtime therefore falls as arithmetic intensity rises, because the tax is proportional to bytes crossed and the useful work is not. A 70B model generating 2,000 tokens from a 30,000-token prompt keeps its weights resident and does quadratic prefill work on data already inside the CPR; a small model answering a short prompt does not. The Hopper benchmark study found exactly that curve: average overhead below 9%, most realistic LLM queries under 5%, approaching zero for large models and long sequences, with the residual attributed to data transfer rather than in-GPU computation (Zhu et al., 2024, arXiv:2409.03992).

Blackwell attacks the seam directly. NVIDIA describes it as the first TEE-I/O capable GPU, with inline protection over NVLink and AES-GCM engines at line rate; when the host also supports TEE-I/O, the bounce buffer disappears and the link joins the trusted boundary. Independent benchmarking supports the direction and adds a caveat worth more than the headline: tuned configurations land at roughly 1% to 3% throughput overhead, while stock inference stacks pay 30% to 40% for reasons that are configuration rather than architecture (arXiv:2608.26575). Flip the flag, measure on day one, and you will measure the bad number and conclude the technology is not ready.

Attestation: the part people skip

Encrypted memory is worthless on its own, because the party who benefits from lying about it is the party running the machine. An enclave nobody can inspect is indistinguishable from an ordinary process claiming to be one.

What makes the claim checkable is that the hardware signs a statement about what it loaded. On SEV-SNP that is a launch digest over firmware, kernel, initrd, command line and launch configuration, signed under a key derived in the AMD secure processor. On TDX the trust domain's measurement registers are quoted through an SGX-based quoting enclave. The GPU signs separately, under a device certificate: firmware versions, confidential-mode state, measurements of loaded microcode. A verifier compares those against golden values in a Reference Integrity Manifest and returns a signed Entity Attestation Token rather than raw evidence (NVIDIA, Remote Attestation Service). Where the CPU is Intel, a full appraisal composes two verifiers: Intel Trust Authority validates the TDX quote and forwards GPU evidence to NVIDIA's service (Intel).

Composition is where implementations go wrong. Two valid tokens do not prove the attested GPU is attached to the attested VM; each is independently valid and neither mentions the other, so an attacker can pair a genuine GPU report from a correctly configured machine with a genuine CVM report from a machine whose GPU is in ordinary mode. The binding has to be built: the CVM includes the GPU evidence, or its hash, in data reported under its own key. Freshness is the cheaper half, a nonce the attester must include, with the enclave's public key bound into the same report so the encrypted channel terminates inside the enclave rather than in front of it. What no nonce fixes is the post-launch gap. An attestation says "this machine started from this image". It never says "this machine has not been compromised since".

From proof to control: gated key release

An attestation token is a document. It becomes a control when something refuses to act without it, and in practice that something is a key: secrets sit encrypted at rest, and the key manager releases the decryption key only against a fresh attestation whose measurements satisfy an appraisal policy.

Read that as a gate rather than an equation: \(m\) is the launch measurement, \(\mathcal{A}\) the allow-list of approved images, \(t_{\min}\) the minimum acceptable platform version, and the last conjunct the CPU-GPU binding above. Every clause dropped for convenience widens the set of machines that can obtain \(k\). The one teams drop first is the TCB floor, and it is the most expensive omission: without it, an attacker running your exact approved image on a platform with a known-broken security processor presents a perfectly valid report and collects your key. Get the gate right and two mutually distrustful parties can share one computation, the model vendor releasing its weight key and the customer its data key to the same measurement, with neither trusting the operator.

What the proof does not cover

Here is the load-bearing claim of this article, stated plainly. A measurement is an identity, not a description. If the approved image logs prompts to a bucket, attestation certifies faithfully that you are talking to the prompt-logging build, the key manager releases the key, and the customer's dashboard shows a green tick.

The gap has two closures, both as social as technical. Make the measurement mappable back to something inspectable, through reproducible builds or published binaries. Then publish every approved measurement to an append-only log and have clients refuse any node whose image is absent from it, which converts "ship one bad build to one user" from an invisible act into a permanently recorded one. Apple's verifiable transparency is that mechanism, and an independent analysis names its limit: compiled binaries rather than source add "a layer of opaqueness" to much of the specification (Unlocking Apple's Private Cloud Compute, 2026, arXiv:2605.24239).

A targeting problem remains that no software property addresses: leave the code honest and route one user to a watched machine. The answer is architectural, and Apple's threat model, which assumes an attacker with physical access to a node and the sophistication to subvert some hardware protections, is what forces each piece: an anonymising relay so nodes never see client addresses and the relay never sees contents, identity separated from payload, node selection outside the requester's control, and stateless nodes, so a seized machine yields nothing (Apple, 2024, Private Cloud Compute).

Seeing It in Motion

Cold start pays for attestation and key release once; every subsequent request pays only the transfer tax.

sequenceDiagram
    participant D as Client device
    participant N as Confidential VM
    participant G as GPU in CC mode
    participant V as Verifier
    participant K as Key manager
    D->>N: request with nonce
    N->>G: collect device evidence
    N->>V: CPU quote plus GPU evidence
    V-->>N: signed attestation token
    N->>K: token plus key request
    K-->>N: weight key if policy holds
    Note over N,G: weights decrypted, staged via bounce buffer
    N->>G: encrypted DMA of prompt
    G-->>N: logits inside enclave
    N-->>D: response over enclave-terminated channel

The second diagram is the one worth pinning above a desk: what is inside the boundary, and what merely looks like it is.

flowchart TB
  subgraph INSIDE[Inside the guarantee]
    W[Weights in HBM]
    KV[KV cache]
    P[Prompt during compute]
  end
  subgraph OUTSIDE[Outside the guarantee]
    T[Response timing and length]
    PW[Power and thermal traces]
    IR[Interrupt and scheduling patterns]
    OUT[Model outputs themselves]
    IMG[Choice of enclave image]
  end
  HV[Hypervisor and operator] -->|blocked| INSIDE
  HV -->|observes| OUTSIDE

  class W,KV,P emerald
  class T,PW,IR,OUT,IMG rose
  class HV slate
  classDef emerald fill:#047857,stroke:#34d399,stroke-width:1px,color:#fff
  classDef rose fill:#be123c,stroke:#fb7185,stroke-width:1px,color:#fff
  classDef slate fill:#334155,stroke:#64748b,stroke-width:1px,color:#e2e8f0

[IMAGE: Two stacked latency waterfalls for the same request, non-confidential above, confidential below, with segments for queueing, prefill, decode and response. The confidential bar adds a small amber "bounce buffer staging" segment at the front of prefill and a proportionally stretched decode segment. Caption: "The tax is paid at the boundary, then again, in miniature, on every token."]

Watch It Run

By the Numbers

Sources: Hopper figures from Zhu et al., 2024, arXiv:2409.03992; TDX serving figures from arXiv:2607.19353; Blackwell figures from arXiv:2608.26575; the MPC row from Dong et al., 2023, PUMA, arXiv:2307.12533. NVIDIA's own "nearly identical" throughput characterisation of Blackwell is a vendor claim; the tuned figure is the closest independent corroboration, and the stock-stack figure is the honest planning number.

The last row is calibration, not comparison: a factor of roughly \(10^{4}\) in latency is why production deployments needing confidentiality at LLM scale reach for hardware rather than mathematics.

[IMAGE: Line chart of confidential-mode overhead against arithmetic intensity, x-axis log scale from a 1B model on a 128-token prompt to a 70B model on a 30k-token prompt, one line for Hopper with bounce buffers and one for Blackwell with TEE-I/O. Both slope downward; the Blackwell line sits near the axis throughout. Caption: "The tax is charged per byte crossed, so the heaviest workloads pay the least of it."]

[IMAGE: Log-scale horizontal bar chart of per-token latency for four approaches: plaintext GPU inference, confidential GPU inference, TEE-based notarisation, and MPC inference (PUMA, LLaMA-7B). Bars annotated with the measured value and the source year. Caption: "Confidential computing is a rounding error next to cryptographic inference. That is the entire reason it won."]

A Concrete Example

Take a single node serving Mistral-7B in FP16 on an H100 inside an Intel TDX trust domain, and walk one cold start plus one request. Percentages below are measured; the baseline latencies are illustrative round numbers chosen to make the arithmetic replayable.

Step 1: launch and attest. TDX produces a quote over the guest launch image; the GPU produces device evidence. Intel Trust Authority validates the quote and forwards the GPU evidence to NVIDIA's verifier, which compares it against the Reference Integrity Manifest. Two signed tokens come back, and the CVM has included the GPU evidence hash in its own reported data, so the binding holds.

Step 2: policy evaluation. The key manager checks six clauses. Certificate chain valid. Nonce matches. Launch measurement sha384:9f2c… present in the allow-list. Platform TCB version 0x0A, above the floor of 0x09 set after the last firmware advisory. Debug bit clear. GPU evidence bound. All six hold, so the weight key is released under a 60-minute lease.

Step 3: load the weights. Mistral-7B in FP16 is about 14.5 GB. At a practical 50 GB/s over PCIe Gen5 that is roughly 0.29 s non-confidentially. In Hopper confidential mode the same bytes are encrypted, written to a bounce buffer, DMA'd, then decrypted and authenticated; model the staging as halving effective bandwidth and the load becomes roughly 0.6 s. That is a once-per-cold-start cost, and the largest single confidential-mode penalty the node ever pays.

Step 4: serve one request. A 600-token prompt, a 300-token answer, baseline TTFT 180 ms, decode at 42 tokens/second. TTFT rises by the measured 21.8% to \(180 \times 1.218 = 219\) ms. Decode throughput falls by the measured 17.7% to \(42 \times 0.823 = 34.6\) tokens/second, so decode time goes from \(300 / 42 = 7.14\) s to \(300 / 34.6 = 8.67\) s. Total wall clock: \(0.18 + 7.14 = 7.32\) s becomes \(0.219 + 8.67 = 8.89\) s, a 21.4% increase.

Step 5: what the operator sees. Nothing readable. Guest memory is ciphertext, HBM is inside the CPR, the bounce buffer holds AES-GCM ciphertext. What the operator does see: a request arrived at \(t_0\), the node was busy for 8.89 seconds, the response ran roughly 300 tokens, and inter-token gaps averaged 29 ms. If the sensitivity is "did this user ask a long question about a merger", memory encryption did not help. That is the observation behind response padding being a real requirement in some deployments.

Step 6: the lease expires. At 60 minutes the lease ends and the node re-attests. If the fleet has meanwhile moved to firmware with a new TCB version, the measurement still matches but the version floor needs raising first, and a node whose policy was not updated stops serving. This is the ordinary way attested fleets break, and it is an operations problem rather than a security one.

Where It Breaks

The hypervisor is adversarial and still holds the levers

A confidential VM names the hypervisor as the attacker, then hands that attacker scheduling, interrupt delivery, page-table management and the ability to read guest memory as ciphertext. AMD SEV encrypts memory deterministically per physical address with no nonce or freshness counter, so identical plaintext at the same address yields identical ciphertext. CipherLeaks monitored ciphertext in the VM save area across context switches, recovered register values, and broke constant-time RSA and ECDSA in OpenSSL (Li et al., 2021, USENIX Security). Nothing there is implemented incorrectly; encryption without freshness simply has that property, which is why mitigations have been compiler-level and partial.

Two families do not merely observe. Heckler has a malicious hypervisor inject interrupts the guest never requested, altering guest data and control flow on both SEV-SNP and TDX (Schlüter et al., 2024, arXiv:2404.03387); WeSee injects #VC exception 29 so the guest's own handler copies data and registers out to the hypervisor (Schlüter et al., 2024, arXiv:2404.03526). These break integrity: the enclave executes code paths the attacker chose. TDXdown then demonstrated single-stepping and instruction counting against Intel TDX while defeating the platform's own countermeasure (Wilke, Sieck & Eisenbarth, 2024, ACM CCS), a reminder that "we mitigate single-stepping" is a version-specific claim. Nor is the root of trust immune: in 2026 researchers reported extracting the root VCEK seed from EPYC Milan parts by software alone, on systems reporting themselves fully patched (arXiv:2605.12990). With a root seed, valid-looking reports can be forged for machines that are not in the state they claim.

The GPU half has had far less adversarial attention

CPU TEEs have absorbed a decade of attacks. Confidential GPUs have had a few years, and the surface differs in kind. Power and thermal traces are analogue, accessible without special privilege, and cannot be virtualised away; Energon recovers transformer architecture details from exactly those channels (arXiv:2508.01768), and NVBleed demonstrates covert and side channels over NVIDIA's multi-GPU interconnect (arXiv:2503.17847). Defensive work proposes PCIe traffic shaping and physical page remapping as a complement to confidential computing rather than a replacement, which tells you how its authors rate the residual risk (CloakLM, 2026, arXiv:2606.18400). Partitioning is not clean either: surveys of A100 and H100 report MIG does not partition the last-level TLB (arXiv:2408.11601).

The operational failures are duller and more likely

Statelessness fights every instinct an on-call engineer has: no request traces, no replay of a failing prompt, no per-user audit trail. Someone adds "temporary" diagnostic logging during an incident, it survives the incident, and the property the architecture existed to provide is gone without anyone deciding to give it up. Allow-lists rot the same way, because every firmware update changes measurements and removing a stale entry breaks a node; a year in, the list holds every image the team ever shipped, bugs included.

Keys cannot be recalled, because there is no mechanism to reach inside a running enclave and take a secret back. Revocation means rotating the key and re-encrypting the data, so the only design that bounds exposure is short leases with frequent re-attestation, which puts cold-start attestation on the critical path after every scale-out, migration or preemption.

Confidentiality is not privacy

The most common category error in procurement. A TEE does nothing about what the model's output reveals. Memorised training data still comes out, membership inference still works, and a model fine-tuned on one customer's corpus still leaks through its answers. Encrypted memory and differential privacy solve orthogonal problems.

[IMAGE: A layered attack-surface diagram of a confidential inference node. Concentric bands from the centre outward: model weights and KV cache (green, protected), enclave software (yellow, protected but chosen by the provider), hypervisor and host (red, excluded but holds scheduling and interrupts), physical platform (red, power and thermal observable). Arrows from named attacks point at the band each one targets: CipherLeaks, Heckler, WeSee, TDXdown, Energon, NVBleed. Caption: "Every published break either exploits the powers the threat model gives the hypervisor, or a channel the threat model never covered."]

Alternative Designs

The comparison that matters most is the first row against the last. Confidential computing and a zero-retention contract address the same worry with different instruments: one makes reading the data technically difficult for a defined adversary, the other makes it legally costly for a cooperative one. Confusing them produces either wasted overhead or unfounded confidence. Zero-knowledge approaches earn their row because they answer a question TEEs cannot, namely "was this really computed by the model you claimed", though as of 2026 the overhead for full network proofs remains orders of magnitude above real-time (arXiv:2502.18535).

How It Is Used in Practice

Three deployment shapes dominate, and they differ mainly in who controls the allow-list.

Provider-run, user-verified. Apple's PCC is the fullest public example: attestation plus a transparency log of approved images, clients that refuse unlogged nodes, stateless computation, an anonymising relay, and a virtual research environment that makes the log more than a gesture (Apple, Security research on Private Cloud Compute). The 2026 extension to Google Cloud kept those properties and added an append-only ledger of the third-party hardware admitted to the fleet, because once the machines are not yours, "which machines are eligible" becomes a targeting surface of its own (Apple, 2026).

Cloud-offered, customer-configured. Azure made confidential VMs with NVIDIA H100 NVL generally available in 2024, on AMD EPYC with SEV-SNP, initially in two regions (Microsoft, 2024). The customer owns the enclave image and therefore owns the guarantee, which is the shape that satisfies a regulator: the party making the privacy claim is the party that chose the code.

Vendor-protected weights. A model vendor deploys into a customer's confidential environment and releases weight keys only to its own approved measurement. The customer gets data locality; the vendor gets protection against weight exfiltration from disk, and none against distillation through the model's own API.

Capacity is the quiet constraint across all three: confidential GPU SKUs are narrower and in fewer regions than ordinary capacity, and TEE-I/O capable hosts narrower still.

[IMAGE: Three-panel comparison of deployment shapes, each showing who holds the allow-list key. Panel 1 "provider-run, user-verified": provider holds it, transparency log below with many client eyes on it. Panel 2 "cloud-offered, customer-configured": customer holds it, cloud operator outside the boundary. Panel 3 "vendor-protected weights": vendor holds it, customer data flows in from the side. Caption: "The architecture diagrams look identical. The question that distinguishes them is who decides which image is approved."]

Insights Worth Remembering

1. A TEE proves identity, not behaviour. The measurement is a hash of what was loaded. An image that exfiltrates prompts attests exactly as cleanly as one that does not, which is why every serious deployment pairs attestation with something that constrains which images may be approved.

2. The overhead is a transfer tax, so it shrinks as workloads grow. Confidential mode costs nothing per FLOP. Budget against bytes crossing the host-device boundary, and expect small models on short prompts to look worst, which inverts the usual capacity-planning intuition.

3. The number you measure on day one is not the number in the paper. Stock stacks on Blackwell give up 30% to 40% where a tuned stack gives up 1% to 3%. Treat a bad first measurement as a configuration bug, not a verdict on the technology.

4. Two valid attestation tokens are not one valid attestation. CPU and GPU evidence must be explicitly bound, or an attacker pairs reports from two different machines and passes your verifier.

5. Released keys cannot be recalled, so leases are the real control. Exposure is bounded by lease duration rather than by any revocation event, which makes re-attestation frequency a security parameter and not an implementation detail.

6. Memory encryption hides contents, never shape. Timing, length, interrupt patterns, power and interconnect traffic sit outside the threat model, and published attacks recover real information from each. If response length is sensitive, pad it; the hardware will not.

7. Confidentiality and privacy are different purchases. A perfect enclave returns the true output, and outputs leak memorised data and membership. Buying a TEE and calling the system private is the most common error in this space.

Open Questions

Can attestation ever say something about behaviour rather than identity? Measured launch is solved; meaningful runtime attestation is not. Work on extending measurement to post-launch events exists, but no shipping platform asserts "this machine has not been compromised since boot". Whether that gap is closable, or only manageable through short leases, is open.

Does a transparency log work without a research community? The property "misbehaviour becomes discoverable" is realised only if people are equipped and motivated to look. Apple's research environment shows the model can be supported; it is not evidence it scales to every provider who publishes a log.

How much does confidential mode cost for training? Most public measurement is inference. Training moves far more data across the boundary, at a different rhythm, with inter-node collectives that confidential networking must also cover. Reported inference overheads should not be extrapolated.

Will side-channel resistance be designed in, or patched in? The recurring pattern is disclosure, targeted firmware fix, new disclosure. Deterministic memory encryption without freshness on SEV is a design property rather than a bug, and the 2026 root-seed extraction on supposedly patched systems suggests mitigation cadence alone may not converge. It is likely, though not established, that the next platform generation treats these channels as in scope from the start.

Does a regulator accept "technically unable to read it"? Vendors increasingly present confidential computing as satisfying data-residency and processor-access obligations. Whether that is equivalent to "not a processor of it" is being worked out jurisdiction by jurisdiction, and the answer will shape adoption more than any benchmark.

Sources and Further Reading

1. Apple Security Engineering and Architecture (2024). "Private Cloud Compute: A new frontier for AI privacy in the cloud." security.apple.com
2. Apple (2026). "Expanding Private Cloud Compute." security.apple.com
3. Apple. "Security research on Private Cloud Compute." security.apple.com
4. Dhanuskodi, G., et al. (2023). "Creating the First Confidential GPUs." ACM Queue, 21(4). queue.acm.org
5. Misono, M., et al. (2024). "Confidential VMs Explained: An Empirical Analysis of AMD SEV-SNP and Intel TDX." Proceedings of the ACM on Measurement and Analysis of Computing Systems. doi:10.1145/3700418
6. Zhu, J., et al. (2024). "Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Study." arXiv:2409.03992
7. "Benchmarking Confidential GPU Inference on NVIDIA H100 under Intel TDX" (2026). arXiv:2607.19353
8. "Benchmarking Confidential Computing Performance on NVIDIA Blackwell GPUs" (2026). arXiv:2608.26575
9. "Confidential Computing on Heterogeneous CPU-GPU Systems: Survey and Future Directions" (2024). arXiv:2408.11601
10. Li, M., Zhang, Y., Wang, H., Li, K., & Cheng, Y. (2021). "CIPHERLEAKS: Breaking Constant-time Cryptography on AMD SEV via the Ciphertext Side Channel." USENIX Security. usenix.org
11. Schlüter, B., et al. (2024). "Heckler: Breaking Confidential VMs with Malicious Interrupts." USENIX Security. arXiv:2404.03387
12. Schlüter, B., et al. (2024). "WeSee: Using Malicious #VC Interrupts to Break AMD SEV-SNP." IEEE S&P. arXiv:2404.03526
13. Wilke, L., Sieck, F., & Eisenbarth, T. (2024). "TDXdown: Single-Stepping and Instruction Counting Attacks against Intel TDX." ACM CCS. doi:10.1145/3658644.3690230
14. "Insecure Despite Proven Updated: Extracting the Root VCEK Seed on EPYC Milan via a Software-Only Attack" (2026). arXiv:2605.12990
15. "Energon: Unveiling Transformers from GPU Power and Thermal Side-Channels" (2025). arXiv:2508.01768
16. "NVBleed: Covert and Side-Channel Attacks on NVIDIA Multi-GPU Interconnect" (2025). arXiv:2503.17847
17. "CloakLM: Obfuscating GPU Memory Layout to Mitigate Model Ex-filtration for Serving" (2026). arXiv:2606.18400
18. "Unlocking Apple's Private Cloud Compute: An Analysis of Privacy-Preserving Artificial Intelligence" (2026). arXiv:2605.24239
19. Dong, Y., et al. (2023). "PUMA: Secure Inference of LLaMA-7B in Five Minutes." arXiv:2307.12533
20. "A Survey of Zero-Knowledge Proof Based Verifiable Machine Learning" (2025). arXiv:2502.18535
21. NVIDIA. "NVIDIA Remote Attestation Service." docs.attestation.nvidia.com
22. Intel. "GPU Remote Attestation With Intel Trust Authority." docs.trustauthority.intel.com
23. Microsoft (2024). "General Availability: Azure confidential VMs with NVIDIA H100 Tensor Core GPUs." techcommunity.microsoft.com
24. Linux Foundation (2019). "Confidential Computing Consortium Establishes Formation with Founding Members and Open Governance Structure." linuxfoundation.org

Free to read, no ads, no sign-up. If it was useful you can buy me a coffee.