One GPU, or something like it

The most consequential object in KEP-5004 is the one that does not change. It is a Deployment: an image, a command, and limits: example.com/gpu: 1. Anyone who has run accelerators on Kubernetes has written it, or inherited it, or copied it out of a vendor's quickstart. The entire purpose of the document it sits in is to guarantee that you will never have to write anything else.

On 26 August, Kubernetes v1.37 — Garhwal — shipped, and that guarantee became permanent. The release notes record it in a single line: Promoted the DRA extended resource feature to GA in v1.37. The enhancement behind it, filed on 3 February 2025 by the contributor @yliaog and owned by SIG Scheduling, is now marked implemented. Everything that follows is read out of the projects' own repositories — the changelog, the proposal, the API types — because that is where this particular story is written down.

It is worth appreciating what this arrives on top of. Dynamic Resource Allocation is one of the largest sustained pieces of API work in Kubernetes right now, and it exists because the old model was wrong. A GPU is not a unit of anything. It has a memory size, a NUMA locality, a driver version, a link topology, a partitioning scheme, and a health state, and two devices that answer to the same integer can differ by an order of magnitude in what they cost and what they can run. So DRA replaced the count with a description: drivers publish ResourceSlice objects listing devices and their attributes, workloads make ResourceClaims, and the scheduler matches the two with CEL expressions. In v1.37 alone the model gained derived attributes computed in CEL, a standard resource.kubernetes.io/numaNode attribute, alpha device compatibility groups, and taints and tolerations for devices at GA. Of the 515 bulleted entries in the v1.37.0 changelog, 53 mention DRA or one of its API objects. It is not a side project.

And at the moment that model reached its maturity, the thing promoted to stable was the right to ignore it.

That is not a criticism of the design, and it is certainly not an accident. The KEP is unusually candid about the trade. Its summary states that extended resources provide "a simple, concise approach to describe resource capacity, and resource consumption", while DRA is "a more expressive, flexible approach, yet more complicated, and harder to use". Its goals then close off the obvious reading: extended resource support "is not added just for easing the transition to DRA for the short term. Its ease of use is one big advantage to keep it remaining useful for the long term." This is not a compatibility shim with a sunset. It is a declaration that the count is the interface, and the description is the implementation.

Which raises the question of where, exactly, the description went — because it did not disappear. It moved.

Here is the mechanism. A cluster administrator creates a DeviceClass, a cluster-scoped object whose spec carries a list of selectors — "Each selector must be satisfied by a device which is claimed via this class" — written as CEL expressions over device attributes. The class may also carry an extendedResourceName. Set it to example.com/gpu, and every pod in the cluster asking for example.com/gpu: 1 is now asking for a device that satisfies that CEL. More striking still, you do not have to set it: the KEP specifies that every device class is implicitly addressable as deviceclass.resource.kubernetes.io/<device-class-name>. There is no device class in a v1.37 cluster that cannot be requested as an integer.

When such a pod is scheduled, the scheduler builds a ResourceClaim on its behalf — the KEP calls it "a special resource claim", the changelog calls them synthesized — instantiating it during the Filter phase and persisting it at PreBind. The pod spec never mentions it. The kubelet then does something quietly radical: it declines to check the request against the node. The KEP is explicit that the pod's extended resource requests "should not be checked against the Node's allocatable, as the resources are in ResourceSlice, not in Node". The device IDs reach the container through CDI, and the only record of which container got which request is pod.status.extendedResourceClaimStatus, a structure holding the generated claim's name and a list mapping container name and resource name to a request name inside it.

Read that as an architect rather than as an operator. The pod spec — the artifact in git, the thing under review, the thing an ADR points at — says 1. The decision about what 1 means lives in a cluster-scoped object that a platform team or a vendor's Helm chart created. The record of what was actually allocated lives in status, which is to say it exists while the pod does and is reconstructed, not remembered. A share of v1.37's bug fixes land in exactly that seam: one change preserves the DRA status fields when an older client sends a status update that omits them, because losing them could leave pods "permanently stuck in Terminating".

The counting problem is sharper still, and the KEP works it through honestly. Three quota mechanisms can now bear on the same physical device: the extended resource quota, the implicit device-class extended resource quota, and the resource claim quota. Before the adjustment the KEP proposes, its own worked example shows five GPUs — allocated variously through a device plugin, an explicit extended resource, an implicit one, a claim and a claim template — producing used counts of 2, 1 and 4 under the three quotas. After the adjustment: 5, 4 and 4. The same five devices. The proposal is right that these should agree, and it is doing the work to make them agree. But the fact that the reconciliation had to be designed, argued and staged across releases is the interesting part. Counting is not a detail bolted on after allocation. Counting is the thing organisations actually govern with.

The strongest case against reading this as a loss is a good one, and it is the case for abstraction itself. Application teams should not write CEL expressions about NUMA nodes. That is precisely the knowledge a platform team exists to hold, and pushing device topology into every Deployment would be a failure of encapsulation, not a triumph of expressiveness. On this view v1.37 is the system working: a rich substrate underneath, a narrow contract on top, and a clean line between who knows what. Kubernetes has always done this. Nobody writes a CSI volume attachment by hand either.

The analogy is where it breaks. A PersistentVolumeClaim names a StorageClass, and the class is in the manifest; you can read a pod spec and know which tier of storage it asked for. The extended resource path deliberately removes that. The DeviceClass is resolved by name matching at scheduling time, and the API's own doc comment on extendedResourceName says it "should be unique among all the device classes in a cluster" — should, not must — and then specifies the tie-break: if two classes claim the same extended resource name, the one created later wins, and if they were created at the same instant, the lexicographically first. The meaning of the number in your manifest is settled by a creation timestamp. That is a reasonable engineering answer to an unavoidable conflict, and it is also an admission that the binding is not something the workload owns.

The deeper consequence is economic, and it is the reason this is worth an architect's attention rather than a release-notes skim. An organisation governs the unit it can count. Quota is denominated in counts. Chargeback is denominated in counts. The capacity plan in the spreadsheet, the reservation with the cloud provider, the argument between two teams about who gets the H100s next quarter — all counts. DRA's expressiveness is real, but it now sits below the line where money is measured. The people who will spend it are driver authors, making heterogeneous silicon schedulable; the people who will not are the ones deciding what to buy and who gets it, because their instrument still reads in whole devices. When a MIG slice and a full accelerator both decrement the same quota by one, the ledger and the hardware have quietly stopped describing the same world, and no amount of attribute richness underneath will reconcile them.

None of this makes v1.37 a wrong release. The alternative — forcing every workload in every cluster to be rewritten as a ResourceClaim — was never going to happen, and pretending otherwise would have left the expressive model stranded in a corner of the ecosystem. The compatibility path is why DRA will actually be adopted.

But there is a version of success worth naming in advance. Years of work produce a device model that can describe anything, deployed under an interface that can describe one thing. The scheduler gets cleverer every cycle; the pod spec, the quota and the invoice stay at 1. The system will run better and the organisation will not see any further than it did before. Expressiveness that never reaches the ledger is not a capability the buyer has. It is a capability the vendor has, on the buyer's cluster, at the buyer's expense — and the first sign that it has been spent well or badly will not appear in any object you are currently watching.