The screen is not a contract

The detail worth sitting with is not the money. It is the hardware.

According to reporting by The Information at the end of August, OpenAI has been buying Mac minis and Mac Studios in the tens of thousands, for reinforcement learning on computer-use agents. Neither OpenAI nor Apple has confirmed it, and Anthropic is said to be renting similar machines through AWS. Take the numbers as reported rather than established. The shape of the claim is what matters, and the shape is this: to teach a model to operate software, you apparently need rooms full of the same desktop computers that people use, running the same operating system, drawing the same windows, so that something can look at them.

Thirty years of integration work went into making that unnecessary. The whole project of services, contracts, message formats and machine-to-machine protocols was an argument that systems should not have to look at each other. And here is the most heavily capitalised industry in the world, buying desktops by the pallet so that its models can learn to squint at a screen.

That is not a failure of ambition. It is a bet, and it is a rational one: that the long tail of the world's software will never expose the interface you need, so the cheapest path is to give up asking and use the interface that already exists. My argument is not that the bet is wrong. It is that the industry is about to file the result in the wrong category. Computer use is a migration technology. It is being booked as an integration architecture. Those are different things with different lifespans, and confusing them is how organisations end up with systems nobody can specify.

Start with what the benchmarks actually say, because they are unusually candid this year. In June, the XLANG Lab published OSWorld 2.0, a set of 108 long-horizon desktop workflows where the median task takes a competent human about 1.6 hours, roughly forty-eight times longer than the two-minute median of the original OSWorld. The paper opens by noting that Claude Opus 4.8 reaches 83.5% on OSWorld-Verified, a number that, in its own words, suggests "desktop computer use is largely solved." On OSWorld 2.0, the best result under a 500-step budget and strict binary completion is 20.6%.

The interesting part is the paper's explanation, which is not about model quality at all: longer horizons "compound execution load and state-management errors." Read that as an architect rather than a researcher and it stops being a benchmark result. State management is compounding badly because there is no state to manage. A screen does not expose one. It shows a rendering of one, at a moment, to whoever is looking.

This is the gap that the word "interface" papers over. An API is a contract: a schema, a version, an idempotency story, defined errors, a rate limit, an audit trail, and somewhere behind it a team who can be held to it. A screen has none of those and was never asked to. It has pixels, and a human on the other side who supplies everything the pixels leave out — memory of what they just did, a sense of whether the save actually happened, and the judgement to stop when something looks wrong.

Read the vendor documentation with that in mind and it becomes remarkably honest. Anthropic's computer use tool gives a model seventeen actions: clicks, drags, keystrokes, scrolls, screenshots. Actions can be batched, but the batch runs sequentially and stops at the first failure, marking the remainder with the literal string "Not executed: an earlier computer action in this turn failed." That is not a transaction. There is no rollback, because there is nothing to roll back to. Each observation costs between a thousand and eighteen hundred tokens, and past twenty images the limits tighten and you have to start pruning history, which is to say you pay continuously to look at something whose entire purpose is to be looked at once by someone who then remembers it.

The security guidance is more telling still. Use a dedicated virtual machine with minimal privileges. Do not give the model access to credentials. Restrict the internet to an allowlist. Ask a human to confirm anything with meaningful real-world consequences, including financial transactions and agreeing to terms. Screenshots are scanned for prompt injection, because a screen is an untrusted input channel the moment something other than a person is reading it. Every one of those is sound advice. Collectively they describe something that is explicitly not an unattended integration. They describe a supervised bridge.

Now the counterargument, which is strong and which I think is largely correct on the facts.

APIs never covered the long tail and never will. The vendor of the twelve-year-old claims system has no commercial reason to build you an endpoint. The internal team that owns the scheduling tool has no budget and a two-year queue. Half the systems that matter in a large organisation are reachable only by a person with a login, and that will still be true in 2030. Against that, an agent that can drive anything with a window is worth more than a contract you cannot obtain. This is not a hypothesis. Robotic process automation became a multi-billion-dollar category on precisely this observation, and it became one because it was right. Universality beats elegance in procurement every single time, and computer-use agents generalise across interfaces in a way that scripted RPA never managed. They will be adopted regardless of anything an architect writes in a decision record.

I agree with all of that, and it is the reason the filing error matters rather than a reason it does not.

Because the lesson of RPA was never that screen automation does not work. It worked. The lesson is what happened next: the bridge became permanent, and the maintenance bill outlived the business case. Organisations discovered they were spending more keeping the robots working than the automation had saved, because a temporary path around a missing interface had quietly become the interface, and nobody owned the thing it depended on. When the vendor moved a button, a process broke, and the breakage was at least loud.

Computer-use agents are better than RPA in exactly the way that makes this worse. A model that can find the button after it moves will not break when the button moves. It will adapt, and keep going, and the failure that used to arrive as an alert now arrives as a success rate of twenty percent on anything that takes longer than an afternoon, distributed across a hundred workflows that each look fine in isolation. Robustness at the mechanism level buys silence at the outcome level. That is the trade, and it is not obviously a good one.

So the practical question is not whether to use computer use. It is what you write down when you do.

Treat every agent-driven screen path as a bridge with an owner and an expiry date. Record the API that should have existed and did not, and what it would have cost, because that number is the only thing that will ever move the vendor or the internal backlog. Measure the completion rate in production rather than quoting the benchmark, and measure it on whole workflows, not steps, because the steps are the part that already works. Keep the human confirmation on consequential actions long after it feels like a formality. And put a date on it, the way you would with any other debt that carries a maturity.

The deeper thing being priced here has nothing to do with Apple's supply chain. If the reporting is right, a great deal of capital is being spent to preserve an interface that exists solely because human beings have eyes, at exactly the moment when the systems using it will not. That is a bet that the world's software will not be re-plumbed. It may well pay off. It usually does, in the sense that the cheap path around a structural problem usually does pay off, right up until the point where the structure is what you needed.

We spent three decades teaching systems to talk to each other. We are now spending a good deal more teaching them to watch each other work. The risk is not that they will be bad at watching. It is that in five years the only surviving description of what a system was supposed to do will be a recording of something doing it.