The search has no error code

An agent asks for a page it is not allowed to have, and the platform says so plainly. The call comes back as an error result — is_error: true, the code url_not_allowed — and that error becomes a line in the session transcript and a tick in the failure count on the console's tools panel. Weeks later, someone reconstructing what the agent did can point at the moment the policy bit.

Now give the same agent a search instead of a fetch. Nine pages would have answered the question; four of them sit on hosts the allowlist does not cover. The tool returns five results. There is no error, no code, no tick. The transcript records a successful call with five results in it, which is exactly what it would have recorded if five were all the web had.

That second behaviour grew teeth on 7 October, when Anthropic's Managed Agents began applying a cloud environment's allowed_hosts list to the two web tools that run on Anthropic's servers rather than inside the sandbox. The release note states both halves in one breath: a web_fetch call for a URL on a host the list does not match "returns a url_not_allowed error result to the agent", and web_search "omits results from such hosts". The reference documentation repeats it as a rule — search omits, fetch errors — and adds the limiting case: when allowed_hosts lists no hosts at all, neither tool returns a page or a search result.

The asymmetry is not an oversight, and it is not a logging bug. It is a difference in kind between two ways of denying something, and only one of them produces a fact about the system. A refused fetch is an event: it has a code, a timestamp, a place in a sequence, and a counter that increments. A filtered search result is an absence, and absences are not events. They are properties of a world that was assembled for the model and then discarded. The transcript can tell you what the agent read. Nothing in the session — not the event stream, not the tools panel, not the downloadable JSON — can tell you what it was kept from reading.

This matters because the two denials fail in opposite directions. A blocked fetch stops the agent from doing something, and the agent notices; a well-built one re-plans, or says it could not get the document. A filtered search does not stop the agent from doing anything. It changes what the agent believes is true, and then lets it proceed with full confidence on a smaller evidence base. The console's tools panel reports call counts, failures and median duration; a refused fetch arrives there as an error result, and a filtered search arrives as a call that worked. The instrument that exists counts exactly the half that was already visible.

Consider what that does to the questions people actually ask after the fact. What did the agent read? is answerable, and the platform answers it well. Was this conclusion drawn from a complete view of the available evidence? has no answer at all, because the only artefact that could contain one — a record of what the policy removed — is never written. The first question is the one asked in a demo. The second is the one asked in an incident review, in a procurement questionnaire, and by anyone deciding whether a recommendation can be acted on. An agent operating under a narrow allowlist and an agent operating under a wide one produce transcripts with the same shape, and nothing in either says which was which.

It gets harder when the lists move. A tool's domain lists can be changed on an idle session, and the new lists apply to the rest of that session, so a single transcript can span two evidence regimes with no marker at the boundary. The two layers of the policy do not even match the same way: an entry in a tool's own list covers its subdomains, while an allowed_hosts entry matches one exact host unless it begins with *.. That mismatch is caught loudly — a session whose allowed_domains strays outside allowed_hosts fails to start with a 400 — which tells you where the platform's attention is. A configuration that contradicts itself is a validation error at create time. A configuration that quietly thins the agent's evidence at run time is just Tuesday.

There is a second structural wrinkle, and it is the more interesting one. To let the web tools reach a host, you add it to allowed_hosts — which, as the release note says, also opens that host to the sandbox, where the agent's own code runs. Reading and reaching are the same switch. Widen what the agent may learn from and you widen where arbitrary code can send bytes. The same release closed the obvious consequence from the other end: web_fetch in a managed session now fetches only URLs that have already appeared in that session, and a URL that exists only in the model's own output, the system prompt, an attached document, or the output of bash or an MCP tool returns url_not_in_prior_context. Anthropic's stated reason is to reduce the risk of data exfiltration, and the design is sound. But look at what the architect is now holding: a single dial where evidence and exfiltration move together, and a provenance rule deciding what is reachable based on where a string came from. Both are reasonable. Neither is recorded as having shaped the answer.

The strongest case against my complaint is a good one, so let me put it at full strength. Telling the model what was withheld would be worse than withholding it silently. A model that can see the shape of its own filter will try to work around it, and the titles and URLs of the suppressed results are precisely the payload an injected page would like returned. Error messages are an exfiltration channel and a reconnaissance channel at once. Quiet denial is the correct behaviour at that boundary, and it is the behaviour of every mature control we already trust: a NetworkPolicy does not explain itself to the pod, and nobody thinks it should.

That defence is right about the model and silent about the operator, and today they are served by the same record. The Kubernetes precedent is worth reading closely, because the project is honest about the gap. The NetworkPolicy documentation lists, among the things the API cannot do, "the ability to log network security events (for example connections that are blocked or accepted)". A decade of flow logs, eBPF tooling and policy-simulation products grew up in that hole, and we tolerated the wait because a dropped packet eventually becomes a timeout, a retry, a stack trace — a consequence with a shape, loud enough that someone goes looking. Evidence has no timeout. A thinner search result set produces a well-formed, fluent, plausible answer, delivered on time and within budget. The old pattern was safe because denial degraded availability. The new pattern is not, because denial degrades conclusions, and conclusions do not page anyone.

This is not really a story about one vendor's beta, which is why the detail is worth knowing. Permission-filtered retrieval does the same thing wherever a system trims its candidate set by the caller's entitlements: return what survives, log the query and the hits, record nothing about the trim. The answer that comes back is shaped by an access-control decision that appears nowhere in the artefact it shaped. We built that pattern deliberately — showing a user the titles of documents they cannot open is a leak — and then we started treating its output as analysis rather than as a view. The fix is not to tell the model more. It is to write down, beside the output and not inside the context, which policy was in force and how many candidates it removed: a policy identifier and a count, never the contents. That preserves every security property of silent filtering and restores the one thing it costs, which is the ability to say later that the view was partial.

Until something like that exists, the honest reading of these controls is that they are access policy wearing the costume of a security control. They are good at the thing that can be enforced at a boundary and blind to the thing that happens afterwards, in the model's estimate of what is true. Most teams will never see it, because an environment created through the API with the networking field omitted gets unrestricted, and unrestricted does not filter these tools at all — the console's form, by contrast, starts at limited with nothing allowed. The default you inherit decides whether you are running with an invisible filter, and the two defaults disagree.

Everything above is argued from one party's own documentation, read this week; there is no independent record of this behaviour, because the behaviour is the product. That is itself part of the point. The agent will not tell you it was shown less. It will simply be more certain than its evidence deserved, and it will be right often enough that you stop asking.