Editorial

The rest of this site teaches, and teaching material has no opinion. This is where the opinion goes. Each piece starts from something that actually happened in artificial intelligence or in software architecture — a release, a result, a failure, a shift in how the work is done — and argues its way to a position: what genuinely changed, what is only being marketed as change, and what a working architect should do differently because of it. Positions are arguable on purpose, and everything they rest on is linked and dated.

50 editorials 6 strands 358 cited sources 82,736 words

Benchmarks, claims, measurement, and the distance between what is demonstrated and what is sold.

Evidence & Evaluation 18 October 2026 8 min read New

A drill the operator chose

On Saturday AWS will cut its European Sovereign Cloud off from the rest of AWS for several hours and show that it keeps running. That is real engineering, and it answers the half of sovereignty that a vendor can schedule for itself.

A disconnection drill proves a sovereign cloud can run unchanged without its parent, but sovereignty is decided by the events nobody schedules, a compelled instruction or a patch that cannot arrive.

Read it → 5 sources sovereign cloudresilienceoperational independencecompliance evidence
Evidence & Evaluation 15 October 2026 7 min read New

Selected is not filtered

A one-line fix in Feast 0.67.0 restores the point-in-time filter for the Polars backend. What it exposes is where that guarantee had been living — not in the feature definition, not in the engine contract, but in an indexing expression whose meaning changes with the dataframe library underneath it.

Feast's point-in-time join lived in a pandas indexing idiom rather than in its backend contract, so under the Polars backend it selected columns instead of filtering rows and a training set kept the future or stopped according to the shape of the joined frame.

Read it → 10 sources feature storespoint-in-time correctnessdata leakageportability
Evidence & Evaluation 14 October 2026 8 min read New

Only the tokens are recorded

On 6 October a general-purpose judgement endpoint appeared in OpenAI's official clients: ask a named question about a piece of text, get back a typed answer with a probability. The response carries no identifier, no timestamp and no way to fetch it again — and it counts its own cached tokens to the unit.

The Decisions API gives an architecture a policy decision point whose answer has no identifier, no timestamp and no retrieval, while metering the same call to cached and reasoning tokens — so a machine judgement's only durable record is its bill.

Read it → 8 sources decision endpointsauditclassifiersagents
Evidence & Evaluation 8 October 2026 7 min read New

The search has no error code

An agent's network allowlist now governs what it may read as well as what it may reach. A blocked fetch leaves an error in the record; a blocked search result leaves nothing at all, and the agent answers anyway.

One allowlist refuses an agent's fetch with a recorded error and its search results with nothing at all, so the half of the policy that decides what the agent comes to believe is the half that leaves no trace in the record.

Read it → 6 sources agentsegress policyauditretrieval
Evidence & Evaluation 6 October 2026 7 min read

Nothing to check still passes

Three of the security notes in pnpm 12.10.0 describe the same defect in different vocabularies: a verifier whose passing state included the case where there was nothing to verify. The format those verifiers read did not change, and neither did the tools that audit builds for everyone else.

pnpm has spent two releases teaching its installer to refuse what the lockfile does not say, because an absent value there has always satisfied the check reading it — and the scanners the industry audits builds with still read that silence as a clean result.

Read it → 8 sources lockfilessupply chainsbomverification
Evidence & Evaluation 5 October 2026 7 min read

Every key you ever served

vLLM now watermarks generated text in the sampler itself, which is the right place for it. But detection needs the exact profile that produced the text, the key is the one field the config refuses to print, and the documented remedy spends your statistical confidence on your own deployment history.

Watermarking makes generated text testable only against the exact serving profile that produced it, so provenance is now a records problem rather than a property of the text, and the remedy the documentation recommends makes the evidence weaker the longer you have been operating.

Read it → 8 sources watermarkingprovenanceinference servingdetection
Evidence & Evaluation 3 October 2026 7 min read

Unstable was the safe part

OpenTelemetry moved the GenAI conventions into a repository of their own and told instrumentation authors to get the schema URL from there. That repository has no releases, no tags, and eight unreleased breaking changes — so the record of what agents did is being written in a vocabulary with no documented way back.

OpenTelemetry built the telemetry schema so unstable attribute names could change without orphaning the data already collected, and agent telemetry is now the one body of data being emitted with no version to migrate it from.

Read it → 10 sources opentelemetryagent telemetryschema versioningobservability
Evidence & Evaluation 28 September 2026 7 min read

A rate cannot say stop

The Green Software Foundation's carbon standard for AI is being edited into ISO shape this month. Its rules for counting emissions are stricter than almost anything in corporate reporting. Its rules for what to divide them by quietly guarantee that the number can never ask anyone to build less.

SCI for AI divides a provider's emissions by what it consumed rather than by what it delivered, so the carbon of a single answer is the one figure the standard cannot produce.

Read it → 6 sources carbon accountingmeasurementstandardssustainability
Evidence & Evaluation 25 September 2026 7 min read

Correctness, within tolerance

Speculative decoding is the one inference optimisation adopted because a proof says it changes nothing. In the test file vLLM names for its correctness, the property that proof describes is not asserted at all — what is checked is a benchmark score within three per cent and an acceptance rate above a floor.

Speculative decoding's lossless guarantee is what keeps a second model out of anyone's change control, and the only instrument production exports for it is acceptance, a throughput number with a floor, so its failures arrive as a disappointing speedup rather than a stop.

Read it → 9 sources speculative decodinginference servingverificationevaluation
Evidence & Evaluation 24 September 2026 7 min read

Behind is not red

Anthropic's continuous integration ran twenty-five times as many jobs in six months, so it built a service that decides which tests not to run. When that service fell twenty minutes behind, nothing turned red. Test selection has given CI a failure mode whose only symptom is a faster, greener build.

Test selection makes a passing build a claim about how fresh a separate service's test history was when it chose, and staleness in that service has no red state: it presents as a green check on a change whose tests never ran.

Read it → 6 sources continuous integrationtest selectionagentic codingverification
Evidence & Evaluation 22 September 2026 7 min read

Retrieval is procurement

The agent discovery specification backed by Google, Microsoft, GitHub and Hugging Face makes ranked search a registry's only mandatory interface, then forbids the orchestrator from reading that ranking as a judgement about trust. The ordering is the one signal guaranteed to arrive.

Agentic Resource Discovery converts the choice of a runtime dependency into a ranked retrieval result computed from text the publisher wrote about itself, then forbids the orchestrator from acting on the only signal that reliably reaches it.

Read it → 6 sources agentsdiscoverystandardssupply chain
Evidence & Evaluation 16 September 2026 7 min read

Seven bits the record cannot carry

A new Linux Foundation specification promises portable, offline-verifiable evidence that an AI agent ran inside trusted hardware. Its own schema closes the door on the configuration bits that decide whether that hardware's guarantee actually held — and the word "attested" will travel to readers who never see the footnote.

Attestation's vocabulary certifies more than its checking does — the platform state that decides whether a TEE's guarantee held is unchecked by default and has no field in the record — so "attested" crosses organisational boundaries carrying a claim nobody verified.

Read it → 7 sources attestationconfidential computingai governanceevidence