Evidence ledger
26 sources
Checked 13 Sep 2026
Evidence ledger
One row per claim in The breach is a missing check. The outage is the checker.: who published it, what grade it carries, when it was written, when the link was last checked, and the quote or figure it rests on. Nothing in the guide is cited from memory, so anything not in this table is not in the guide.
Topic: how production systems answer "may this caller act on this object" on every request,
and how that answer fails. Research date: 2026-09-13. All sources were retrieved during the
research session. Retrieval note: this session ran inside a network-restricted environment;
github.com and raw.githubusercontent.com were fetched directly, and all other hosts were
read through the search tool's server-side retrieval, which returns page content and verbatim
passages. Quotes marked (retrieval summary) are the retrieval tool's close paraphrase rather
than a hand-copied sentence; every other quote was returned verbatim by retrieval. URLs are
reproduced exactly as returned by search or fetch; none are reconstructed from memory.
| # | Org | Title | Tier | Published | Checked | URL | Claim I take from it | Supporting quote or figure |
|---|---|---|---|---|---|---|---|---|
| 1 | Zanzibar: Google's Consistent, Global Authorization System (USENIX ATC '19) | paper | 2019-07 | 2026-09-13 | https://www.usenix.org/system/files/atc19-pang.pdf | Scale and latency of the reference system: trillions of ACLs, millions of checks/s, p95 < 10 ms, > 99.999% availability | "Zanzibar scales to trillions of access control lists and millions of authorization requests per second … has maintained 95th-percentile latency of less than 10 milliseconds and availability of greater than 99.999% over 3 years of production use" (abstract) | |
| 2 | Zanzibar paper, zookie protocol | paper | 2019-07 | 2026-09-13 | https://www.usenix.org/system/files/atc19-pang.pdf | Consistency is scoped: causal ordering between ACL and content updates, freedom elsewhere | "It ensures that Zanzibar respects causal ordering between ACL and content updates, but otherwise grants Zanzibar freedom to choose evaluation timestamps so as to meet its latency and availability goals" | |
| 3 | Zanzibar paper, Leopard index | paper | 2019-07 | 2026-09-13 | https://www.usenix.org/system/files/atc19-pang.pdf | Deep group nesting is made fast by a denormalized transitive-membership index, not by traversal | "Leopard produces an index that enumerates all the users which the group hierarchy implies are transitively members of each group" (retrieval summary) | |
| 4 | Zanzibar paper, content-change check | paper | 2019-07 | 2026-09-13 | https://www.usenix.org/system/files/atc19-pang.pdf | The client stores a consistency token next to each content version; the check is anchored to it | "A Zanzibar client requests an opaque consistency token called a zookie for each content version, via a content-change ACL check when the content modification is about to be saved" (retrieval summary) | |
| 5 | USENIX / Google | ATC '19 conference presentation of Zanzibar (Ruoming Pang et al.) | talk | 2019-07 | 2026-09-13 | https://www.usenix.org/conference/atc19/presentation/pang | The paper was presented with recorded talk; presentation page carries the abstract claims above | Abstract on the presentation page repeats the trillions-of-ACLs / millions-of-requests figures. Timestamped video claims could not be verified this session (video host unreachable from the research environment) |
| 6 | AWS | Cedar: A New Language for Expressive, Fast, Safe, and Analyzable Authorization | paper | 2024-04 | 2026-09-13 | https://arxiv.org/abs/2403.04651 | The policy-language camp optimises for analysability: Cedar is modelled in Lean and benchmarked against OpenFGA and Rego | "they found that Cedar has equally or more readable policies, but performs far better" (retrieval summary of the paper's comparison claim) |
| 7 | Figma | How we built a custom permissions DSL at Figma (Jorge Silva) | blog | 2024-03-13 | 2026-09-13 | https://www.figma.com/blog/how-we-rolled-out-our-own-permissions-dsl-at-figma/ | A product with one shared object graph rejected OPA, Zanzibar-style systems and Oso, and built an in-app DSL instead | "Figma considered Open Policy Agent, Zanzibar, and Oso … none of them got to the core of what they were trying to solve" (retrieval summary) |
| 8 | Figma | Same post, motivation | blog | 2024-03-13 | 2026-09-13 | https://www.figma.com/blog/how-we-rolled-out-our-own-permissions-dsl-at-figma/ | The trigger was correctness drift, not scale | "In early 2021, the engineering team encountered a number of bugs, support tickets, and delayed projects rooted in the complexity of their permissions rules" (retrieval summary) |
| 9 | Figma | Same post, evaluation mechanics | blog | 2024-03-13 | 2026-09-13 | https://www.figma.com/blog/how-we-rolled-out-our-own-permissions-dsl-at-figma/ | Policies declare what they need; the engine owns data loading, replicas and caching | "contains a context_path object with all the necessary IDs needed to load all data required to run any given permission check … The backend engine has full control to decide how to load data: in which order, using which queries, using replicas or primary databases, and using caching" (retrieval summary) |
| 10 | Airbnb | Himeji: a scalable centralized system for authorization at Airbnb (Alan Yao) | blog | 2021-05 | 2026-09-13 | https://medium.com/airbnb-engineering/himeji-a-scalable-centralized-system-for-authorization-at-airbnb-341664924574 | Measured scale of a Zanzibar-derived service one year in | "throughput has scaled up from 0 in March 2020 to 850k entities/sec in March 2021, while maintaining 99.999% availability and 12-millisecond latency at the 99 percentile" |
| 11 | Airbnb | Same post, write-time fan-out | blog | 2021-05 | 2026-09-13 | https://medium.com/airbnb-engineering/himeji-a-scalable-centralized-system-for-authorization-at-airbnb-341664924574 | Read-heavy workloads move the fan-out to the write path | "Instead of fanning out at read time, they write all permissions data when resources are mutated, fanning out on the writes instead of the reads" (retrieval summary) |
| 12 | Airbnb | Same post, cache design | blog | 2021-05 | 2026-09-13 | https://medium.com/airbnb-engineering/himeji-a-scalable-centralized-system-for-authorization-at-airbnb-341664924574 | The check service is, operationally, a cache: sharded, replicated per AZ, ~98% hit target | "The caching layer, which is sharded and replicated (one instance per AZ per shard) … They target a ~98% hit rate on the cache" (retrieval summary) |
| 13 | Carta | AuthZ: Carta's highly scalable permissions system | blog | 2021-06 | 2026-09-13 | https://medium.com/building-carta/authz-cartas-highly-scalable-permissions-system-782a7f2c840f | The starting condition is usually several conflicting permission systems, not one bad one | "instead of maintaining one legacy system, they were maintaining five, and the permissions could conflict and were impossible to extend" (retrieval summary) |
| 14 | Carta | User authorization in less than 10 milliseconds (Aaron Tainter) | blog | 2021-09-03 | 2026-09-13 | https://medium.com/building-carta/user-authorization-in-less-than-10-milliseconds-f20d277fec47 | A Zanzibar-style graph over RelationTuples answers checks in under 10 ms at Carta | Title claim; "AuthZ uses RelationTuples to create a permissions graph … consumers combine RelationTuples using a domain-specific language and query the graph to check system access" (retrieval summary) |
| 15 | Slack | Role Management at Slack | blog | 2021-05 | 2026-09-13 | https://slack.engineering/role-management-at-slack/ | The RBAC vocabulary in a large product: permission = action, role = set of permissions, delegated per org or workspace | "a permission is defined as the ability to perform some action in Slack (such as inviting a user to a workspace or archiving a channel), while a role is defined as a set of permissions" (retrieval summary) |
| 16 | Gojek | We built a Zanzibar-style IAM system from scratch (Armaan Jain) | blog | 2026-06-04 | 2026-09-13 | https://medium.com/gojekengineering/we-built-a-zanzibar-style-iam-system-from-scratch-heres-everything-we-learned-bceb2686cad0 | The in-app alternative decays into unmaintainable conditionals; Gojek rebuilt on SpiceDB + Go + PostgreSQL | "a check_permission function becomes 400 lines of nested if-statements held together by comments like // TODO: fix this properly" (retrieval summary) |
| 17 | Netflix | Authorization at Netflix Scale (Travis Nelson, QCon Plus 2021) | talk | 2021-11 | 2026-09-13 | https://www.infoq.com/presentations/authorization-scalability/ | A central authorization service can sit in the critical path of a multi-million-RPS product if the client embeds caching and failure isolation | "how to centralize authorization in the critical path of a multi-million RPS online service … Services interact with PACS using a gRPC model with a lightweight client … Custom filters were implemented in the gRPC client to handle more advanced caching" (retrieval summary; talk timestamps could not be verified this session) |
| 18 | Oso | Why Authorization is Hard (Sam Scott) | blog | 2021-09-15 | 2026-09-13 | https://www.osohq.com/post/why-authorization-is-hard | The hard parts are enforcement placement, list filtering, and decisions needing data from more than one service | "expanding the enforcement API to not just answer if the user can perform an action, but what actions the user can perform, and how to handle authorization decisions that need data from more than one service" (retrieval summary) |
| 19 | AuthZed | Enforcing Causal Ordering in Distributed Systems (new-enemies post) | blog | 2023 (undated in retrieval) | 2026-09-13 | https://authzed.com/blog/new-enemies | The new-enemy problem is permission changes and content changes applied out of order; avoiding it costs consistency machinery | "The New Enemy Problem … occurs when authorization changes and content updates happen out of order, potentially granting access to users who should no longer have it" (retrieval summary) |
| 20 | AuthZed | SpiceDB repository README | source | checked at head | 2026-09-13 | https://github.com/authzed/spicedb | Vendor claim of check performance; consistency configurable per request | "5ms p95 when scaled to millions of queries/s, billions of relationships" (vendor claim); "consistency configured per-request unlocks correctness while maintaining performance" |
| 21 | AuthZed | Zed File #Z-4902: Spanner Spikes | casestudy | undated (retrieved 2026-09) | 2026-09-13 | https://authzed.com/blog/zed-file-z-4902-spanner-spikes | Naive per-object check fan-out overwhelms the checker: p95 rose to ~1 s; fix was bulk checks | "check latencies up to a full second at the 95th percentile … a single user made over 600 requests within 250 milliseconds for the same permission check … move from fanout CheckPermission calls to BulkCheck" (retrieval summary; fixes landed in SpiceDB v1.28) |
| 22 | Ory | keto issue #517: Provide Consistency Guarantees using Snapshot Tokens (zepatrik) | source | 2021-04-01, open as of 2026-09-13 | 2026-09-13 | https://github.com/ory/keto/issues/517 | The paper's consistency machinery is the part reimplementations postpone: zookies have been an open feature request for over five years | "The tokens are opaque and could encode arbitrary data that can be used by Keto to guarantee that it considers all relation tuples at least as old as the object version" — issue open, no linked implementation |
| 23 | Cedar (AWS) | rfcs PR #14: create a batch version of is_authorized (glb) | source | opened 2023-07-11, closed unmerged 2023-08-22 | 2026-09-13 | https://github.com/cedar-policy/rfcs/pull/14 | Batch check APIs get rejected because fixing which fields vary across the batch is inflexible; the cost lives in data loading, not call overhead | Labelled rejected; maintainer review preferred "exposing opaque references to pre-constructed PolicySet and Entities objects across FFI boundaries" and per-construction attribute evaluation (cedar#227) (retrieval summary) |
| 24 | Envoy (CNCF) | ext_authz filter proto, failure_mode_allow | source | current head | 2026-09-13 | https://github.com/envoyproxy/envoy/blob/main/api/envoy/extensions/filters/http/ext_authz/v3/ext_authz.proto | The industry's most deployed enforcement point fails closed by default | "When set to false, the filter will reject client requests and return Forbidden if communication with the authorization service has failed, or if the authorization service has returned an HTTP 5xx error … Defaults to false." Default error status: "HTTP 403 Forbidden" |
| 25 | Kubernetes | KEP-3221: Structured Authorization Configuration | adr | approved 2023-06-15, alpha in v1.29 (2023-12) | 2026-09-13 | https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/3221-structured-authorization-configuration/README.md | The fail-open/fail-closed choice is per-authorizer configuration: webhook failurePolicy is Deny or NoOpinion | failurePolicy Deny: "reject the request without consulting subsequent authorizers"; NoOpinion: "continue to subsequent authorizers to see if one of them allows the request" |
| 26 | OpenFGA (CNCF) | RFC 20220714: ListObjects API (jon-whit) | adr | 2022-07-14 | 2026-09-13 | https://github.com/openfga/rfcs/blob/main/20220714-listObjects-api.md | List filtering is priced separately from point checks: bounded results, deadline, no pagination, because it is graph traversal | "the query cost of supporting a generic API endpoint like this could be quite expensive"; "Without proper configuration of, and use of the ListObjects endpoint, the server's memory footprint could be profoundly impacted"; limits ListObjectsMaxResults and ListObjectsDeadline |
| 27 | Google Cloud | Incident report 2025-06-12: Multiple GCP products experiencing service issues | postmortem | 2025-06 | 2026-09-13 | https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW | The authorization/quota gatekeeper (Service Control) crashed in every region at once on a bad policy row; outage ran 10:51-18:18 US/Pacific | Policy data with "unintended blank fields" replicated globally hit a new quota-check code path; the binary crash-looped in every region (details corroborated by [28]) |
| 28 | The Register | Google Cloud caused outage by ignoring its usual code quality protections | postmortem (secondary) | 2025-06-16 | 2026-09-13 | https://www.theregister.com/2025/06/16/google_cloud_outage_incident_report/ | The check's code path was unguarded: the report says it lacked error handling and a feature flag; remediation includes failing open | Quotes the incident report: the change "did not have appropriate error handling nor was it feature flag protected"; remediation includes modularizing Service Control so a failing check fails open (retrieval summary) |
| 29 | ACMA (regulator) | ACMA statement on 2022 Optus data breach + Federal Court filing coverage | postmortem | 2024-05-22 (statement); filing 2024-05-20 | 2026-09-13 | https://www.acma.gov.au/acma-statement-2022-optus-data-breach | The breach was an access-control coding error dormant for four years on a forgotten domain; 9.5M people's records | "In September 2018, a coding error was introduced to one of the access controls … left these access controls ineffective for both domains"; error "first noticed in August 2021" and fixed on the main domain only; attack ran 17-20 September 2022 (retrieval summary of ACMA filing, corroborated by [30]) |
| 30 | iTnews | Optus breach allegedly enabled by access control coding error | postmortem (secondary) | 2024-06 | 2026-09-13 | https://www.itnews.com.au/news/optus-breach-allegedly-enabled-by-access-control-coding-error-608985 | Independent account of the ACMA claim: internet-facing dormant API with bypassed access controls | "ACMA claimed that Optus had access controls in place for the API but a coding error inadvertently weakened these controls allowing them to be bypassed" (retrieval summary) |
| 31 | NYDFS (regulator) | Consent order, First American Title Insurance | postmortem | 2023-11-27 | 2026-09-13 | https://www.dfs.ny.gov/industry_guidance/enforcement_discipline/ea20231127_first_american | One missing object-level check exposed 885M documents 2014-2019; the company's own pen test flagged it months before disclosure | "First American failed to maintain and implement effective governance and classification, access controls and identity management, and risk assessment policies"; vulnerability found in a December 2018 penetration test with the instruction to "address as soon as possible"; $1M penalty (retrieval summary) |
| 32 | Krebs on Security | First American Financial pays farcical $500K fine | blog | 2021-06 | 2026-09-13 | https://krebsonsecurity.com/2021/06/first-american-financial-pays-farcical-500k-fine/ | The SEC penalty for the same exposure was roughly $500K, from the reporter who disclosed it | Headline figure: "$500K fine" (SEC settlement, 2021) |
| 33 | Meta | Security Update (2018-09-28) | postmortem | 2018-09-28 | 2026-09-13 | https://about.fb.com/news/2018/09/security-update/ | Three individually shipped features composed into a token-minting hole in View As; ~50M tokens reset plus 40M precautionary | "This vulnerability consisted of three separate bugs and allowed them to steal Facebook access tokens"; bug 3: the video uploader "generated the access token not for you as the viewer, but for the user that you were looking up" (retrieval summary) |
| 34 | Meta | An Update on the Security Issue (2018-10-12) | postmortem | 2018-10-12 | 2026-09-13 | https://about.fb.com/news/2018/10/update-on-security-issue/ | Final measured blast radius: 30M tokens actually stolen; window July 2017 to September 2018; spike detected September 14 | "about 30 million actually had their tokens stolen"; attack spike began 2018-09-14, identified as attack 2018-09-25 (retrieval summary) |
| 35 | CISA / ACSC / NSA | AA23-208A: Preventing Web Application Access Control Abuse | vendor | 2023-07-27 | 2026-09-13 | https://www.cisa.gov/news-events/cybersecurity-advisories/aa23-208a | Three national agencies formalised the design rule: check every request that touches sensitive data | "ensure software performs authentication and authorization checks for every request that modifies, deletes, and accesses sensitive data" |
| 36 | OWASP | API1:2023 Broken Object Level Authorization | vendor | 2023 | 2026-09-13 | https://owasp.org/API-Security/editions/2023/en/0xa1-broken-object-level-authorization/ | The most common API failure is the absent per-object check, top of the OWASP API list two editions running | "Every API endpoint that receives an ID of an object, and performs any action on the object, should implement object-level authorization checks" |
Sources consulted but not carried into the page
- InfoQ news summary of Himeji (2021-05) — duplicate of the primary Airbnb post.
- Security Boulevard, dev.to, Medium reading-notes on the Zanzibar paper — restatements of [1].
- ThousandEyes outage analysis (2025-06) — corroborates [27] timeline; not independently load-bearing.
- Permify marketing figures (4.3B checks/day) — vendor claim on a PR site, could not be traced to an engineering account; excluded.
- Hacker News thread on the Figma DSL post (2024-03) — host unreachable from this session; comments not citable.