Customer 360 Enterprise Data Platform — Denodo on Azure

Architecture Views

39 views, in reading order. Every view ships three ways: an HTML page, an SVG that re-opens in diagrams.net fully editable, and draw.io source.

Denodo is the enterprise logical data layer: one governed, semantic, secured view of a customer assembled at query time from Salesforce, SAP, Oracle, ServiceNow, Marketing Cloud, an Azure lakehouse and a digital event stream. Read the set in order — the boundary and the governing principle first, then who it is for and what they get to do, and only then the structure that has to justify itself against them. One decision carries the whole design: Denodo owns access, never truth. Identity is mastered outside it, history lives in the lakehouse, streaming stays in Event Hubs, and every source system keeps writing its own records. That boundary is why the identity crosswalk is the one join the platform is forbidden to cache, why the freshness contract in view 17 is architecture rather than tuning, and why the AI assistant in view 24 needed no security design of its own.

Context and scope

What sits inside the boundary, what the platform only reads, and the principle that decides everything after it.
01 Systems of record Salesforce CRM profile, segment SAP and Oracle ERP orders, billing ServiceNow CSM cases, SLA Enterprise services Microsoft Entra ID Microsoft Purview People it serves Service Agent 1,800 seats Relationship Manager 420 seats BI Developer 90 authors Data Steward Customer Data Team Analytical and SaaS sources Databricks Lakehouse CLV, ML scores Marketing Cloud campaigns Digital Event Stream web and mobile Customer 360 Platform Denodo logical data layer Consuming applications and metadata Power BI 1,100 report users Agent Desktop contact centre Customer AI Assistant Azure OpenAI Data Science Notebooks masked cohorts 360 view account book certified data governs REST JDBC REST daily hourly events SQL REST API tools JDBC auth lineage Customer 360 Platform — System Context External / third party Security / platform Person or role synchronous batch event / async Out of scope: campaign execution, CRM customisation and remediation inside the source systems. The platform reads; the owning system still writes. v 1.0 · owner Data & AI Global Practice · date 2026-09 System Context Who uses the Customer 360, what it reads, and what it deliberately does not touch. HTML page SVG draw.io
02 Sources Salesforce CRM SAP S/4HANA ServiceNow CSM Digital Events Move only what must move Azure Event Hubs Kafka protocol CDC and Batch Load Debezium, ADF Delta Lakehouse ADLS Gen2 Resolve identity Entity Resolution Zingg on Spark ECID Crosswalk PostgreSQL Denodo logical layer Virtual DataPort federated engine Semantic Model Customer360 Cache and Summaries PostgreSQL, MPP Delivery JDBC and ODBC REST and GraphQL API Management Azure APIM Consumers Power BI Agent Desktop AI Assistant Notebooks clickstream nightly join key High-Level Architecture — Federate by Default, Materialise by Exception External / third party Queue / topic Interface / broker Data store Application we own event / async batch synchronous Only digital events, deep history and model output are physically moved. Profile, order and case attributes are read where they live. v 1.0 · owner Data & AI Global Practice · date 2026-09 High-Level Architecture The shape of the platform in one picture: federate by default, materialise by exception. HTML page SVG draw.io
03 Denodo Platform — owns the access contract Semantic Model one Customer360 Federated Execution pushdown, MPP Policy Enforcement row, column, mask Data Services SQL, REST, GraphQL Catalog and Lineage certified assets Specialist platforms — own the truth Identity Entity Resolution Zingg on Spark ECID Crosswalk Survivorship Rules versioned in Git History and models Delta Lakehouse CLV and Segments daily model run Pre-aggregations Movement Event Hubs Stream Processing Spark Streaming Systems of record Salesforce SAP S/4HANA ServiceNow joins ECID reads gold reads live applies The Governing Principle — Denodo Owns Access, Not Truth Application we own Data store Queue / topic External / third party synchronous batch No box in the upper container stores an authoritative customer fact. Removing Denodo would slow every consumer down; it would not lose a record. v 1.0 · owner Data & AI Global Practice · date 2026-09 The Governing Principle Which responsibilities Denodo takes, and which it must be prevented from taking. HTML page SVG draw.io

People and journeys

Who the 360 is for, and the three journeys whose worst moments the rest of the set has to answer.
04 People who take value from it Customer Service Agent 1,800 seats Goal — Know who I am talking to before they finish saying their name, and tell whether their open case is really still open. Core journeys Answer an inbound call 9,400 calls a day Check an order or invoice Raise a complaint Relationship Manager 420 seats Goal — Walk into a meeting knowing what this account bought, what broke, and what we last promised them. Core journeys Prepare for an account meeting Review my book weekly BI Developer 90 authors Goal — Build a customer report on one certified dataset instead of joining five systems whose table names I do not know. Core journeys Find a certified dataset Build a Power BI report Data Scientist 35 users Goal — Pull a governed cohort into a notebook with the personal fields already masked, without raising a ticket first. Core journeys Pull a masked cohort Publish a score back People who run and govern it Data Steward Customer Data Team Goal — Prove where every attribute on the 360 came from, and merge two records without anyone losing history. Core journeys Resolve a duplicate customer Certify a dataset Answer a lineage question Platform SRE follow the sun Goal — Find out that a source has gone slow before the contact centre tells me. Core journeys Triage a slow query Handle a source outage Privacy Officer DPO office Goal — Answer a subject access request inside the statutory window without a manual trawl through six systems. Core journeys Fulfil a DSAR Approve a masking exception Systems that consume it Agent Desktop App 60 rps at peak Goal — Get the whole customer panel in one call, in under two seconds, or a clear reason why not. Core journeys Load the customer panel Customer AI Assistant Azure OpenAI Goal — Answer in the caller's words using only what the agent asking is allowed to see. Core journeys Answer a grounded question Partner Portal 4 partners Goal — Read the subset of the 360 our contract allows, and nothing beyond it. Core journeys Fetch an entitled subset Machines that act unasked ECID Resolution Job nightly, 02:00 UTC Goal — Reconcile every new source identifier to one enterprise customer before the business day starts. Core journeys Resolve new identifiers Publish crosswalk deltas Cache Refresh Scheduler Denodo Scheduler Goal — Keep each cached view inside its declared freshness window without stampeding the source. Core journeys Refresh on schedule Invalidate on ECID change Model Scoring Job daily, Databricks Goal — Land lifetime value, churn and segment scores where the semantic layer can join them by morning. Core journeys Score and publish Who the Customer 360 Is For, and What Each of Them Gets to Do Person or role Journey / task External / third party Security / platform v 1.0 · owner Data & AI Global Practice · date 2026-09 Actors and Their Core Journeys Who the platform is for, in their own words, and what each of them gets to do with it. HTML page SVG draw.io
05 Customer Service Agent contact centre, 1,800 seats Goal — Answer the caller without asking them to repeat what we already know Trigger — An inbound call arrives, with a caller-line match or none at all Done when — Resolved in one call, against an accurate order and case picture 1 · Take the call 0-5 s 2 · See who it is 5-8 s ◆ moment of truth 3 · Look it up 8-40 s ◆ moment of truth 4 · Act 5 · Wrap up What they do Answers and greets Confirms identity Reads orders and cases Raises or updates case Logs the outcome Touchpoint Softphone Agent Desktop Customer 360 panel ServiceNow form Wrap-up notes How it feels In control Coping Exposed Behind the glass CTI screen pop ECID lookup Customer360 REST ServiceNow write Interaction log Where it hurts No caller-line match Support panel blank Down, or truly empty? What would fix it Search on three fields Unavailable marker Last known, timestamped How we know Match rate 94% P95 1.6 s Handle time 4m 10s Journey — A Service Agent Answers an Inbound Call The trough is not slowness. It is an empty support panel that cannot be told apart from a source outage — which is why partial results carry an explicit per-domain status. v 1.0 · owner Data & AI Global Practice · date 2026-09 Journey — Answering an Inbound Call What happens to an agent in the first forty seconds, and the moment where the platform lets them down. HTML page SVG draw.io
06 BI Developer reporting team, 90 authors Goal — Ship a churn-by-region report this sprint without learning SAP table names Trigger — A business request that needs profile, orders and lifetime value together Done when — A published report reading one certified dataset, refreshing inside its SLA 1 · Find Data Catalog 2 · Understand 3 · Try 4 · Ask ◆ moment of truth 5 · Publish What they do Searches for customer Reads the glossary Runs a sample query Requests access Builds the report Touchpoint Denodo Data Catalog Asset page Design Studio Access request form Power BI Desktop How it feels Productive Coping Blocked Behind the glass Catalog search index Glossary and lineage Sample execution Approval workflow DirectQuery to VDP Where it hurts Lifetime value masked Approval takes six days What would fix it Mask reason on the field Named owner on asset One working day SLA Journey — A BI Developer Builds a Customer Report The dataset was certified and still blocked the work. Discovery without a named owner and a bounded approval turns a catalogue into a directory of things you cannot have. v 1.0 · owner Data & AI Global Practice · date 2026-09 Journey — Building a Customer Report How a report author finds, understands and is blocked by a dataset that was certified all along. HTML page SVG draw.io
07 Data Steward Customer Data Team, 6 people Goal — Merge two records for the same person without anyone losing history Trigger — A duplicate-suspect pair scores above the review threshold overnight Done when — One enterprise id, both source ids retained, every consumer consistent within the hour 1 · Notice queue 2 · Investigate 3 · Decide ◆ moment of truth 4 · Merge ◆ moment of truth 5 · Verify What they do Opens the queue Compares both records Confirms same person Approves the merge Rechecks the 360 Touchpoint Quality dashboard Side-by-side compare Steward console Crosswalk API Customer 360 panel How it feels Confident Cautious Alarmed Behind the glass Zingg candidate pairs Source attribute trace Survivorship rules Crosswalk write Cache invalidation Where it hurts Old id still cached Split history for a day What would fix it Crosswalk never cached Merge event invalidates Journey — A Data Steward Resolves a Duplicate Customer A merge is correct the instant it is approved and wrong for every consumer until the cache agrees. That gap is what makes the crosswalk the one join Denodo is forbidden to cache. v 1.0 · owner Data & AI Global Practice · date 2026-09 Journey — Resolving a Duplicate Customer The one journey where the failure is a wrong answer rather than a slow one. HTML page SVG draw.io

Structure

The layering rule, the deployable units, every interface, and how five source identifiers become one customer.
08 Consumption Power BI Agent Desktop AI Assistant Notebooks Partner Portal Delivery JDBC and ODBC REST and OData GraphQL API Management quota, keys Business views Customer360 Customer Profile Customer Orders Customer Support Customer Value Integration views iv_customer ECID join iv_customer_order iv_customer_case iv_customer_activity iv_dq_exception quality flags Base views bv_sf_account bv_sap_kna1 bv_snow_case bv_lake_clv bv_ecid_xref Physical sources Salesforce SAP S/4HANA ServiceNow Delta Lakehouse ECID Crosswalk HTTPS publishes composes normalises delegates Layered Architecture — Dependencies Point Downward Only External / third party Interface / broker Application we own Data store synchronous Security, catalogue and lineage are not a layer — they are enforced at the integration layer and inherited upward by every view and every delivery channel above it. v 1.0 · owner Data & AI Global Practice · date 2026-09 Layered Architecture What depends on what, and why no consumer can reach a source structure directly. HTML page SVG draw.io
09 Azure — Customer 360 landing zone Edge and delivery Front Door and WAF TLS 1.3 Denodo Load Balancer JDBC and ODBC API Management quota, keys, JWT Denodo query tier — AKS node pools VDP Interactive 3 pods, BI and desktop VDP Data Services 2 pods, REST VDP Analytical 2 pods, science Embedded MPP Presto, 6 workers Denodo control tier Solution Manager licence, promotion Design Studio modelling Data Catalog search, certification Scheduler cache refresh State Cache Database PostgreSQL Flexible Metadata Database VDP catalogue ECID Crosswalk PostgreSQL Query Log Log Analytics Azure data platform Databricks SQL serverless warehouse Delta Lakehouse ADLS Gen2 Event Hubs Kafka protocol Unity Catalog lake grants Platform services Microsoft Entra ID Key Vault source credentials Azure Monitor metrics and logs Microsoft Purview enterprise catalogue Salesforce SAP S/4HANA Oracle Billing ServiceNow Marketing Cloud REST JDBC cache read offload Parquet scan OAuth REST refresh Container Architecture — The Deployable Units and What They Talk To Interface / broker Application we own Data store Queue / topic Security / platform External / third party synchronous batch Three VDP pools, not one cluster: a data scientist scanning three years of orders cannot take latency from the contact centre. Source credentials are drawn on view 35, not here. v 1.0 · owner Data & AI Global Practice · date 2026-09 Container Architecture The deployable units, the technology in each, and what talks to what. HTML page SVG draw.io
10 Physical sources Salesforce SAP S/4HANA ServiceNow Delta Lakehouse Layer 1 — base views bv_sf_account no logic bv_sap_kna1 no logic bv_snow_case no logic bv_lake_clv no logic Layer 2 — integration views iv_customer ECID, survivorship iv_customer_order iv_customer_case iv_customer_value Layer 3 — business views Customer Profile Customer Orders Customer Support Customer Value Layer 4 — the 360 Customer360 certified Published SQL endpoint REST resource GraphQL schema renames publishes The Denodo View Hierarchy — Four Layers, One Naming Standard External / third party Data store Application we own Interface / broker synchronous Base views carry no transformation and no business name. Security is declared once at layer 2, so no consumer can reach a source structure that has not passed through it. v 1.0 · owner Data & AI Global Practice · date 2026-09 The Denodo View Hierarchy The four modelling layers, the naming standard, and where each kind of logic is allowed to live. HTML page SVG draw.io
11 Consumers that call us Power BI 1,100 users Agent Desktop 60 rps peak AI Assistant tool calls Notebooks 35 users Partner Portal 4 partners Microsoft Purview nightly scan Denodo Virtual DataPort Customer 360 Logical Layer one access contract Sources we call Salesforce CRM on demand SAP S/4HANA on demand Oracle Billing cached 30 min ServiceNow CSM cached 15 min Marketing Cloud cached 1 hour Databricks SQL gold tables only Delta and ADLS MPP scan ECID Crosswalk never cached ODBC REST MCP JDBC APIM meta REST JDBC JDBC REST REST JDBC Parquet JDBC Integration Surface — Every Interface In and Out External / third party Security / platform Application we own Data store synchronous batch Fourteen interfaces, one contract. No consumer holds a credential for a source, and no source is reachable except through a layer 2 view. v 1.0 · owner Data & AI Global Practice · date 2026-09 Integration Surface Every way the platform touches another system, with protocol, cadence and who owns it. HTML page SVG draw.io
12 Connector Authentication What is delegated Access strategy SaaS CRM Salesforce adapter OAuth 2.0 JWT bearer SOQL filters, limits Federated, no cache ERP SAP HANA JDBC Vault-held tech user Joins, aggregates Federated plus partial Billing Oracle JDBC Vault-held tech user Joins, aggregates Partial cache, 30 min ITSM ServiceNow REST OAuth 2.0 Filters only Cached 15 min Marketing Marketing Cloud REST OAuth 2.0 Filters only Cached 1 hour Lakehouse Databricks JDBC Managed identity Full SQL pushdown Federated on gold Lake files Parquet base view Managed identity Partition pruning MPP acceleration Streaming Delta streaming table Managed identity Partition pruning Federated, 60 s lag Identity crosswalk PostgreSQL JDBC Managed identity Full SQL pushdown Never cached Source Connectivity — Connector, Identity, Delegation and Strategy The two SaaS rows delegate least, which is why they are the two that get cached. Delegation capability, not preference, decides the strategy column. v 1.0 · owner Data & AI Global Practice · date 2026-09 Source Connectivity For each class of source: the connector, how it authenticates, what it can be asked to do, and what follows from that. HTML page SVG draw.io
13 Collect identifiers CRM ids CUST-1001 ERP ids 000045678 Support and marketing USER-9876 Device and web ids DEVICE-2345 Standardise Name and address parse libpostal Email and phone E.164, RFC 5322 Country reference Block and score Blocking keys cuts the pair space Deterministic rules tax id, email Zingg similarity Spark ML Decide Auto-merge score above 0.95 Steward review 0.80 to 0.95 Keep separate below 0.80 Master and publish Survivorship rules source precedence ECID assignment stable, never reused ECID Crosswalk source id retained Serve and notify Denodo join key iv_customer Merge event invalidates cache on approval on change Identity Resolution — How Five Source Ids Become One Enterprise Customer External / third party Application we own Data store Decision point Queue / topic synchronous event / async Denodo consumes the crosswalk; it does not produce it. Matching rules, thresholds and survivorship live in Git and run on Spark, where they can be tested against a labelled set. v 1.0 · owner Data & AI Global Practice · date 2026-09 Identity Resolution How five source identifiers become one enterprise customer, and why none of it happens in a view. HTML page SVG draw.io

Data

What is copied and what is only read, who owns each byte, the model consumers see, and the freshness contract that decides between them.
14 Sources Salesforce CRM SAP and Oracle ServiceNow CSM Web and mobile Capture Event Hubs Kafka protocol Debezium CDC 5 min Data Factory nightly extract Land and curate Bronze raw, 30 days Silver conformed, 13 months Gold analytical, 7 years Enrich Entity Resolution Zingg, nightly Quality Rules Soda Core Model Scoring CLV, churn Serve logically Denodo base views Denodo cache TTL per view Embedded MPP Presto over Delta Consume Power BI Agent Desktop AI Assistant Notebooks 5 min 03:00 daily no copy cached Data Flow — What Is Copied, What Is Only Read, and How Often External / third party Queue / topic Interface / broker Data store Application we own event / async batch synchronous The two gutter arrows are the point of the platform: profile and case data reach the consumer without ever landing in the lake. v 1.0 · owner Data & AI Global Practice · date 2026-09 Data Flow What is copied, what is only read, and the cadence on every arrow. HTML page SVG draw.io
15 Systems of record — the platform never writes here Salesforce CRM profile, segment SAP S/4HANA orders, invoices Oracle Billing payments ServiceNow CSM cases Marketing Cloud campaigns Azure lakehouse — rebuildable from source in 14 hours Bronze Event landing 30 days CDC change log 30 days Silver customer_activity 13 months order_history 7 years Gold customer_value daily snapshot segment_and_churn daily snapshot Platform-owned state — the only data this platform masters ECID Crosswalk backed up, PITR 7 d Survivorship Rules Git, versioned DQ Exceptions 90 days Denodo Metadata views, policies Disposable acceleration — losing all of it costs latency, not data Cache tables TTL 15 min to 24 h Summaries rebuilt nightly MPP result cache in memory CDC nightly invalidates Storage Zones — Who Owns Each Byte, and What Could Be Rebuilt External / third party Data store event / async batch Only the third box needs a backup strategy. Everything above it is authoritative elsewhere, and everything below it is rebuildable on demand. v 1.0 · owner Data & AI Global Practice · date 2026-09 Storage Zones Who owns each byte, what would have to be restored, and what could simply be rebuilt. HTML page SVG draw.io
16 source_customer_xref xref_id PK ecid FK -> enterprise_customer source_system source_customer_id match_score valid_from customer_contact contact_id PK ecid FK email PII phone PII postal_address PII is_primary customer_account account_id PK ecid FK product_code opened_on account_status customer_order order_id PK account_id FK order_date net_amount currency customer_activity activity_id PK ecid FK event_time event_type channel enterprise_customer ecid PK primary_name customer_status segment_code authoritative_source resolved_at customer_case case_id PK ecid FK opened_at sla_due_at case_state channel customer_invoice invoice_id PK order_id FK due_date paid_flag outstanding dq_exception exception_id PK ecid FK rule_code severity detected_at customer_value ecid PK, FK lifetime_value churn_score scored_on customer_campaign campaign_ref PK ecid FK sent_at responded_flag channel 1 : N 1 : N 1 : N 1 : N 1 : N 1 : N 1 : N 1 : 1 1 : N 1 : N Customer 360 — The Logical Data Model Consumers See The enterprise id is the only join key above layer 2. Source keys survive in the crosswalk so any figure can be traced back to the system that produced it. v 1.0 · owner Data & AI Global Practice · date 2026-09 The Customer 360 Logical Model The entities a consumer sees, the one key they all join on, and where the source keys survive. HTML page SVG draw.io
17 Freshness required Access strategy Where it materialises Refresh trigger Customer profile Near real time Federated Nowhere Not applicable Contact details Near real time Federated, masked Nowhere Not applicable Account status Near real time Federated Nowhere Not applicable Orders, last 90 days Near real time Federated Nowhere Not applicable Order history, 3 years Daily MPP over Delta Silver tables Nightly ELT Open support cases Under 15 minutes Partial cache Cache database Scheduler, 15 min Marketing engagement Under 1 hour Full cache Cache database Scheduler, hourly Digital activity Under 60 seconds Federated on Delta Silver streaming Continuous Lifetime value, scores Daily Federated on gold Gold tables Model run, 03:00 Aggregated KPIs Daily Denodo summary Summary table After model run Identity crosswalk Immediate Federated only Never Merge event Freshness Contract — What Is Federated, Cached or Materialised, and Why This table is the architecture, not a tuning guide. A view author cannot choose a strategy; they declare a freshness class and the platform assigns one. v 1.0 · owner Data & AI Global Practice · date 2026-09 The Freshness Contract Which attribute groups are federated, cached or materialised — and the rule that decides, rather than the author. HTML page SVG draw.io
18 Request Customer360 query SQL or REST Planner decides Cache still valid? Summary covers it? Source can delegate? Served from Cache tables PostgreSQL Summary tables pre-aggregated MPP over Delta Presto workers Source system delegated SQL Kept fresh by Denodo Scheduler per-view TTL Incremental refresh on change column Merge event listener ECID changes Bounded by Never-cache list crosswalk, consent Stampede control single refresher Stale-read budget declared per view hit delegate invalidate Caching and Acceleration — What Answers a Query, and What Keeps It Honest Application we own Decision point Data store External / third party Queue / topic Risk / gap synchronous failure / alternate A cache miss is a latency event; a stale cache after a merge is a correctness event. Only the second one is drawn in red. v 1.0 · owner Data & AI Global Practice · date 2026-09 Caching and Acceleration What can answer a query, how each store is kept fresh, and where the platform refuses to help. HTML page SVG draw.io
19 Detect Classify Expose Remediate Prove Completeness Missing email rule Warn dq_flag on the 360 CRM owner task Coverage 98.2% Validity Phone format rule Warn dq_flag on the 360 Source correction Valid 96.4% Uniqueness Duplicate candidates Block Held out of the 360 Steward merge Duplicates 0.4% Consistency Address conflict Warn Survivorship pick Precedence rule change Conflicts 1.1% Timeliness Stale cache check Block Row marked stale Forced refresh Freshness 99.5% Data Quality — Detected at the Source, Exposed on the 360, Fixed by the Owner The platform never silently repairs a value. It publishes the defect beside the record and routes the fix to the system that owns it. v 1.0 · owner Data & AI Global Practice · date 2026-09 Data Quality How a defect is detected, what it does to the record, and who is expected to fix it. HTML page SVG draw.io

Runtime

A query, an API call, a stream, an AI answer, a source outage, a subject access request and a merge — each followed end to end.
20 Power BI Denodo Load Balancer Virtual DataPort ECID Crosswalk Cache Database Salesforce SAP S/4HANA Embedded MPP 1. SQL on Customer360 2. route to interactive pool 3. validate token, resolve roles 4. expand views, prune branches 5. enterprise ids for this region 6. 4,812 ids 7. open cases, refreshed 4 min ago 8. case rows 9. SOQL, filtered and projected 10. delegated join and aggregate 11. profile rows 12. pre-aggregated totals 13. three-year history scan 14. aggregate over Delta 15. apply row filter and column masks 16. result set, 1.9 s Runtime — One Customer360 Query, End to End Five sources, two of them never touched: the branch pruner removed marketing and billing because no projected column came from them. v 1.0 · owner Data & AI Global Practice · date 2026-09 A Customer360 Query, End to End What actually happens between a report refreshing and rows arriving, across five systems. HTML page SVG draw.io
21 Parse Incoming statement SQL, REST or GraphQL Metadata lookup views, stats, policies Rewrite View expansion Predicate normalisation Constant folding Prune Branch pruning unreferenced sources Column trimming Partition pruning Delta files skipped Delegate Capability check what the source can do Aggregation pushdown Join delegation same-source joins Execute Cost-based join order statistics driven MPP offload when scan exceeds 10 M Parallel fetch per data source pool Return Row filter and masking Result to client two sources left cannot delegate Query Optimisation — How a Five-Source View Becomes Two Source Calls Application we own Data store Decision point Security / platform synchronous failure / alternate Masking is applied after execution, never pushed down. A masked column is never sent to a source system as a predicate, so masking cannot leak through a WHERE clause. v 1.0 · owner Data & AI Global Practice · date 2026-09 Query Optimisation The stages between a statement and a plan, and which of them do the real work. HTML page SVG draw.io
22 Agent Desktop Front Door and WAF API Management Microsoft Entra ID VDP Data Services ECID Crosswalk Source Systems Log Analytics 1. GET /v1/customers/{id}/summary 2. forwarded, WAF passed 3. validate bearer token 4. claims and group memberships 5. quota check, 600 per minute 6. REST call with delegated token 7. resolve caller id to enterprise id 8. one enterprise id 9. delegated reads, four domains 10. three succeed, one times out 11. mask by role, build status block 12. 200 with per-domain status 13. response in 1.4 s 14. who asked, what was returned Runtime — A Data Service Call from the Agent Desktop A partial answer is a 200 with an explicit per-domain status, not a 500 and not a silent omission. The client renders the gap; it never has to guess at it. v 1.0 · owner Data & AI Global Practice · date 2026-09 A Data Service Call One REST call from the agent desktop, including the part where a source does not answer. HTML page SVG draw.io
23 Emit Web SDK page and product views Mobile SDK logins, taps Transport Event Hubs 32 partitions Schema Registry Avro, versioned Process Structured Streaming Databricks job ECID enrichment device to customer Sessionisation 30 min window Land Silver activity Delta, 13 months Checkpoints exactly once Unmatched holding device not yet resolved Expose bv_activity_delta federated, no cache Customer Digital Activity business view Consume Agent Desktop last activity AI Assistant Power BI hourly refresh 60 s target no ECID yet Real-Time Digital Activity — Sixty Seconds from Tap to the 360 External / third party Queue / topic Data store Application we own event / async failure / alternate Denodo reads the Delta table directly rather than the stream. Nothing in the logical layer is stateful, so an outage here shows as staleness, not as a broken view. v 1.0 · owner Data & AI Global Practice · date 2026-09 Real-Time Digital Activity The one genuinely streaming path, and why Denodo does not read the stream. HTML page SVG draw.io
24 Service Agent Assistant UI Microsoft Entra ID Azure OpenAI Denodo MCP Server VDP Data Services Sources and Lakehouse Audit Log 1. Why is this invoice unpaid? 2. on-behalf-of token for this agent 3. delegated token, agent scopes 4. prompt and tool schema 5. call get_customer_summary 6. query as the agent, not as a service 7. row filter and masks applied 8. delegated reads 9. rows 10. masked rows with lineage refs 11. tool result and citations 12. call get_full_card_number 13. denied, not in the agent's grants 14. answer grounded on what was returned 15. answer with named sources 16. prompt, tools called, rows returned Runtime — The AI Assistant Inherits the Agent's Entitlements The model has no credential of its own. Step 13 is refused by the same policy that would refuse the agent in Power BI, which is why the assistant needed no security design of its own. v 1.0 · owner Data & AI Global Practice · date 2026-09 The AI Assistant How a grounded answer is produced, and why the model needed no security design of its own. HTML page SVG draw.io
25 Healthy Detect Degrade Inform Recover Source ServiceNow responding Latency past 3 s Errors above 25% Vendor incident open Probe succeeds Denodo Full 360 assembled Timeout at 3 s Circuit opens Partial result built Per-domain status set Half-open retry Consumer Every panel populated No visible change Support panel greyed Cases unavailable, 09:14 Panel repopulates Operator Source SLO green Alert on error budget Runbook opened Status page updated Post-incident review When a Source Goes Down — Partial, Labelled, and Never Silent This is the answer to the agent journey's worst moment. An empty panel and an unreachable source must never look the same, so the contract carries a status per domain rather than one status per response. v 1.0 · owner Data & AI Global Practice · date 2026-09 When a Source Goes Down The degradation path, drawn across all four parties, from healthy through to recovery. HTML page SVG draw.io
26 Data Subject Privacy Officer DSAR Console ECID Crosswalk Virtual DataPort Source Systems Lakehouse Audit Log 1. subject access request 2. open case, verify identity 3. every source id for this person 4. five identifiers, five systems 5. assemble the full record, unmasked 6. delegated reads 7. activity, scores, history 8. complete record with provenance 9. export pack, signed 10. erasure instruction per system 11. erase and rewrite partitions 12. tombstone the enterprise id 13. invalidate every cached row 14. who exported, who erased, when Runtime — Subject Access and Erasure Across Every Copy The crosswalk is what makes erasure provable: it is the only place that knows all five identifiers. Denodo finds the copies; the owning system performs the deletion. v 1.0 · owner Data & AI Global Practice · date 2026-09 Subject Access and Erasure Finding every copy of one person across six systems, and who is allowed to delete them. HTML page SVG draw.io
27 Approved Steward approves two records, one person Crosswalk changes Losing id tombstoned never reused Both source ids kept traceability Survivorship applied attribute precedence Notify Merge event published Event Hubs topic Invalidate Cache listener VDP stored procedure Affected rows dropped both enterprise ids Summaries flagged rebuild on next run Rebuild Next query repopulates cold read, 2.6 s Nightly summary rebuild 03:30 Verified Reconciliation check hourly Consistent within 1 hour at-least-once listener down, stale to TTL A Merge Propagates — From Steward Approval to Every Consumer Application we own Data store Queue / topic Opportunity event / async failure / alternate The red path is the residual risk: if the listener misses an event, the reconciliation job is what closes the gap, and the cache TTL bounds it at 24 hours. v 1.0 · owner Data & AI Global Practice · date 2026-09 A Merge Propagates From a steward's approval to every consumer agreeing, and what happens if the notification is missed. HTML page SVG draw.io

Operations

Where it runs, how it survives a zone and a region, how a view reaches production, what is watched, and what it costs.
28 Azure North Europe — primary, active Availability zone 1 VDP Interactive 1 pod, 8 vCPU VDP Data Services 1 pod, 8 vCPU MPP workers 2 nodes Availability zone 2 VDP Interactive 1 pod, 8 vCPU VDP Data Services 1 pod, 8 vCPU MPP workers 2 nodes Availability zone 3 VDP Interactive 1 pod, 8 vCPU VDP Analytical 2 pods, 16 vCPU MPP workers 2 nodes Zone-redundant services PostgreSQL Flexible ZRS, HA pair ADLS Gen2 ZRS Key Vault zone redundant Event Hubs ZRS namespace Azure West Europe — warm standby Compute, scaled down VDP pods 1 per pool, idle Solution Manager standby node Replicated state PostgreSQL replica geo, RPO 5 min ADLS GRS asynchronous Metadata replica revision synced Salesforce SAP S/4HANA ServiceNow Traffic Manager health-probed failover geo-replication GRS failover Deployment — Three Zones Live, One Region Warm Application we own Data store Security / platform Queue / topic External / third party Interface / broker event / async failure / alternate Losing a zone loses a third of capacity and no session state. Losing the region costs 30 minutes to RTO, and the cache is deliberately not replicated because it is rebuildable. v 1.0 · owner Data & AI Global Practice · date 2026-09 Deployment What runs where, what survives a zone, and what a region failure actually costs. HTML page SVG draw.io
29 One Denodo estate, three isolated workload classes Interactive — target under 2 s VDP Interactive 3 pods Connection pool 200 per source Query timeout 8 seconds Cache-first plans Data services — target under 2 s VDP Data Services 2 pods APIM quota 600 per minute Query timeout 5 seconds No ad hoc SQL published views only Analytical — minutes are acceptable VDP Analytical 2 pods MPP workers 6, autoscaled Query timeout 30 minutes Row limit 50 million Shared below the line, and therefore governed by quota Source connection pools per-class ceilings Cache database shared, sized for peak Metadata database read-mostly Licence cores the real ceiling capped at 40% priority share Workload Isolation — Why a Notebook Cannot Slow the Contact Centre Application we own Decision point Risk / gap Data store failure / alternate synchronous Isolation stops at the source. A single ERP connection pool is the one thing all three classes still share, which is why it is drawn as a risk and capped per class. v 1.0 · owner Data & AI Global Practice · date 2026-09 Workload Isolation Three classes with different latency contracts, and the one resource they still share. HTML page SVG draw.io
30 Author Design Studio development server VQL export text, diffable Version Git repository Azure Repos Pull request one view set Validate VQL lint naming, layering Regression suite 180 queries Policy scan no unmasked PII Approve Peer review Steward sign-off certified views only Promote Solution Manager revision, one artefact Environment properties no hard-coded hosts Environments Test masked subset UAT production shape Production change window row counts differ rollback to prior revision How a View Change Reaches Production, and What Stops a Bad One Application we own Data store Decision point failure / alternate The regression suite asserts row counts and column sets against a frozen dataset, so a widened join is caught before UAT rather than by a report that quietly doubles. v 1.0 · owner Data & AI Global Practice · date 2026-09 Environments and Delivery How a view change reaches production, and the three gates that stop a bad one. HTML page SVG draw.io
31 Emitted by Collected by Stored in Consumed by Acted on by Query performance VDP JMX metrics Prometheus exporter Azure Monitor Grafana dashboard SRE on call Source health Data source probes Denodo Monitor Log Analytics Source SLO board Circuit breaker Cache effectiveness Hit and miss counters Prometheus exporter Azure Monitor Materialisation review Scheduler tuning Slow and failed queries VDP query log Diagnostic tool Log Analytics Weekly slow report View owner API usage APIM telemetry Application Insights Log Analytics Consumer report Quota change Dataset usage Catalog access log Catalog exporter Log Analytics Certification review Data steward Access and audit VDP audit events Diagnostic tool Microsoft Sentinel Detection rules Security operations Cost Licence core usage Azure cost export Cost workspace Chargeback report Platform owner Observability — Every Signal, From Emission to the Person Who Acts Every row ends in a named person or an automated control. A signal with no one in the last column is a dashboard, not an operational capability. v 1.0 · owner Data & AI Global Practice · date 2026-09 Observability Every signal from emission to the person or control that acts on it. HTML page SVG draw.io
32 Usage telemetry query log, catalog hits Slow query analysis top 20 by cost Freshness class review with the data owner Cache or summary change one artefact Promotion through gates same pipeline as a view Measure the effect P95 and source load Materialisation loop what is actually used candidates agreed staleness revision new baseline next month's signals The Operating Loop — How the Platform Decides What to Materialise Next Materialisation is a governed change, not a tuning knob. A summary added without an owner agreeing its staleness is a correctness decision taken by an engineer. v 1.0 · owner Data & AI Global Practice · date 2026-09 The Operating Loop How the platform decides what to materialise next, and why that is a governed change. HTML page SVG draw.io
33 Fixed, whatever the load Denodo licence cores largest single line Control tier Solution Manager, Studio Baseline AKS nodes 3 zones, minimum Scales with concurrency VDP pods per workload class API Management tier premium, zonal Source connections vendor call limits Scales with data MPP workers per TB scanned Cache database storage and IOPS ADLS and Delta 7-year retention Databricks SQL gold serving Scales with movement Event Hubs units throughput CDC and egress on-premises links Geo-replication DR only Levers that actually move it Summary over MPP cheapest per query Shorter TTL only where needed Federate rather than copy no storage, no pipeline Right-size the analytical pool Cost Model — What Drives Spend, and Which Levers Move It Virtualisation removes pipeline and storage cost and adds query cost. The platform is cheaper than a physical hub only while the freshness contract keeps expensive queries off the critical path. v 1.0 · owner Data & AI Global Practice · date 2026-09 Cost Model What drives spend, what it scales with, and the four levers that actually move it. HTML page SVG draw.io

Assurance

Trust boundaries, identity, the single policy set every channel inherits, sensitive data, governance, and every failure mode named.
34 Internet — untrusted Browser and mobile Credential stuffing Scripted extraction Perimeter — Azure edge Front Door and WAF TLS 1.3, OWASP set API Management JWT, quota, schema DDoS Protection Application — private virtual network VDP pods no public address Data Catalog Design Studio named users only Solution Manager Data — reachable by private endpoint only Cache database ECID Crosswalk ADLS Gen2 CMK encrypted Databricks no public workspace Control — separate subscription, separate administrators Key Vault HSM-backed keys Microsoft Entra ID conditional access Microsoft Sentinel write-once sink Audit archive 13 months, immutable TLS 1.3, WAF blocked bearer, mTLS Private Link managed identity audit Trust Zones — What Crosses Each Boundary, and What Proves It May External / third party Risk / gap Interface / broker Security / platform Application we own Data store synchronous failure / alternate event / async No secret is held by a consumer and none by a pod. Every credential is fetched at runtime under a managed identity, and the audit path is one-directional by design. v 1.0 · owner Data & AI Global Practice · date 2026-09 Trust Zones Where the boundaries are, what crosses each one, and where an attacker actually arrives. HTML page SVG draw.io
35 Relationship Manager Power BI Microsoft Entra ID Denodo Load Balancer Virtual DataPort Key Vault SAP S/4HANA Microsoft Sentinel 1. opens the account report 2. OIDC authorisation code flow 3. access token with group claims 4. ODBC connection, bearer token 5. least-busy node 6. validate signature, read claims 7. roles: rm_emea, no_payment_data 8. map roles to view grants 9. fetch the SAP service credential 10. secret, 60-minute lease 11. rows for the delegated predicate 12. row filter by book, mask payment columns 13. authorised rows only 14. principal, view, row count, masks Identity — From a Sign-In to a Filtered, Masked Result Set The source sees the platform identity, not the user. That is a deliberate trade: pass-through authentication was rejected because SAP and Salesforce cannot express the enterprise role model, so the filter must be applied where the model exists. v 1.0 · owner Data & AI Global Practice · date 2026-09 Identity and Access From a sign-in to a filtered, masked result set — and the one trade-off in the chain. HTML page SVG draw.io
36 Datasets granted Column policy Row policy Channels Service Agent Customer360, Support Contact masked to last 4 Own contact-centre queue Desktop REST, assistant Relationship Manager Customer360, Orders, Value Full contact, no payment Own account book Power BI, Excel BI Developer Certified business views Value masked until granted All rows, aggregate only Design Studio, Power BI Data Scientist Integration and business views Pseudonymised keys Sampled cohorts JDBC from notebooks Data Steward Every view and the crosswalk Unmasked, reason recorded All rows Studio, catalog, console Privacy Officer DSAR assembly view only Unmasked, case-scoped One subject at a time DSAR console Platform SRE Metadata and logs only No customer columns No customer rows Monitor, Solution Manager AI Assistant Whatever the caller has Whatever the caller has Whatever the caller has MCP tools only Partner Portal Contracted subset No personal data at all Own contracted accounts REST through APIM Authorisation — One Policy Set, Inherited by Every Channel The assistant row is the whole security argument for AI: it has no grants of its own, so nothing had to be reasoned about twice. v 1.0 · owner Data & AI Global Practice · date 2026-09 Authorisation Model One policy set per role, and every channel inheriting it without restating it. HTML page SVG draw.io
37 Classify Tag Policy Enforce Prove Direct identifiers Name, tax id pii.direct Deny unless granted Full redaction Access log per query Contact data Email, phone, address pii.contact Partial by role Last four shown Mask applied count Financial Card, IBAN, balance pii.financial Deny by default Tokenised at source Never cached Behavioural Clickstream, sessions pii.behaviour Pseudonymised Hashed device id Re-identification review Derived scores Lifetime value, churn restricted.model Approved purpose only Row filter by book Purpose recorded Sensitive Data — Classified Once, Enforced Everywhere Classification is declared on the integration view, not on each business view. A new business view inherits every mask the moment it is created, which is why the layering rule is a security control. v 1.0 · owner Data & AI Global Practice · date 2026-09 Sensitive Data Five classes of sensitive data, classified once and enforced everywhere above it. HTML page SVG draw.io
38 Describe Business glossary one definition each Technical metadata harvested from VQL Named owner person, not a team Classify Sensitivity tags drives masking Retention class Permitted purpose Certify Steward review Certified badge with an expiry Deprecation notice 90 days Discover Denodo Data Catalog search and sample Purview federation estate-wide search Request Access request on the asset page Owner approval one working day Grant provisioned Entra group Trace Column lineage to the source field Impact analysis before any change Usage statistics who, what, how often recertify on change Governance — From a Business Definition to a Provable Column Lineage Application we own Security / platform Decision point batch Lineage is derived from the view definitions themselves, so it cannot drift from the code. The part that can drift is ownership, which is why certification expires. v 1.0 · owner Data & AI Global Practice · date 2026-09 Governance and Lineage From a business definition to a column-level trace, and the loop that keeps ownership honest. HTML page SVG draw.io
39 Source-side — expected, absorbed Source unavailable partial result, view 25 Source slow timeout, circuit opens API quota exhausted backoff, cache serves Schema drift base view breaks in CI Platform — degraded, not lost VDP pod loss retried, no session state Cache database loss cold reads, 3x latency MPP cluster loss falls back to the source Region loss RTO 30 min, RPO 5 min Correctness — the class that actually matters Stale cache after a merge reconciler bounds at 1 h Crosswalk corruption point in time, 7 days Wrong survivorship pick replayable from source ids Masking regression policy scan blocks release Programme — slow, and therefore easy to miss Uncertified view in use promotion gate blocks it Owner leaves, asset orphaned certification expires Extract copied to a spreadsheet usage telemetry only Denodo becomes the bottleneck the real programme risk same root cause concentration risk Failure Modes — What Breaks, What Absorbs It, and What Is Left Over Risk / gap synchronous failure / alternate The last box has no technical mitigation. Every consumer depending on one logical layer is the price of the design, and it is managed by workload isolation and honest capacity planning rather than by architecture. v 1.0 · owner Data & AI Global Practice · date 2026-09 Failure Modes Every named way this breaks, what absorbs it, and the one class with no technical answer. HTML page SVG draw.io
Open svg/<view>.svg or drawio/<view>.drawio in draw.io Desktop or at app.diagrams.net to edit. The SVG carries the diagram inside it, so it is both the picture and the source. This folder is self-contained — copy it whole and every link still resolves.
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 39 views · Denodo Platform, Microsoft Azure and open source39 views · HTML / SVG / draw.io