Architecture FAQ

How to read this FAQ

Provisa provides the mechanisms; operators configure them for their use case. Where a behavior depends on business requirements (stale-data notification, compaction schedules, cross-border transfer rules), Provisa supplies the control and the customer sets it.

Architecture at a glance

Clients connect only to Provisa's in-region data nodes, never to engines or sources. Each request is compiled with governance built in, run on your engine, and streamed back: passed straight through when client and engine both speak Arrow, otherwise encoded to the client's protocol with bounded memory. Nothing is stored, and nothing leaves the region.

• Control nodes provide the UI for exploration and configuration management. They are identical and can be hosted anywhere; host one in-region for regional exploration.
• Data nodes are the client endpoint for a region. They compile requests to the IR, translate them to the engine's dialect, dispatch to the engine or directly to a source driver, and encode results. They are stateless, CPU-bound with bounded memory, and load-balanced horizontally per region.
• Regions are engines with a stated purpose: a jurisdiction, a department, or a processing class such as transactional or analytical. Regions operate independently under local regulation and policy, coordinating through shared configuration.
• Engines (Trino, ClickHouse, Postgres, DuckDB and others; each release lists the engines and versions it certifies) are replaceable execution targets. Engines never talk to each other.
• High availability: the control plane runs in HA mode with HA Postgres as its state store.

Every region runs this same shape under one shared configuration; engines in different regions never talk to each other.

Execution and correctness

1. How are cross-region jobs handled when a join spans sources assigned to different engines?

Engines never talk to each other. All of them share one rule set, interpreted through one compiler. When a request needs data from another region, the engine reads from the source directly, or in place from the home region's replica if the table is homed, with the reader's row and column rules in the statement. An unhomed replica or materialized view is built in every region, each in that region's own store, so every region reads it locally. A homed one is built and refreshed only in its home region, by that region's coordinator; other regions read it in place and never hold a copy, and it is never cached outside its home region.

Homing governs internal handling only. Each region sets its own cross-border transfer rules under local regulation, and the requesting region's identity is carried on every connection so the receiving region can apply them.

2. What happens when a compiled plan uses a construct the target engine lacks?

It is refused at compile time, never sent to fail. Clients speak Provisa's supported languages (SQL, GraphQL, Cypher), and every request compiles to one IR. The IR decomposes each request into a set of CTEs (effectively subroutines) and a final simple query; the translator chooses how each CTE executes on each engine. Functions an engine lacks are replaced by inline macros from a per-dialect polyfill library.

What an engine cannot express, even with a polyfill, is refused by name at compile time, saying which function and which engine. Which functions each engine serves, and which it refuses, is published per engine on the conformance pages.

Provisa publishes benchmarks for a wide range of workloads across all supported engines, and ships the same benchmarking tool to customers as a sizing tool.

3. How is cross-engine semantic equivalence verified (nulls, decimals, collation, time zones)?

Each supported engine is tested against PostgreSQL's own answers, for every protocol suite and every function, in CI. The results are published per engine on the conformance pages: what matches exactly, what is equivalent under a stated rule, what is refused by name, and every known difference with its workaround. The polyfill list is documented.

4. How are long-running jobs handled: retries, partial failure, checkpointing, and cancellation?

Execution retries, checkpointing, and fault tolerance are the engine's responsibility; Provisa does not duplicate them. Engine errors propagate to the client, and client cancellations propagate to the engine.

State and freshness

5. How does incremental materialization work: change detection, merge semantics, late-arriving data, backfills?

Provisa replicates two ways.

• Table-level replicas are whole-table copies. A refresh fills a fresh table and swaps it in atomically when complete, so a reader never sees an empty or half-filled replica and a failed refresh leaves the previous one intact. On Iceberg each append is one snapshot. When to refresh comes from change detection: CDC where the source supports it, and otherwise a combination of TTL, hashes, ETags, modification dates and size, chosen per source according to what it exposes.
• Row-level replication is a query-driven row cache. A row is fetched from the live source and upserted into the replica only when a query's compiled plan names concrete primary-key values, so it requires a unique primary key.
• Compaction of replica and view tables stored in Iceberg is the operator's responsibility. (Provisa compacts its own telemetry tables.)

6. How is freshness defined and enforced, and what does a consumer see when a materialized view is stale?

Staleness is determined by the same per-source change detection used for replication. Each materialized view is configured for one of two behaviors when a query arrives and the view is stale:

1. Force a refresh, then serve fresh results.
2. Serve the stale result and refresh in the background. Background refreshes are debounced.

Freshness is set per source. A source with no TTL or freshness guarantee is always queried directly. A TTL of -1 marks reference data that is never refreshed.

Stale-result notification to end users is not built in. Where it matters, configure it for your use case.

7. How does the control plane learn job outcomes from engines that have no callback mechanism?

The question does not apply. Every replica build and materialized-view refresh is recorded centrally and run by a node that submits it to the engine, holds that connection and receives the result or error directly. Data nodes, change signals and model changes request a rebuild (a data node does when a query finds a replica stale); the region's coordinator nodes claim and run builds and scheduled refreshes. There are no unobserved jobs.

Governance in the data plane

8. Since the engine executes the plan, what stops someone with direct engine access from bypassing policy?

Clients connect only to data nodes, and only Provisa holds engine and source credentials, so there is no path around governance. Even when client and engine both speak Arrow, the connection is mediated by the data node. Anyone who holds direct credentials outside Provisa (a DBA, a legacy application) is outside Provisa's scope by definition; that is an access management decision the organization already makes for every system.

The practical result is a security improvement in itself: analysts, applications, and AI agents no longer need standing credentials to data systems.

9. How do credentials flow?

Provisa is the sole credential holder for engines and sources. Clients hold none.

10. How is the audit trail correlated between the compiled plan and what actually ran?

The data node compiles each request and delivers its results, so it is a single audit point in the region where the request ran.

Scale and operations

11. What are the limits of the control plane itself: model size, number of sources, concurrent compilations?

No practical limit has been found. Testing covers several thousand tables, including a 4 TB warehouse with 54 schemas and about 1,000 tables, registered across duplicated connections to build very large, complex workloads.

Runtime cost does not grow meaningfully with model size: once the model is compiled, query compilation is almost entirely direct lookups. The cost of a very large model shows up at authoring time, because a configuration change made in the UI recompiles the model, so editing becomes less responsive.

Data nodes are stateless and scale horizontally per region. Control nodes handle only UI and configuration.

12. How are model changes promoted without breaking running jobs or consumers?

Changes are promoted through environments (feature, dev, test, preprod, prod) using Provisa's environment branching and approval, with testing at each stage to catch changes that would break consumers. Patching production configuration without that path is not recommended.

Configuration is applied to a running production instance. A model change recompiles the served schema; requests arriving during that short recompile wait for it, and depending on query state an in-flight job may occasionally be interrupted. Clients should handle brief interruptions, as they would with any service.

13. What is the upgrade path when an engine version changes its dialect or behavior?

Tens of thousands of test cases (unit, integration, conformance and UI end-to-end) gate every release; nothing ships without them passing. Each release publishes the list of supported engine versions. New engine versions are certified as quickly as possible, but acquiring, certifying and adjusting for them takes time. Running engine versions not on the list is at the customer's own risk.

Performance

14. Does Provisa use engine statistics, or rely entirely on the engine's own optimizer?

Cost-based optimization uses the engine's statistics and the engine's own optimizer. Provisa applies optimizations that need no statistics before the plan reaches the engine:

• Routing single-source queries directly to the source driver.
• Transparently rewriting queries onto fresh materialized views.
• Choosing how each CTE executes on each engine.
• Inlining small, frequently used tables into the query, so a cross-source join becomes a single-source one.

15. What is the compile overhead per query for interactive workloads?

Overhead scales with query complexity and with the number of governance features applied: under 5 ms for minimal queries and under 50 ms for larger ones. Compiled plans are cached, keyed on the query's shape, the caller's role and identity, and the model version, so a plan compiled for one caller is never served to another. Results by workload are in the published benchmarks.

Streaming

16. Is there a streaming execution model, or is Kafka only used for change notification? Where does Provisa stop and Flink begin?

Kafka and Debezium are dependencies of specific features (Kafka sources, CDC change detection, client subscriptions), not part of Provisa's execution architecture. Provisa does not perform stateful stream processing (windowing, streaming joins, continuous aggregation). That is the role of a stream processor such as Flink, whose output topics or tables can be registered in Provisa as governed sources.

Subscriptions are governed like every other path. There is no route to data that bypasses governance.