concept · concepts/Data Contract.md

Data Contract

status: active · created 2026-06-26 · 2 sources · #data-engineering #data-modeling #pipelines #interfaces #craft

The agreed interface between a data producer and its consumers — schema, units, semantics, and the derivations performed at the boundary. The governing rule: derive once, at the contract, so every consumer inherits a correct value instead of re-deriving (and re-bugging) it.

Grounded 2026-06-26 by Sanderson — The Rise of Data Contracts (2022), the article credited with defining the modern movement — fetched under the auto-ground convention. It confirms the heuristic's framing and supplies the "nonconsensual API" diagnosis and the boundary-enforcement rationale. (Andrew Jones of GoCardless is a widely-credited parallel originator; see the source page.)

Explanation

A data contract is where producer and consumer agree on what the data means, not just its column names. The source (Working Heuristics — Data, Matching, Product, Org, Tooling) makes one move central: derivation math belongs in the contract. Timezone conversion, unit normalization, format coercion — do them once at the boundary and publish the corrected value. The alternative, letting each consumer re-derive, guarantees that the same bug (an off-by-one timezone, a unit mismatch) gets re-implemented N times and fixed inconsistently.

This is an instance of the broader Do It Once, at the Right Place principle — the same logic that puts compute where the data lives and persists a human decision so it's paid once. Re-derivation, like re-computation and re-confirmation, is the waste to design out.

Chad Sanderson (Sanderson — The Rise of Data Contracts, 2022) sharpens why the boundary is the place: without a contract, a production database is a "nonconsensual API" — engineers never agreed to expose operational tables for analytics, so a dropped column silently breaks the pipeline downstream (his example: "the pricing algorithm that generates 80% of your company's revenue is broken after the column is dropped"). The contract moves the commitment to the producer side — strongly-typed schemas, versioning, semantics — instead of consumers "stitching the world together retroactively through SQL." Kimball's grain declaration is the same instinct one layer in: he too calls it "like a contract" (One Fact Per Grain).

Why it matters / where it shows up

  • The boundary is the cheapest place to be correct. Fixing a derivation at the contract fixes it for everyone downstream atomically; fixing it per-consumer is N edits and N chances to diverge.
  • It pairs with grain and lineage. A contract that publishes a correctly-derived value at the right grain, over a legibly-named pipeline, is what makes a dataset trustworthy without re-reading the source each time.
  • Sensitive fields belong in the contract too. Deciding what to expose is a contract decision — and the source's rule there is Allowlist over Blocklist: name the fields you keep, drop the rest.

Open questions

  • When derivation legitimately differs per consumer (different rounding, different fiscal calendars), how much should the contract publish vs. leave to the edge?

Connections

Sources

The wiki is the source. This door reads it at build time and never writes to it.