Giving the models away

Situation

This is usually filed as a tooling decision and it is not one. It is an org-design decision about where authority sits, and the tooling is what makes the chosen arrangement survivable. Teams that approach it the other way round, adopting the tool and expecting the operating model to follow, get the tool and keep the bottleneck.

The bottleneck itself is common and self-inflicted: every model change routes through a central data team that is also responsible for the platform itself. Both jobs suffer. Analysts wait days for changes they could make themselves; the platform work gets deferred indefinitely because request queues are visible and technical debt is not.

The shape of the queue matters more than its length. A backlog of large, genuinely complex requests is a staffing problem. What actually accumulates is different: a long tail of small changes (add a column, adjust a filter, rename a field to match what the business now calls it) each taking twenty minutes of work and sitting for a week. The ratio is the tell. When most of the queue is work that takes less time to do than to triage, the queue is not a capacity problem. It is a permissions problem wearing a capacity problem’s clothing.

The cost compounds in two directions. Analysts, entirely rationally, stop asking. They build the logic in their BI tool instead, or in a spreadsheet, or in a CTE they paste into every query. The warehouse stops being the place the definitions live, and the central team’s ownership becomes nominal, they own a shrinking fraction of the logic that actually drives decisions, and they own it carefully.

Meanwhile the platform work never happens. Nobody escalates a deferred migration the way they escalate a blocked dashboard.

Constraint

• Varying SQL and version-control fluency across analysts. Some are effectively engineers. Some are fluent in SQL and have never opened a pull request. Designing for the average serves nobody.
• Production models must stay safe from well-meaning mistakes. The risk is not malice; it is a subtly wrong join in a model that feeds a board report.
• No headcount to add dedicated reviewers. The review burden has to be absorbed by automation or by the contributors themselves, because there is no budget for a gatekeeping function.

The fluency spread is the hard one. It means the safety layer cannot rely on contributors knowing what they are doing. It has to catch the mistakes of someone competent in SQL and new to engineering practice.

Decision

Give the ownership away, and build the guardrails that make it survivable. The governance is the product here, not the tooling.

Testing requirements as the entry price. Every model a domain team owns carries tests: not-null and uniqueness on keys, accepted values on anything categorical, relationship tests on foreign keys. This is unglamorous and it is the single highest-leverage thing in the system. A contributor who cannot write the model correctly can usually still state what must be true about its output, and the test catches the gap between the two.

CI that runs on every change, before merge. Build the modified models and everything downstream of them, run the tests, block the merge on failure. The state:modified+ pattern, rebuilding only what changed and its descendants, keeps this fast enough that people do not learn to route around it. A CI check that takes forty minutes will be bypassed within a month.

A review model that scales without reviewers. Domain teams review their own changes for business logic, because they are the ones who know whether the definition is right. The central team reviews only changes to shared upstream models: the ones many domains depend on. Most changes never reach the central team at all, which is the entire point.

Naming and ownership conventions, enforced mechanically. Every model has an owner recorded in code. Layer prefixes make the dependency direction visible in the name. A linter enforces it. Conventions that rely on people remembering are conventions that decay.

Training that continues past launch. The migration is the easy part. What makes federation stick is ongoing support: office hours, worked examples, someone to ask. Teams that hand over ownership with a one-off workshop get federation on paper and a central team quietly fixing things on Fridays.

The industry has converged on the same conclusion from a different direction. The domain-ownership and mesh literature makes the identical argument about scaling data work without scaling the central team proportionally. The pattern is well-trodden. What the literature consistently underweights is how much of the outcome depends on the test layer being in place first.

The tradeoff I accepted

The honest version, which people implementing this tend to skip: model quality becomes less uniform. A central team produces consistent work slowly. A federated one produces more work, faster, with more variance in how it is built.

You will get models that are correct but inefficient. Duplicated logic across domains that should have been a shared upstream model and was not, because the team that needed it did not know it existed. Documentation that is excellent in one domain and absent in another. None of this is hypothetical. It is the predictable consequence of trading control for throughput, and pretending otherwise is how people get talked into federation and then resent it.

I judged the throughput worth the variance, but that judgment depends entirely on having tests and CI good enough to keep the variance from reaching production. Do this without that layer and you have not federated ownership, you have distributed the ability to break things.

What it actually costs, concretely:

• Duplication gets worse before it gets better. Domains solve the same problem independently. Some of it needs consolidating later, and that consolidation is work nobody asked for and nobody is measured on.
• The central team’s job changes, and not everyone wants the new one. Owning the platform, the standards and the shared layer is a different role from owning every model. It is more leveraged and less immediately visible. Some people took the change as a demotion, and that reaction deserves a straight conversation rather than a reassurance.
• Incidents shift in character. Fewer bottleneck-induced problems, more “a domain changed something upstream of another domain and nobody noticed.” Contract testing on shared models addresses this, and it is worth building before you need it rather than after the first incident.

Outcome

The metric that matters is not throughput, though throughput improves markedly: changes that waited a week go out same-day once the contributor is competent.

The metric that matters is the number of people who can safely make a change. Going from a handful of central engineers to several times that number of independent contributors changes what the organization is capable of, and it does so permanently. Throughput is a consequence; capability is the asset.

Second-order effects worth naming:

• The central team finally does platform work, because the interrupt-driven queue is gone.
• Definitions move back into the warehouse from BI tools and spreadsheets, because contributing is now faster than working around.
• Test coverage becomes something domain teams care about, because it is the thing standing between them and a broken dashboard with their name on it.

What I’d do differently

Build the test and CI layer before announcing the change, not alongside it. Every problem federation caused traced back to a gap in that layer. Running them in parallel means the first wave of contributors experiences the version without guardrails, and their early bad experience sets the tone for everyone watching.

Start with one domain, not a phased rollout across all of them. A single team taken all the way to genuine independence teaches you what the guardrails are missing, at a scale where fixing it is cheap. A broad phased rollout surfaces the same gaps everywhere at once.

Define the shared layer boundary explicitly, on day one. The ambiguity about which models are “everyone’s” and which belong to a domain caused more friction than any technical decision. Write the boundary down, name an owner for everything on the central side of it, and revisit it deliberately rather than letting it drift.

Say the tradeoff out loud to leadership, in advance. “Quality will become less uniform and we are accepting that deliberately” is a sentence that is easy to say beforehand and very hard to say for the first time while explaining an incident.