Building a data function from zero

Situation

I joined to build a data team that did not exist. Not an underperforming one, an absent one.

What existed instead is worth describing precisely, because it is a specific and very common arrangement that nobody designs on purpose:

• Data engineers working in a vacuum. Competent people building pipelines without a clear line to the decisions those pipelines fed. Technically capable, organizationally unmoored.
• Product managers asking developers for numbers. With no analytics function to ask, questions went to whoever could write a query against a production database. That is expensive engineering time spent on reporting, and it is reporting done by people with no context on what the number is for.
• Analysts hired by individual departments. Each department, reasonably, solved its own problem by hiring its own analyst. Each analyst was good. Each produced numbers by their department’s definition of the metric, and nobody, anywhere, had written those definitions down.

That last point is the one that mattered. The numbers were not wrong. Every number was correct within its own frame. No shared frame existed, and the definitions driving the business lived in the heads of four or five people who had never compared notes. There was no formal code review, no data governance, and no owner for any data source.

You could see it in the most basic number. Finance’s total revenue and the growth team’s total revenue had never matched, so average order value, revenue per user and every other ratio built on them never matched either. Users and customers meant one thing to the developers, another to the data people, and something else again in each business department. Executives knew, and had settled into the view that the numbers would never match but were close enough. When that stopped being tenable, I was hired to fix it at the root and lift the data platform with it.

There was also a great deal of data. This was not a company short of information. It was drowning in it, with no agreed way to turn it into a claim anyone could defend. Abundant data and absent governance is a harder starting point than scarce data, because everyone already believes the data problem is solved.

The trigger for change is almost always the same: two right numbers in one meeting that do not match, and forty minutes spent working out whose is which instead of deciding anything.

Constraint

• Nothing could stop working. Every report running had someone depending on it. A centralization that breaks month-end in its first quarter does not get a second quarter.
• The analysts did not report to me, and their managers had no reason to want that changed. Each department had a responsive, dedicated analyst who understood its business. From where they sat, the arrangement was working.
• No mandate worth relying on. Authority on paper does not make a department head stop trusting their own analyst. Trying to enforce it burns exactly the goodwill the work depends on.
• Early hires are disproportionate. The first few people set the culture of the function for years. Hiring fast to show progress is the most expensive mistake available.
• Evaluated on visible output while doing invisible work. Foundations do not demo.

The second constraint defeats most attempts, and it deserves respect rather than irritation. The departments were not being obstructive. They had built something that worked for them, and I was asking them to give it up for a promise.

Decision

Centralize the definitions first, the people second, the tools last. That ordering is the argument, and it is close to the reverse of how these programs usually run.

Most start with tooling, because tooling can be procured, it is visible, and it feels like progress. A warehouse gets bought, a migration begins, and eighteen months later there is a better platform serving the same irreconcilable definitions to the same skeptical departments. The technology improved. Nothing else did.

Form a core team first

The engineers already in the building were the starting point. Pulling them into a core data team, with a clear remit, a shared roadmap and a line to the decisions their work fed, solved the vacuum problem immediately and cost no headcount. It also gave me something to show departments other than a plan.

Bring analysts to the center without taking them from the business

This was the hardest conversation, and the one the whole thing turned on.

The proposition I put to each department was not “give me your analyst.” It was: your analyst joins the central team for governance, standards, tooling and career development, and keeps serving you, with the central platform behind them instead of a spreadsheet. Where a department needed more dedicated capacity, it could keep or hire its own business analysts, now building on governed data rather than inventing definitions.

That framing mattered enormously, and it is the part I would not compromise on again. The department kept what it valued: proximity, responsiveness, and someone who knew its business. What it gave up was the freedom to define metrics privately, which was never a benefit but the source of the problem. Stated that way, most department heads agreed. What the pitch undersold was control over the analyst’s week. Keeping proximity real took protected capacity and a way to settle competing priorities, and that loss had to be negotiated, not explained away.

Getting there took a great many conversations, one department head at a time, which I have written up in Data in the boardroom.

Take custody of the definitions

The first genuinely central asset is not a platform. It is an agreed, written, owned set of definitions for the metrics that appear in decisions. Who counts as active. What revenue means before and after which adjustments. When a customer is churned.

In practice that meant three things. Each business owner got their KPIs and metrics, and became the approver of any change to a definition. The business logic moved into the governed model, so no analyst could come up with their own version of a metric. And a company-wide data dictionary and governance tool tracked every definition where anyone could read it, which kept ownership from becoming one more job for the business. Each definition was labeled corporate standard, the same in every report, or corporate custom, kept for the one department that needed it under a name of its own.

This is deliberately separable from reporting lines. An analyst can build against a governed definition regardless of who signs their review. Once several departments build against the same definitions, centralization has happened in the way that matters, and the org-chart question becomes one about capacity, accountability and career development.

The published example I point to is Airbnb’s account of Minerva, its shared metrics platform, written up by its engineering team in 2021: more than two hundred data producers contributing to one set of definitions, held in a shared repository with review and validation. It is the same idea at a far larger scale: one set of definitions, contributed to across functions.

Build the platform to serve contributors, not to control them

The platform’s job is to make the governed path the easy path: version-controlled transformations, tests that run before anything merges, and lineage documented well enough that a number can be traced without asking anyone. If doing it correctly is harder than doing it privately in a spreadsheet, people will do it privately in a spreadsheet, and they will be right to.

Train relentlessly: this is the deliverable

The part that took longest and mattered most.

Analysts arriving from departments knew their business cold and had mostly never worked in version control, written a test, or had their work reviewed. Engineers knew the practices and often did not know what the numbers meant. Data scientists, added later, needed both, plus a reproducibility discipline neither group had by default.

What worked was sustained rather than intensive: office hours, worked examples from their own domain, pairing on the first few changes, and generous review of early work. Not a workshop. A workshop produces people who attended a workshop.

The goal was a specific and initially uncomfortable standard: everyone speaking the same language, and every metric reproducible by someone who did not build it. That took time. It was also the clearest single marker of success, because once it held, the function stopped depending on anyone’s memory.

Hire for the shape of the problem

The first hires should be people who can talk to a department head in the morning and write a tested model in the afternoon. That combination is rare and worth waiting for. Pure engineers build something technically excellent that the business does not use. Pure analysts produce answers faster than anyone can govern them.

I would rather run understaffed for a quarter than fill a seat with someone who does one half of the job.

When I would not do it this way

If the departments already have capable data leads, reliable shared definitions and a working way to settle cross-team dependencies, I would start with common standards and shared services and leave the reporting lines alone. This approach suited a missing function and fragmented ownership. It is not an argument that every data team belongs in the center.

The tradeoff I accepted

I traded speed of centralization for durability of adoption, and it cost visibly.

A mandate-driven version can be announced in a week. Reporting lines change, a platform is selected, and there is a clean before and after that looks decisive on a slide. Earning one department at a time, governing definitions before people, took considerably longer to reach the same point on the org chart, and for most of that time the true status was “partially centralized,” which is a difficult thing to defend in a leadership meeting.

I accepted it because the failure of the fast version is not slower adoption. It is false adoption: departments comply on paper, keep their real numbers privately, and the central team discovers much later that it governs the dashboards nobody uses for decisions while the actual decisions run on a spreadsheet maintained by someone’s analyst. That is nearly impossible to detect from inside and very hard to reverse, because by then the central team has a credibility problem rather than a coverage problem.

The real costs:

• Sustained ambiguity about ownership. For several quarters some numbers were governed and some were not, and it was not always obvious which. That produced recurring questions I could not always answer cleanly.
• Training competed directly with delivery. Every hour spent pairing with an analyst learning version control was an hour not spent shipping. The pressure to shortcut it is constant, and shortcutting it produces people who can follow the process without understanding it, which fails the first time something unusual happens.
• A slower headcount ramp than the workload justified. Holding the hiring bar meant the existing team absorbed more than was reasonable for a period. That is a real cost paid by real people, and it is only defensible if you say so out loud and it actually ends.
• Politically exposed sequencing. Choosing which department to serve first is a visible allocation of a scarce resource. The ones not chosen notice.
• No moment of completion. Nothing ever launched. There was no day the data function existed where it had not the day before, which makes the work hard to recognize and reward, including for the people doing it.

Outcome

The signal I cared about was not platform adoption or ticket throughput. It was whether departments stopped keeping private numbers. Exploration in a spreadsheet is fine. A private number in a decision is the test.

It is binary per department. Either finance walks in with the governed number or they bring their own. When the private spreadsheets stop being maintained, not because they were banned but because they stopped being necessary, the function exists.

Getting there took a lot of convincing, a lot of negotiation over each department’s capacity, and some skepticism that only results could answer. The results did: a governed, stable platform feeding the reporting and the self-service tools raised how available the company’s data was, and how far people trusted it.

You could see it in how people worked:

• Meetings started from the number rather than arriving at it. The time spent reconciling two versions went back to deciding things.
• Engineers stopped working in a vacuum, and product managers stopped asking developers for numbers. Both groups got their actual jobs back.
• The shared language held. Analysts, engineers and data scientists describing the same metric the same way, with reproducible logic behind it, is what let the team grow without fragmenting again.
• Delivery became repeatable. Code review, deployment standards and data quality checks became part of ordinary delivery and outlasted the projects that introduced them.
• Departments asked for data people in their planning rather than sending requests afterward. That inversion is the clearest evidence that proximity survived centralization.
• The team could absorb someone leaving. A function that depends on one person knowing where everything is has not been built yet.

What I’d do differently

Write down what the function will not do, on day one. I defined the mandate by what the function would own. The gap that left was everything adjacent: operational reporting, one-off extracts and tooling for other teams, all arriving as requests with no basis for declining them. A short published statement of what sits outside the function is worth more than a detailed one of what sits inside.

Name the sequencing publicly, with reasons. I chose which department to serve first on good grounds and communicated it poorly, so the others experienced the sequencing as favoritism. Publishing the order and the reasoning, including when each department could expect to be next, would have cost nothing.

Start the training earlier, and treat it as a first-class workstream. I treated enablement as something that accompanied the platform. It is the platform. The capability to use it correctly is the deliverable, and resourcing it as an afterthought slowed everything downstream.

Protect the embedded model in writing before it is under pressure. Proximity to the business is the first thing sacrificed when the team is overloaded, because pulling people back to a central queue is the obvious efficiency. It is an efficiency that destroys what made the function valuable, and it happens quietly unless the structure is explicitly defended.