Warehouse consolidation across mergers, silos, and migrations
One number, two histories
Mergers, departmental silos, platform migrations: the surface story differs every time. The invariant is that two groups have incompatible definitions of the same metric, and the pipelines are the easy part.
A pattern I have run repeatedly, in several different forms
Observed changePreviously separate warehouses brought together into a shared reporting foundation
What this is about
- Definition reconciliation is the project; the pipeline work is support
- Agree on the metric before writing the model, or relitigate it forever
- A slower visible start buys a result people actually trust
Drawn from post-merger data integrations and warehouse consolidations I have completed. The examples combine patterns that recurred across them.
Situation
This problem arrives wearing several different costumes, and it took me a few rounds to notice they were the same problem. I have met it in more than one company merger, and again in technical integrations inside a single company.
Sometimes it is a merger: two companies, two warehouses, two reporting stacks. Sometimes there is no merger at all, and marketing and finance simply built separate platforms over several years because neither could wait for the other. Sometimes it is a migration, where the new warehouse was supposed to be a technical lift-and-shift and turned out to carry two incompatible definitional histories across with it. Occasionally it is the acquisition of a product rather than a company, which is the same thing at smaller scale with less political cover.
The surface stories differ. The invariant does not: two groups arrive with their own accumulated history of how numbers came to be defined, leadership wants one set of numbers, and what exists is two sets that disagree, plus a third assembled by hand in spreadsheets whenever somebody needs an answer that spans both.
The disagreements are rarely dramatic, and that is what makes them dangerous. Both sides report active customers. Both are correct by their own definition. One counts an account, the other counts a paying entity, and a customer with three subsidiaries is one number on the left and three on the right. Nobody is wrong. There is simply no such thing as the active customer count until somebody decides there is.
Two patterns showed up almost every time. First, the divergence was older than anyone still working there: the definitions were set by people who had since moved on, for reasons that were sound at the time and were now undocumented. Second, the spreadsheet layer was load-bearing. In one case it was a single person in accounting, checking vendor invoices in their own spreadsheets. Those spreadsheets never matched what the database had captured, so accounting’s numbers and the data team’s numbers rarely agreed, and only a few people knew why. A private reconciliation like that is the real system of record, and migration plans almost never list it.
Constraint
- Reporting continues throughout. There is no freeze window. The business does not stop needing numbers because the data team is busy.
- The deadline, where there is one, comes from the financial calendar, not from engineering. A first combined quarter-end is a fixed date, and it arrives whether or not the platform is ready.
- The team doing the consolidation is the team already running everything else. The work is additive. Nothing gets paused to make room for it.
The calendar binds hardest, and in an awkward way: it sets a date for the output while saying nothing about the definitional work that has to come first. That asymmetry is what pushes teams toward lift-and-shift.
The two political situations bind differently. With a merger, the mandate is handed to you and the deadline is real, so the difficulty is speed. With departmental silos there is no deadline and no mandate. Each department is functioning adequately by its own lights, and consolidation is something you want and they have not asked for. That version needs a demonstrated win before anyone will fund the rest, and choosing which department to start with matters more than any architectural decision in the project.
Decision
The core call: treat definition reconciliation as the actual project and the pipeline work as the supporting task. The intuitive ordering is the reverse, and it is the wrong one.
In practice that means getting finance, growth, and analysts from both sides to agree on a definition and an intended use for each shared metric before anyone rebuilds its model. Not a mapping document. Not a spreadsheet of “brand A calls this X, brand B calls it Y.” An actual decision, with a named owner, about what the combined organization means by the word. Where the difference is legitimate, keep it and name it: booked revenue, recognized revenue and cash received answer different questions, and erasing the difference for the appearance of consistency just moves the argument somewhere harder to see.
What works:
- Inventory by usage, not by existence. Rank every metric by how often it appears in decisions people actually make. The long tail does not need reconciling on day one, and treating it as though it does is how these projects stall.
- One named decider per metric. Not a committee. Committees produce definitions with conditional clauses, which are definitions that have not been agreed.
- Write the decision down before the model. A short definition record takes twenty minutes to write: what counts, what is excluded, why, who decided and when. Skipping it means having the same conversation every quarter, with worse recall each time.
- Rebuild the reconciled metrics, and leave the rest alone on purpose. Not everything needs to move. Deciding to leave a legacy report running until it ages out is a legitimate architectural choice, and it is cheaper than migrating something nobody will read next year.
None of this means stopping the engineering. Source discovery and infrastructure work can run in parallel. The line I hold is that a shared metric does not cut over while its consumers still disagree about what it represents.
The tradeoff I accepted
Front-loading the definition work pushes the first visible deliverable out by weeks compared with a lift-and-shift. For a meaningful set of core metrics, expect the reconciliation alone to run six to ten weeks before most of the models are written, and during that time what the business sees from the data team is meetings. That is an uncomfortable position to hold, especially when a parallel workstream elsewhere in the integration is shipping something visible every week. The way through is to make the progress visible anyway: publish each definition as it is decided, and ship one small reconciled slice early.
I took the slower start in exchange for not relitigating every number afterward. The failure I was avoiding is common and expensive: the migration technically completes, the dashboards render, nobody believes the output, and the spreadsheets never go away. At that point you have paid the full cost of the migration and kept the full cost of the manual reconciliation it was meant to eliminate.
The real cost is that the definitional debate surfaces organizational disagreement the two sides had avoided by not talking to each other. Some of those arguments are not really about data and cannot be settled by a data team. Deciding whose customer count wins is occasionally a decision about whose function matters more, and it escalates. Budget for that escalation. It is not a sign the process is failing. It is the process surfacing something that needed deciding.
Outcome
Across those mergers and consolidations, I brought separate warehouses into a shared foundation for reporting. The scope differed each time, and consolidation never meant moving every historical report or forcing every measure into one definition.
The result worth aiming for is not a faster close, though that usually follows. It is that the reconciliation spreadsheet gets deleted and stays deleted. The signs I look for:
- The close shortens by several days, because the manual reconciliation step is gone rather than automated. Check that the work disappeared rather than moved to another team.
- Disagreements about a number become disagreements about the definition, which have a written answer, a named owner, and a route to resolution.
- New metrics get defined before they get built, because the pattern is now established and the cost of the alternative is fresh in everyone’s memory.
The last one is the durable win. The migration is finite; the habit outlives it.
What I’d do differently
Name the decider earlier, and more bluntly. The instinct to build consensus is the right instinct for adoption and the wrong one for speed. I would start by asking sponsors to assign a single decider per contested metric, in writing, in the first two weeks, before the debates begin rather than after they stall.
Start with an inventory of how the work is actually done. Every existing process, who owns it, and how each team reconciles its numbers, before any model is touched. That is what opens up the siloed processes where only a few people know the numbers do not match. The manual reconciliations people keep privately are the best documentation available of where definitions diverge, and they surface late because nobody volunteers that their spreadsheet is load-bearing.
Publish the definition records to everyone, not just the working group. The people most likely to relitigate a number are the ones who were not in the room. A definition record nobody outside the project can find is a decision that will be made again.