After the wrong number has been used
A corrected dashboard does not update the staffing plan made from it. Following a data error into the decisions it reached, returning the choice to its owner, and making the cost of recovery visible.
Say a weekly capacity report shows a support queue growing. The manager moves people from another queue to cover it. Later an analyst finds that a join to case history counted some open cases twice. The team repairs the model, checks the corrected totals against the source, and refreshes the dashboard.
The staffing plan is still in effect. Nobody has taken the corrected backlog back to the manager who moved the people.
That gap is easy to leave out of an incident process, because the engineering work has a visible finish: the repair is deployed, the history is rebuilt, the checks pass. The decision made from the old number lives somewhere else, in a schedule, a meeting note, or someone’s sense of what needs attention, and refreshing the dashboard reaches none of those.
It happens at national scale too. In October 2020, Public Health England found that 15,841 positive COVID-19 cases from late September and early October had been left out of the daily figures, because spreadsheets in its reporting pipeline had hit Excel’s size limits. Restoring the figures was the easy part. The contacts of those people still had to be traced, days late, and correcting the numbers did nothing for that.
In Earning trust in the numbers I describe how I run the response to a wrong number: catch it before anyone else does, announce it before anyone asks, state the blast radius, and change what let it through. This is the part I would add. Once a wrong number has been used, someone has to follow the correction into the decision it informed.
Follow the number past the dashboard
Lineage can find the models and reports downstream of a faulty source, and usage logs can show who opened them. Neither tells you what anyone decided after reading. So start with the affected metric, period and population, find the reports and regular meetings that used them, and ask each owner something specific: what did you change, approve or defer after using this report? “Did the error have any impact?” is too easy to answer with a glance at the corrected chart.
Keep a short record of the decisions you find, who owns them, and when a change would still be useful. A report being opened is evidence of possible exposure. A manager confirming it informed the staffing plan is evidence of use. They deserve different entries. You do not need every screenshot before acting: start with the decisions that carry the largest consequences or are about to become hard to reverse, and write down the limits of the search instead of letting an incomplete usage log imply nobody else was affected.
A dashboard can be corrected in one place. The decisions made from it have to be found one by one.
Send the warning while the answer is still being checked
Waiting for a complete root cause can leave the wrong figure in use through another planning cycle. Once there is credible evidence that a consequential number is unreliable, the people using it need to know what they can safely do in the meantime. For the queue, the first message might read:
The open-case totals in Monday’s capacity report include duplicate cases. The size of the overstatement is still being checked. Please pause further staffing changes based on that report and flag any already agreed. The report owner will provide a checked replacement or a progress update before tomorrow’s scheduling meeting.
It names the affected result, the uncertainty, the immediate request and the next update, and it gives the manager something to decide while the repair continues. If there is a fallback, a previously validated extract or a narrower slice that has been checked, say what its limits are. If there is no reliable answer, say that. Supplying another plausible total to keep the meeting moving repeats the failure.
Someone needs explicit responsibility for this communication, and in a large incident it should not be the engineer investigating the join. Google’s SRE workbook, published in 2018, in its chapter on incident response, separates the operational work from a communications role that updates stakeholders and handles their questions. I would make the same split for a data incident, and give the communications role one more job: following up the decisions already made.
Give the decision back to its owner
A corrected number can change the case for a decision. It does not give the data team authority to reverse it. The manager might keep the staffing change anyway, because of absences, the age of the unresolved cases, or a release coming next week, and automatically undoing it would swap one incomplete account for another.
Hand back the original result, the checked replacement, and what changes in the interpretation. If the recommendation depended on a threshold the corrected number no longer crosses, say so plainly. Then ask the owner to keep, change or stop the action, and record the reason. An acknowledgment that the corrected report arrived is not the same event as reviewing the staffing plan. This is also why I want an analyst’s recommendation to name a decision and an owner when it is first made: the record gives a later correction somewhere to go.
Some decisions will be too late to reverse. Then the review is about what can still be recovered and what changes next time. Calling the data accurate again does not give back the hours already spent on the wrong queue.
Keep the corrected history visible
A backfill changes what the warehouse says about the past. The record of what people knew when they acted still has to be explainable. Keep enough of the affected report to show its period, filters, definition and version, mark it as superseded, and link it to the correction, under the access rules that already apply. Otherwise, months later, the only surviving report carries the repaired figure and the original staffing choice looks inexplicable.
Send the notice through the channels that carried the error. A banner on the dashboard helps the next visitor, but the people reading an exported deck need the correction attached to that deck. The same goes for a scheduled AI briefing: repairing its source does not withdraw yesterday’s paragraph, and a freshly generated answer can be right while the old recommendation is still in use.
Record the repair and the decision review separately
Technical recovery can finish while the follow-up on decisions is still open, so I track them separately. For each consequential decision found, the follow-up shows one of four states:
| State | Evidence to keep |
|---|---|
| Awaiting review | Named decision owner, next contact, and deadline before further action |
| Changed | What the owner changed after reviewing the correction, and who will carry it out |
| Kept | The owner’s reason for retaining the decision on the corrected evidence |
| No longer reversible | Consequences reviewed, with any remaining action assigned or explicitly declined |
A few lines in the existing tracker are enough. If the owner has not responded and the consequences are material, escalate through the person accountable for the decision. Silence is weak evidence that the follow-up is complete.
“Data restored” should tell you what is safe to use now. It should not hide what still needs a decision.
The correction takes longer than the backfill
Following decisions costs time a pipeline repair would not: an analyst goes back to a meeting, a manager reconsiders a plan, and other work waits. That cost belongs in the incident record, or the platform looks cheaper to run than the people using it experience.
It also costs some comfort. A quiet correction is easier than a message to the people who acted on the wrong number, and the team may have to explain how the error got past its checks while the recipient explains a choice made from it. Keeping the account on the evidence available at each point makes that conversation useful. It does not make it comfortable.
Keep the response proportionate. A rounding error that could not change a decision does not deserve the same search as duplicated cases in a capacity report, and the number of reversed decisions is not a success measure: a careful review may leave the original choice standing. What matters is that the owner received the correction, considered it, and recorded what follows.
The manager may move the people back, or keep them where they are. Either way, the reason belongs beside the decision, where the wrong number was used.