Count the decisions, not the dashboards

Ask a data professional what they have done and the answer is a list of things that exist: dashboards, pipelines, models, a migration, a platform. Ask what changed because of them and the answer gets slower. Ask an interviewer’s version of the same question, “tell me about a time your work changed a decision,” and the most common answer is a description of a dashboard.

That is not a failure of the person. It is what the record shows. The visible record of a data career is deliverables, because deliverables are what gets counted: tickets closed, dashboards shipped, lines of SQL, the number of sources in the warehouse. None of it says whether a decision was different for the work having been done. The platform has the same problem. It is a cost until it changes a decision, and most of what is counted about it is cost.

So I keep a different record, and I ask the people I work with to keep one.

The record a platform keeps

The organization’s version of the question is the one I would put to any platform: which decisions did a number inform last quarter, and what happened next? The answer is rarely in the warehouse. It is in a planning document, a budget line, a staffing roster, a feature that shipped or did not.

The individual’s version is the same table with a name in it: the decisions you were part of, what you brought to them, and what came of it. This is the piece about keeping that evidence.

Four columns

The log is a table, and it has four columns.

Column	What goes in it	The question it answers later
What I recommended	The decision as the owner framed it, the recommendation, the alternative I would also have accepted, and the condition that would change my view	Did I bring a decision, or a finding?
What was decided	The choice made, by whom, and on what, including when it was not mine and when it was made before my work arrived	Was the evidence in the room in time?
What happened	At the review date I set when I wrote the row: what the next period showed, as far as it can be known	Was the condition met or not?
What I would change	About the recommendation, the evidence, the timing, or the way it was carried into the room	What does the next row look like?

The first column is the one most people skip, and it is the one that makes the rest possible. A recommendation written as decision, alternative and condition can be checked later; a finding with caveats cannot. It is the shape I ask for in Recommendations that go nowhere, and the reason to ask for it is partly this log. The condition is what the third column tests.

The second column records something people are reluctant to write: decisions that were made before the analysis arrived, or made by someone the analyst never met, or made on grounds that had nothing to do with the evidence. Write them down anyway; what they add up to is read below, by the manager.

An illustrative row, kept short the way a real one is:

Recommended: replace the activation definition from the next quarter and publish a restated history; alternative, keep the old definition and add the new one beside it under its own name; condition, if the restated history cannot be rebuilt for the last four quarters, publish the break instead. Decided: the alternative, by the product lead. Happened: both measures in use three months on, and the old one retired at the next planning cycle without a meeting. Change: I should have asked who owned the target before proposing the change; the target moved a month later with nobody named.

Ten minutes to write, most of it spent on the first column.

Why the record has to be written at the time

The reason the log is written when the recommendation is made, and not reconstructed afterward, is that memory revises. After the outcome is known, the recommendation remembers itself as more confident, or more hedged, than it was.

Michael Mauboussin, speaking to investors in a 2012 interview with Farnam Street, recommends a decision journal: whenever you make a consequential decision, write down what you expect to happen and why, and how you feel about it, so that the record exists before the outcome is known. He attributes the practice to Daniel Kahneman and gives its purpose as preventing hindsight bias, the tendency to look back on a decision and tilt the account of it in your own favor. Applying it to a data professional’s recommendations is my extension.

The data version of hindsight bias is specific. Once the quarter’s numbers are in, the analysis that pointed the right way remembers itself as the one you were sure of, and the one that pointed the wrong way remembers itself as the one you had doubts about. The log written at the time says what you actually knew, which is the only version worth learning from.

What the log does to the work

Keeping it changes the work before it changes the record.

You start asking for the decision before building. A request that cannot produce a first column, no decision, no owner, no date, is a request for a chart, and it goes in the queue behind the ones that can. That is the test Built, launched, ignored applies to a dashboard request, applied by one person to one week.

You notice when you are brought in late. The second column fills with decisions already made, and after a few of those the useful move is to ask to be in the room earlier, with the rows as the evidence for asking. What to do once you are there is Data in the boardroom.

You get better at the condition. The first conditions people write are vague, “if it doesn’t work, revisit.” After a few rows where the third column could not say whether the condition was met, the conditions get specific: a number, a date, a person who would know.

And you find out what you are actually for. Over a year, the rows cluster. Some people’s logs are full of definitions settled and numbers made trustworthy; some are full of recommendations that moved budget; some are full of decisions they were handed and carried through. All three are careers in data, and knowing which one is yours is what the next role should be chosen on.

Reading it at review

For the person leading the team, the other reader of this site, the log is better evidence than the ticket count, and it is read differently.

The hit rate is the least useful thing in it. A log where every recommendation was accepted and every condition was met is a log of safe recommendations, or an edited one. What I read for is the quality of the first column, whether the condition was testable, whether the alternative was real, and the honesty of the fourth column. Someone who writes “I should have asked who owned the target” is developing judgment faster than someone whose fourth column is empty.

I also read it for the organization. Three analysts whose second columns say “decided before my work arrived” are not three analysts with a timing problem. They are one organization with a decision process that commissions analysis after the fact, and that is the manager’s finding to act on, not theirs. The structural version of this, developing the judgment and then making room for it, is in When every decision still needs you.

The log is also the honest input to the conversation about what someone does next. The questions in What would this work change? are the questions a single row has to answer, and a year of rows says which of them the person has learned to ask without being prompted.

What it costs

Exposure. The log is a record of being wrong, kept in your own hand. Some rows will say the recommendation was taken and the condition failed. A finding with caveats never has to admit that, which is most of why findings are more comfortable to send and why they go nowhere.

Time. Ten minutes a decision, and the discipline to set the review date and keep it. The third column is the one that lapses, because by the review date the decision is old news and the next one is urgent.

Humility. Most rows, kept honestly, say that no decision changed. The work was received, the meeting was polite, and the thing that was going to happen happened. That is the real baseline of a data career, and seeing it written down is uncomfortable in a way that makes the next recommendation better.

The temptation to log only the wins. A log that starts after a good quarter and skips the bad one is a portfolio, and portfolios are for selling, not learning. The rows that were wrong are the ones that change the fourth column.

The dashboards will still get built, and some of them will matter. But the record that says which ones mattered is not the list of dashboards. It is the list of decisions, with your name in the rows where you changed one.