Skip to content
Anil Thapa
Tradeoffs

A result from the exercise · Fictional scenario

Freshness alerts and evidence on the AI assistant

Four demands, capacity for two. One of six possible pairs: what it buys, what it assumes, what it leaves exposed, and what would change it.

A fictional quarter in a central data team at a mid-size company. Four requests arrived, each sized to fill one of the team's two delivery slots, so two are done and two wait on their interim paths. Three facts are not yet known: when annual planning starts, whether the staffing view would read from the pipeline that has run late, and whether the executive team will decide on the assistant with or without evidence. Each pair leans on some of them, and its review says which.

  1. DeferredOne margin for planningThe CFO: Finance and Sales each report gross margin, and the two disagree by enough to change a conversation. Annual planning will set targets and spending for a year on one of them. The CFO wants the margin planning uses agreed, with an accountable owner and a stated bridge to the other view.Today An analyst reconciles the two by hand each month, about two days of work, and each report carries a footnote explaining the other.
  2. ChosenFreshness alertsThe managers who run the morning meeting: Three times this quarter the morning report was stale, and the first person to notice was in the meeting that used it. They want to be told when a pipeline is late, before the report is opened, and told what to use instead.Today An analyst checks the load timestamps before eight. It works on the mornings the analyst remembers and is in, and when it fails the meeting finds out for itself.
  3. DeferredA staffing viewThe head of operations: Next week's roster is set every Monday from a spreadsheet an analyst pulls that morning, and it is committed for the week: too many people on shift costs money, too few costs service. They want a view that puts the demand forecast against capacity while the decision is still open.Today The Monday extract. When the analyst is out, the roster is set on the previous week's numbers, and it is set either way.
  4. ChosenEvidence on the AI assistantThe executive team: They have seen a demo of an assistant that answers questions from the warehouse and want to know whether to roll it out. Someone has to run the evaluation: the question set, the data the assistant may see, who in the business reviews its answers, and what evidence would justify proceeding, narrowing or stopping.Today Nothing is launched. The question comes up every month, and a vendor demo is the only evidence anyone has.

The platform learns to say when it is late. The evaluation learns what the assistant does with a margin that has two definitions, and writes it down.

What this choice buys

The morning meeting is told when a pipeline is late, by whom, and what to use instead. The executive team gets an evidenced recommendation on the assistant, and the evaluation includes late and unsupported data as cases: whether the assistant says so when a table has not landed, and what it does with a question whose definition is disputed.

When it holds

It holds when the executive team will decide on the assistant this year with or without evidence, and when the margin question can be handled inside the evaluation: scoped out of the first question set, or kept in so the finding records that the assistant cannot answer it until the business agrees. A recommendation to narrow is a result, not a failure.

What you accept

One margin for planning, deferred
Finance keeps reconciling by hand, two days a month, and planning runs on two numbers if it starts this quarter, within a limit the CFO has written down. The evaluation surfaces margin early, because it is the question people most want to ask. The honest finding is that the assistant should not answer it yet, and it is said before the first demo rather than after.
A staffing view, deferred
The roster stays on the Monday extract with a named backup, and the Mondays it ran on old numbers are counted and reviewed at quarter end. The one person who asked for something they would use every week gets nothing new this quarter. The alerts do cover their source, so the extract is at least known to be late when it is.
The chosen work, once delivered
Alert thresholds need tuning, and an evaluation set is rerun after every change to the definitions or the model, which is someone's standing work.

What changes the choice

Replace the alerts with the margin work if planning starts this quarter and the evaluation's first question set is mostly margin and profitability questions. The planning definition is then what both other items are waiting on, and detection can stay a person one more quarter.

Make your own choice

Read next