The tools I use

I keep a list of what I actually use, and I keep the old ones rather than editing in place. The 2025 version is in the archive. Reading them side by side is more honest than either one alone, because it shows what I got wrong about where things were heading.

One thing worth flagging before the list. The 2025 version was written by someone whose day was mostly spent in a SQL client, and it reads that way. This one isn’t, and a good third of what follows isn’t data tooling at all. That shift is the part I would have found least interesting five years ago and find most interesting now.

Also: most of this is stack I have run somewhere, not stack I run in one place. Anyone who tells you they use exactly one warehouse has either been somewhere very disciplined or very briefly.

Warehouses

I’ve run production on Redshift, Snowflake and Databricks, and spent enough time in BigQuery to have opinions I would defend in a review. The honest summary is that the choice matters considerably less than the volume of argument it generates.

Redshift rewards a team that will think about distribution and sort keys and punishes one that won’t; the performance is there but you have to go and get it. Snowflake removes most of that thinking in exchange for money, which is usually the right trade right up until the bill becomes a board topic. Separating storage from compute makes cost legible, and legible cost attracts scrutiny that a fixed cluster never had to survive. That is a feature, but nobody warns you it is coming. Databricks earns its complexity when the work is genuinely mixed, notebooks and ML and SQL over the same tables, and is overhead when it isn’t. BigQuery is the fastest of the four to start on and the easiest to leave running by accident.

What I would tell someone choosing: pick for the shape of your team and the governance you can realistically enforce, not for the benchmark. All four hold. The expensive mistake is almost never the engine, it is running several of them at once without meaning to.

Open table formats, Iceberg mostly, are the omission that dates the 2025 list hardest. The interesting thing about them isn’t the format, it is that they turn “which warehouse” from an architectural commitment into something closer to a procurement decision. That is a slower change than the launch posts suggested and a bigger one.

Moving and shaping data

Airflow orchestrates. Still batch, still the right tool for it, and the when not to use it section of an older piece still covers the cases where I wouldn’t. Dagster is the one I would evaluate seriously on a greenfield build, for asset lineage rather than task lineage. I have not moved anything existing to it, and “orchestrator migration” is a poor answer to most questions.

dbt for transformation, with model ownership sitting with the analysts rather than engineering. That was an organizational decision far more than a tooling one, and it cost central control to buy throughput.

Fivetran and Airbyte for connectors. The rule I hold to is that nobody should hand-write an extractor for a SaaS API that a vendor already maintains, and nobody should pay connector prices for the handful of sources that carry the actual business. Managed for the long tail, owned for the core. The line between them is where the real engineering went.

Python, with Polars and DuckDB where a warehouse round-trip isn’t worth it. DuckDB in particular has quietly absorbed a category of job that used to require provisioning something.

What people actually see

Power BI, Tableau, Metabase, Looker. I’ve shipped against all four. Power BI is the one most organizations end up on for reasons that are commercial rather than technical, and it is better than its reputation among engineers. Tableau still wins on exploratory analysis and loses on governed distribution at scale. Metabase is the correct answer far more often than its price suggests. Looker’s modelling layer is the genuinely differentiated part, and the thing every competitor has since tried to copy.

The 2025 list had Power BI and Tableau on it. The first draft of this one had no BI tool at all, which was an oversight rather than a decision, and a revealing one: the further you get from the dashboard, the easier it becomes to forget that it is the only part of the platform most of the company will ever touch. Dashboards that nobody acts on are a design failure long before they are a tooling one.

Catalog, quality and governance

OpenMetadata for catalog and lineage; DataHub solves the same shape of problem and I would not argue hard with someone who picked it. This category moved from nice-to-have to load-bearing faster than I expected, and the reason is in the last section.

dbt tests first, then something heavier. Most teams reach for a data quality platform before they have exhausted what schema and uniqueness tests in the transformation layer already give them for free. Great Expectations, Soda and the observability vendors are worth their cost once you have real volume and real consequences, and are a way of avoiding the boring work before that. The hard part was never detection anyway, it was what happens in the hour after detection.

Day to day

VS Code with Claude Code. This is the one that would have been unrecognizable on the 2025 list, where the entire AI section was a single line about Copilot generating boilerplate. It isn’t a completion tool. It works across repos, reads context, and does real multi-step work, and it changed how I spend a working day more than anything else on either list.

Git, on GitHub or GitLab. I have run teams on both and the difference has never once been the thing that mattered. What matters is whether review is real.

DBeaver, still, with an LLM integration for query tuning. Several waves of new tooling later and the SQL client I open is the same one it was in 2021.

Excel for analysis. I’m not going to defend this and I’m not going to stop.

Running the function

This is the section the 2025 list had no equivalent of, and it is now most of the week.

Notion for documentation and, more importantly, for decision records. The single highest-leverage document I keep is a running log of decisions: what was decided, when, what was known at the time, and what was traded away. It costs ten minutes a decision and it settles arguments eighteen months later that would otherwise be relitigated from memory by people who weren’t there. Confluence does this too. The tool is irrelevant; the habit is not.

Jira, Linear, ClickUp. Any tracker does this job and teams waste real energy arguing about which. What matters is the column structure: there should be a stakeholder-review state sitting before closed, because the work isn’t finished when the code merges. That single column changes behaviour more than a migration between trackers ever has.

Miro and Excalidraw for architecture sketches and, just as often, org design. A data function’s shape is a diagram problem in the same way a pipeline is, and the shape is usually the thing that is wrong.

Slack, plus Loom for anything that would otherwise be a meeting. A three-minute recording of a walkthrough is worth more than the thirty-minute call it replaces, and it stays searchable.

Slides, unglamorously. A meaningful share of platform work is won or lost in a deck that has to survive an executive audience with no context and no patience. Treating that deck as a data product rather than an afterthought is the single highest-return skill I’ve picked up, and it is not taught anywhere.

Automation and glue

Supabase, n8n, an LLM API, and Slack for the small internal systems that don’t justify a real application but do justify existing. Most of the useful internal tooling I’ve built in the last year is some combination of those four. The failure mode is obvious and worth naming: glue built this way has no owner, no tests and no runbook, and the moment one of these becomes load-bearing it needs to graduate into something that does.

What actually changed

Comparing this to the 2025 list, four things stand out.

The core didn’t move. Warehouse, orchestrator, transformation layer, SQL client, editor, version control. Same shape, mostly the same products. Five years of churn in the category and the things I touch every day barely shifted. Worth remembering the next time a launch feels urgent.

The AI line went from a footnote to the center. In 2025 I filed Copilot under pro tips and described it as a boilerplate generator. That was a reasonable read of what existed and it was wrong within a year. The thing I now use for hours a day, in a terminal, against real repositories, didn’t have a category when I wrote that list.

A compliance tool became a runtime dependency. Catalog tooling isn’t on the 2025 list at all, and had it been, I’d have filed it near Great Expectations under things you install because someone asks you to.

It is now closer to the warehouse in importance than to the checklist. Not because governance got more fashionable, but because the catalog stopped being something people browse and started being something a system reads. Context that used to live in an analyst’s head now has to be written down somewhere machine-readable or it may as well not exist: which table is current, which is the deprecated copy, which column means what it says. That is the reason agents against a real warehouse behave so differently from agents in a demo.

Half the list stopped being data tooling. The 2025 list is twenty-odd products and every one of them is something you install. This one has a section on decision logs and slide decks, and if I’m honest about where the leverage sits, that section outperforms the warehouse section most weeks. That isn’t a seniority platitude. It is that the technical choices above are now mostly solved problems with well-understood tradeoffs, and the organizational ones still aren’t.

---

In two years this list will read the way the 2025 one does now: mostly right about the plumbing, embarrassingly wrong about whatever turns out to matter. I’d rather keep the evidence than quietly edit it.