There is a cycle in mid-market and higher ed data programs that I want to name out loud, because I think most of us are living inside it without having stepped back to recognize the shape.
The cycle goes like this. A team selects and installs a data stack. Maybe a warehouse, a transformation layer, a BI tool, some assortment of governance and modeling and visualization platforms. They configure it for the org's specific needs, build pipelines, train people, put it into production. Two years pass.
Then key people leave. The architect who designed the stack moves on. The senior analyst who built the canonical models takes a job somewhere else. The director who championed the program retires or gets reassigned. New people are hired into those seats. They look at the inherited stack with the eyes of fresh hires who weren't in the rooms where the original decisions were made, and they conclude, more or less universally, that the existing stack is the problem.
Sometimes they're right. Often they're not. It almost doesn't matter. What matters is that the new team has every incentive to propose replacement, and almost no incentive to defend the inherited choice. Leadership has no institutional memory of the previous selection, they remember complaints about the current stack, not the equivalent complaints about the stack two cycles ago. The vendors are happy to encourage the conversation, because every replacement is a new sale. So the cycle restarts. New tools get selected. The migration eats eighteen months. The new stack goes live. Two years pass.
I have seen this cycle run three times at organizations I've been close to. Each iteration costs seven figures, eats institutional attention for years, and ends roughly where the previous iteration ended. Different vendor names. Same set of unanswered questions. Same complaints from the same constituents about the same gaps. The data is not measurably more trustworthy than it was three stacks ago. The decisions made on top of the data are not measurably better.
This is a structural problem dressed up as a maturity problem.
The reason it's structural is that nobody in the loop is incentivized to interrupt the cycle.
Outgoing teams don't defend the stack on the way out, because they're already mentally checked into their next role. The defense would be unrewarded, and depending on the politics of the departure, sometimes counterproductive. So they leave the documentation thin, the institutional knowledge in their heads, and a backlog of complaints that they never had time to address.
Incoming teams don't inherit the defense. They inherit the complaints. The complaints are real, every data stack has them, but without context for why the previous team made the choices they did, the complaints look like evidence of bad architecture rather than evidence of normal trade-offs. Replacement starts to feel like the responsible move. It's also, conveniently, the move that justifies the new team's salary, gets them visibility with leadership, and aligns them with vendors who want to sell.
Leadership doesn't remember. Most cabinets don't have the technical depth to evaluate a "we should replace the stack" pitch on its merits, and they don't have the historical memory to recognize that they greenlit an identical pitch four years ago for the previous stack. The pitch is well-rehearsed. The current stack has accumulated complaints. The new team has credibility. Approval is the path of least resistance.
The vendors are running the same playbook everyone else is. Every two-year migration cycle is a sale for somebody. The vendors with the most aggressive sales motions are the ones whose pitches happen to land just as the previous stack is generating its peak frustration, which is often by design. The data tooling market is structured to expect this churn, not to interrupt it.
Nobody is acting in bad faith. The outgoing team is moving on. The incoming team is doing what they were hired to do. Leadership is making the decision they're equipped to make. The vendors are selling. Everyone is rational at the local level, and the system as a whole produces a result that nobody would have endorsed if they could see it from a higher altitude.
The result is that the same project gets executed every two years with different vendor names, and the underlying problems persist because they were never about the tools.
Here's what I think is actually going on.
The data tooling ecosystem is built around an assumption that humans will hold continuity that humans don't actually hold. The stack assumes that the people who built it will be there to maintain the institutional knowledge it represents, what the canonical models mean, why the joins work the way they do, what the edge cases are, which dashboards are actually being used and by whom, what the unspoken conventions are that keep the whole thing coherent. When those people leave, the knowledge leaves with them. The stack remains, but stripped of the context that made it valuable.
The replacement instinct is, I think, mostly a misdirected response to that knowledge loss. The new team isn't really concluding that the technology is bad. They're concluding that they don't understand it, can't reconstruct why it was built this way, and would rather start fresh than archaeologize a system whose original logic is no longer documented or available. From their perspective, this is reasonable. From the institution's perspective, it's a recurring seven-figure tax on staff turnover.
If the diagnosis is right, the answer isn't different tooling. It's a continuity layer that survives the staff turnover. Something that captures the why of the stack, not just the what. Something that makes a new team's first three months a process of inheriting institutional knowledge rather than a process of forming impressions about it.
This is, I think, where AI actually has a role to play in data programs, not the role most of the AI-in-data marketing is pitching.
The popular AI-in-data narrative is "AI will replace your analysts" or "natural language interfaces will eliminate the need for SQL." Both are mostly wrong, and both miss the actual opportunity. Analysts aren't the bottleneck. SQL isn't the bottleneck. Continuity is the bottleneck.
The interesting application of AI in this space, to me, is as institutional memory. A layer that knows the history of the stack, the rationale behind decisions, the conversations that led to specific choices, the unwritten conventions, the half-finished projects, the past complaints and how they were addressed, the dashboards nobody uses anymore and why they were built originally. The kind of knowledge that, today, lives in the heads of three or four people and walks out the door with them.
This is genuinely hard, technically. It's also fundamentally different from what most data tooling vendors are building. Most data tooling is optimized for throughput, how many pipelines, how many dashboards, how many users. The continuity problem is optimized for retention of context, which is a different shape entirely. Most vendors aren't going to build it because it doesn't sell the way pipelines and dashboards sell. It's a slow-burn value proposition that doesn't generate the metrics that drive the data tooling market.
I'll be honest: I'm working on this problem. Not because I particularly want to be in the build-tools business, but because I've watched the cycle run enough times that the urge to interrupt it has become hard to ignore. The data ecosystem has a half-dozen vendors selling tools that all solve approximately the same surface-level problem, and approximately none of them solving the underlying continuity problem that makes the surface-level tools churn.
I'm not going to pitch what I'm building here. This isn't that kind of post. But I'll say this much: if you're a data leader who's about to greenlight your second or third stack replacement, the question I'd ask first is whether your people problem is solved before your tooling problem gets re-solved. Because the new tools are going to inherit the same continuity gap that broke the old tools, and you're going to be having this conversation again in 2028.
The data landscape isn't insane because there are too many tools, although there are. It's insane because the tools are being asked to solve a problem that isn't actually a tooling problem, and the failure to solve it is being interpreted as evidence that we need different tools.
We don't need different tools. We need the institutional knowledge that makes the tools work to outlast the people who build it.
That's an unsexy answer. It's not going to win awards at a data summit. But it's the only answer I've seen that actually addresses the cycle, and it's where I'd be putting my own attention if I were running a data program right now.
Which, in a manner of speaking, I am.
— Chris