How Control Gets Lost in Working Systems
What AI systems reveal about the loss of corrective control
The Problem Starts When We Stop Being Able to Intervene
I’ve been struck by how many systems today feel like they’re still functioning, but no longer fully under anyone’s control. Whether it’s political discourse online, platform governance, or economic systems responding to shocks, the pattern feels the same: decisions happen faster than people can meaningfully intervene, and by the time problems surface, the ability to correct them has already weakened.
This essay is an attempt to name that pattern more precisely.
Where this pattern shows up first
Imagine a social media platform that changes how posts are ordered in people’s feeds.
After the change, posts that provoke strong reactions, anger, outrage, ridicule, start appearing more often. Arguments show up earlier. Comment sections get louder. Threads spiral faster, and users begin to notice that the tone of what they see has shifted.
Some users respond by reporting posts they believe violate the platform’s rules, whether that’s harassment, misinformation, or hate speech. Each report sends the post to a moderation team that has to decide whether it stays up, gets labeled, or comes down.
At the same time, users whose posts are removed or restricted submit appeals, asking the platform to reverse those decisions. Each appeal requires a human to review it, which means cases pile up, decisions slow down, and some borderline content stays up longer simply because no one has reached it yet. The moderation team works through the backlog as quickly as it can.
Meanwhile, the ranking system keeps promoting posts based on engagement, because that’s exactly how it was designed to work.
Nothing dramatic has happened. The system is operating as intended. Engagement is up, and from the outside the platform looks healthy. What has changed is that the people responsible for managing the consequences are now constantly playing catch up.
That shift is where misalignment starts.
What I mean by AI systems
When I say AI systems, I’m not talking about abstract models or futuristic intelligence. I’m talking about software that makes decisions, rankings, or recommendations that people are expected to act on, often at a scale where human judgment still exists but cannot realistically be applied to every case.
These systems do not replace people outright. They change how work gets done. They decide what rises to the top, what gets flagged, what gets ignored, and what demands attention first. Over time, people adjust their behavior around them.
That adjustment is where drift begins.
What changes before anything breaks
In the platform example, nothing breaks. The ranking system continues to optimize for engagement. The moderation team continues reviewing content. Reports and appeals still move through the system.
What changes is how easy it is to intervene.
As queues grow and time pressure increases, fewer cases receive careful attention. Edge cases take longer to resolve. People rely more on rules and thresholds because there is no time to slow things down and argue through judgment calls. When something feels off, it is often easier to move on than to escalate.
The system still works. In some ways, it even looks more efficient than before.
It is just harder to slow down, question, or correct.
Why this matters beyond AI
You see the same pattern anywhere AI systems are embedded in real work.
Imagine a fraud detection system that flags transactions for review. Early on, analysts spend time with borderline cases and override the system when something does not look right. As volume grows, thresholds get adjusted so the work stays manageable. Some transactions are approved automatically, others rejected automatically, and appeals begin to stack up. Reviews turn into triage.
Nothing is broken. Fraud is still being caught. By most measures, the system is doing its job.
What has changed is how easy it is for a human to step in and correct it when something feels off.
When intervention slips away
What we should really be concerned with is whether a system is still steerable. Whether people can slow it down when something feels off. Whether they can question it, raise alerts when it misses the mark, or flag something incorrectly without worrying about penalties, delays, or being ignored. Whether feedback can actually change what the system does next week, rather than just documenting what it did last week.
It is when those capacities quietly weaken that something more fundamental is happening. While systems are still functional, the space for intervention narrows, and control becomes more symbolic than real.
That shift matters well beyond AI.
You see it in political discourse that polarizes faster than anyone can moderate, in platforms that optimize engagement while governance teams react from behind, in trade and policy systems that respond to shocks at speeds that leave deliberation behind. The common thread is not failure, but loss of leverage.
The real risk of modern systems is that they continue operating while fewer and fewer people are able to meaningfully intervene when direction starts to drift.
Once intervention becomes impractical, control hardens into rules no one actively chose but everyone has to live with. And by then, the system is no longer just misaligned. It is no longer fully ours.