Resilience is dead, long live antifragility

It's official. AI is non-deterministic, and it will confidently produce things you never asked for and never wanted. I know, you're shocked 😆. There's some excellent work in play to manage this drift. Angie Jones pointed me to Sonar's AC/DC — the Agent Centric Development Cycle. Artem Yakimenko designed the lovely Claude watchdog, a critical post-mortem that runs on every Claude Code session.

But exactly what are your guardrails managing the drift from — the map, or the terrain?

The word drift smuggles in an assumption — that our map of the world was right at the start, and a guardrail's job is to hold the system to it. But here's the thing we all know in software engineering: the map starts changing the moment it hits the terrain, because of what we don't know, but also because the terrain is constantly changing. Technology, requirements and teams change too fast for the decisions we made at the start to stay true. It's probably the one constant we can cling to on any software project. Something is going to change.

It's the whole reason agile was invented, the whole reason we ship MVPs — so we can learn and correct as we build. And AI is no different. Our existing guardrails and controls are only those we think of at a given point in time. The biggest silent failure will be the incompleteness of our guardrails. They can't be the whole picture — there's too much complexity to hold in advance, and by the time the system is real, the ground under it has already moved.

So a control built to defend the original map is defending something we already know is incomplete. A compliance control answers one question: what do we block? A resilient one adds a second — how do we get back to where we were? Neither asks if "where we were" was right in the first place.

The alternative is to design for antifragility — controls that get better because they failed, not despite it. Controls that treat a breach not as damage to repair but as the one signal worth having: the place where the map was wrong.

That's a different design brief. A control that learns has to do five things.

1. Defence in depth 
Layer your controls. Why? Because every barrier has holes you can't see in advance. Never trust one to be complete. That's the Swiss cheese model: each layer has holes, and a failure gets through only when they line up. The failure that gets through is the signal — it shows you a hole that was always there, what kind it is, and which layer it's in. That's invaluable information for any AI system: it tells you what failure modes actually exist in yours. Layers for defence in depth — think software testing, observability, monitoring, alerting.

2. Correct the system, and revise the control 
When a control fires, don't just instruct the AI to fix the defect and close the ticket. Use the signal to revise the controls themselves as well. A second loop does root-cause analysis on the control itself — which layer should own this failure — instead of bolting the same fix onto every layer.

3. Hunt for the failure 
The control's job is not "did it pass?" It's "where is it failing?" Think Karl Popper: you never prove a system right, you only find where it's wrong, and the only thing that teaches you is the case that breaks. Train your agents and controls to go looking. Don't wait for a breach to be logged — try to cause one. Red-team it, break it in a sim, run it at the edges. Apollo's simulation team had one job: break the crew on the ground so reality couldn't. A control that only catches what wanders into it is a control still hoping it thought of everything.

4. Process the signal 
A raw signal isn't a decision. Between the breach and the response sits real work that AI is great at:

• detect the gap
• assess the context
• prioritise by impact
• Identify the best intervention (it might not be a code fix)

5. Make failure survivable 
You can only hunt for failure and learn from breaches if failure is cheap. Some of that comes from defence in depth — a hole in one layer is a near miss, not a catastrophe. The rest you build deliberately: staged rollouts; a live abort; an expiry on every control so it's revisited before it hardens into dogma.

None of this is exotic. It's what you already do when you build software — ship small, watch what breaks, change your mind. The mistake to avoid is stopping the moment we call something a "control," and reverting to the fantasy that this time we thought of everything. We didn't. We never do. So build controls that go looking for where you're wrong, and get stronger every time they find it — because the case that breaks teaches you what conformity never will.

Sources:

• Sonar — AC/DC, the Agent Centric Development Cycle (Guide → Generate → Verify → Solve).
• Artem Yakimenko — claude-watchdog, a post-session post-mortem for Claude Code.
• Map Theatrvm orbis terrarvm https://blogs.loc.gov/maps/2020/04/ortelius-a-legendary-mapmaker/