Why your AI agent makes catastrophic decisions (and how to stop it)

An AI agent managing customer support answers 10,000 tickets per month. 99.7% of responses are perfect. The remaining 0.3% promise unauthorized refunds, share sensitive data with the wrong users, or escalate delicate situations with responses that make the crisis worse. Out of 10,000 tickets, that's 30 disasters per month. Every month.

This scenario isn't hypothetical. It's the reality of any autonomous system that optimizes for average performance without considering the distribution of consequences.

The fundamental problem: optimizing for the average

The language models powering AI agents are trained to maximize the probability of the most appropriate response in the typical case. They work extraordinarily well in most situations. But "most" isn't "all," and in high-impact decisions the difference between 99% and 100% isn't a percentage point — it's the boundary between efficiency and catastrophe.

The problem is structural, not technical. It can't be solved with better prompts, larger models, or more training data. It's solved by changing the agent's decision-making framework.

Where prompt engineering fails

The first reaction of any team that discovers this problem is: "let's add more specific instructions to the prompt." But prompt engineering has an architectural limit: the prompt defines *what* the agent should do, not *how to evaluate the consequences* of what it does.

A prompt can say "don't promise refunds over 50 euros." But it can't teach the agent to recognize that a specific situation has catastrophic asymmetry: the gain from a quick response is minimal, but the cost of a wrong response is enormous.

This is the domain of Asymmetry in the DAMM framework — the second pillar that evaluates the ratio between potential gain and potential loss of every decision.

Asymmetry as the first filter

In DAMM, before an AI agent executes a decision, it must answer one question: *"If this decision goes wrong, is the damage proportional to the benefit I'm trying to achieve?"*

When asymmetry is unfavorable — meaning when the potential loss is much larger than the potential gain — the framework activates a protection mechanism. It doesn't block the agent: it prevents it from proceeding without verification.

In practice, this means a DAMM-protected agent:

• Automatically identifies high-asymmetry decisions before executing them
• Calculates a Health Score (0-100) that quantifies the overall risk level
• Triggers escalation when the score is below threshold, delegating the decision to a human operator or review process

A concrete example: the financial agent

An AI agent managing automatic portfolio rebalancing analyzes markets every 15 minutes. In most cases, its decisions are correct and generate returns. But in a volatile market, the same agent might sell losing positions to comply with a stop-loss rule, realizing permanent losses on assets that would have recovered within days.

Without DAMM, the agent sees only: "the price dropped 15%, the rule says sell." With DAMM, the agent evaluates: - Delimitation: what are the boundaries of my authority? Can I liquidate positions above X euros without approval? - Asymmetry: selling now crystallizes a certain loss. Not selling exposes to a potentially larger loss but also to recovery. What's the ratio? - Margin: is there sufficient liquidity to absorb a continued decline without being forced to sell? - Minimum Move: instead of selling everything, can I reduce the position by 20% and reassess tomorrow?

The difference isn't in the model's quality. It's in the decision-making structure.

Practical implementation

Integrating DAMM into an AI agent doesn't require rewriting the agent. It requires adding an evaluation layer between intention and action:

1. **The agent formulates the action** it wants to take
2. **The DAMM layer analyzes the action** through the 4 pillars
3. **If the Health Score is above threshold**, the action is executed with a complete audit trail
4. **If it's below threshold**, the action is blocked and flagged for review

This approach transforms every agent decision from an opaque action into a documented, verifiable process — exactly what the EU AI Act will require for high-risk systems by 2026.

The question to ask today

If your AI agent makes 1,000 decisions per day, how many of those have an asymmetry you've never measured? How many have the potential to cause damage disproportionate to the benefit they generate?

The DAMM framework doesn't slow down your agents. It protects them from decisions they should never have made on their own.