blog

Security Analysts

AI Agents

AI Security

Why Risk Managers Don't Trust AI. And Why They're Right to Ask.

Why Risk Managers Don't Trust AI. And Why They're Right to Ask.

Saul

|

|

Reading Time:

4 min

min

Table of Contents

If you spent your career in risk management, you were trained to ask one question first. How do you know?

How do you know the control works? How do you know the owner is right? How do you know the exception was approved, and by whom?

Then someone hands you an AI system and says: trust it, it is usually right.

Of course that feels wrong. It should.

Trained to distrust the black box

Risk management is the discipline of not being surprised. You build controls because people make mistakes. You demand evidence because memory fails. You keep a record because one day someone will ask what happened and why.

Most AI tools fail that test on day one. They give an answer with no source. They sound equally confident when they are right and when they are guessing. They cannot tell you who approved the action, because nobody did.

So when a risk manager hesitates, it is not fear of technology. It is the job working as designed.

Confidence is not the same as evidence

The real problem with AI in risk is not that it makes mistakes. People make mistakes too. The problem is that a fluent answer hides how much it actually knows.

A good analyst tells you how sure they are. “I’m sure.” “I think so.” “I need to check.” A risk process depends on that difference. An AI that cannot express it, and cannot show its work, removes the signal the whole process is built on.

What earns a risk manager’s trust

When we design TReAT, we start from the risk manager’s questions, not the model’s answers.

Show the evidence. Every case carries the evidence, the reasoning and the precedent behind the recommendation. Not a summary. The actual sources.

Grade the certainty. Take ownership mapping. In TReAT, an owner counts as confirmed only when two independent signals agree. For example, the owner list and the ticket history. Otherwise TReAT marks it likely, or asks you. It never guesses from a job title.

Stop at the point of consequence. Agents run the investigation, then stop and hand a person an approve or reject. Nothing acts until you say so.

Keep the record. Every step is documented and reversible. You see what ran, in which tool, under whose permissions, and who approved it.

That is not witchcraft. It is AI producing the evidence a risk manager already demands from people.

Shadow first, then graduate

There is a second habit risk managers have: you test a control before you rely on it.

TReAT works the same way. Agents run in shadow first, against your real queue, side by side with your team. You see what they would have done. You compare it to what your people did. Only then do you decide what graduates, and you set what always needs a person.

Trust is not granted on day one. It is earned case by case, on your own data.

The job does not disappear. It moves up.

The fear underneath all of this is simple. If the machine does the analysis, what is left for me?

The answer is the part that was always the job. Setting the policy. Deciding what risk is acceptable. Catching what the system gets wrong. Owning the calls that need a human name on them.

Risk managers should not relax their standards for AI. They should hold AI to exactly the standard they hold everyone else. The right system will pass.