The AI risk matrix: scoring likelihood and impact

A risk matrix turns a vague worry ("this model could be biased") into a number you can rank, escalate, and defend. It multiplies how likely a failure is by how bad it would be, then a detectability factor decides whether your monitoring earns you a lower score. That number feeds the risk tier, which decides how much governance the system gets.

Two AI systems can carry the same headline concern and deserve completely different treatment. A customer-service chatbot that occasionally gives a wrong answer and a resume-screening tool that occasionally discriminates are both "the model gets it wrong sometimes." One is a low-risk annoyance. The other is a critical-risk legal exposure. The RAI risk matrix is how we tell them apart without arguing from vibes, and it's a different tool from the RACI matrix that assigns who's accountable. This one scores the risk itself.

What the matrix computes

The core of our risk scoring matrix is one line:

Risk Score = Likelihood × Impact × Detectability Factor

Likelihood and impact are each rated 1 to 5. Multiply them and you get a base score from 1 to 25. That base score is the classic 5×5 matrix. The detectability factor, a multiplier between 0.5 and 1.5, then adjusts it based on whether you'd catch the problem before it does damage. The final number lands in one of five bands, and each band carries a required response.

The 5×5 grid

Here's the base matrix, likelihood down the side, impact across the top:

The bands: 20-25 is critical, 10-19 high, 5-9 medium, 3-4 low, 1-2 minimal. Critical means unacceptable, may trigger stop-the-line authority, needs executive sign-off to proceed. Minimal means document it and carry on.

The shape of the grid tells you something the arithmetic hides. A catastrophic-impact risk is never below medium, even when it's rare, because the bottom-right column never drops under 5. That's deliberate. Low-probability, high-severity failures are exactly the ones organizations talk themselves out of taking seriously, and the grid refuses to let a "5" impact score all the way down to minimal.

Score likelihood and impact honestly

The matrix is only as good as the two numbers you feed it, and both are judgments with anchors, not guesses.

For likelihood, we tie each level to evidence. Level 5, almost certain, is for failures where similar incidents happen frequently and no controls are in place, like an LLM producing factually wrong information without any RAG grounding. Level 1, rare, requires defense-in-depth controls and multiple simultaneous failures to occur. If you can't point to why a risk sits at its level, you haven't assessed it yet.

For impact, score across several dimensions (financial, operational, reputational, regulatory, and safety or rights) and take the highest applicable level. This matters because AI harms don't distribute evenly. A model can be a rounding error financially and still be catastrophic on rights, like a system causing mass civil-rights violations. Use the worst dimension, not the average, or you'll dilute the harm that actually decides the tier.

The detectability factor people forget

The multiplier is what separates this from a naive 5×5. Two risks with identical likelihood and impact are not equally dangerous if one is caught instantly and the other surfaces only after the damage is done.

The factor runs from 0.5, immediately detectable with real-time automatic response, up to 1.5, undetectable until harm has already occurred. A worked pair from the framework shows the swing. Take a risk at likelihood 4, impact 3, base score 12. If it's difficult to detect (1.25×), the score becomes 15 and it's high risk. Add monitoring that catches it early so it's easily detectable (0.75×), and the same risk drops to 9, medium. Same failure, same stakes, different score, because your detection changed the exposure.

This is the part of the matrix that rewards engineering investment. You can't always lower the likelihood or the impact of a failure, but you can very often lower its detectability factor by instrumenting for it, and the matrix gives you credit for that.

From score to tier

The score isn't the destination. It feeds risk tiering, and the bands map onto EU AI Act categories so one assessment serves both internal governance and regulatory classification. Critical scores line up with prohibited or high-risk (Annex III) systems. High scores are high-risk. Medium maps to limited risk with its transparency obligations. Low and minimal land in minimal risk with no mandatory requirements.

That mapping is why the matrix and the deeper impact assessment work as a pair. The matrix is the fast, comparable score that ranks a portfolio of systems and decides which ones get escalated. The impact assessment is the full algorithmic, privacy, and human-rights evaluation you run on the systems the matrix flags as high or critical. Score first to triage, assess deeply where the score demands it.

Where the matrix earns its keep

Run three real cases through it and the value shows.

A resume-screening tool: likelihood 4 (historical bias in screening is common without mitigation), impact 4 (employment discrimination carries serious legal and individual harm), base 16. Bias emerges gradually and needs specialized analysis to spot, so detectability is 1.25×. Final score 20, critical. It gets RAI Council review, comprehensive bias testing, mandatory human review of decisions, and high-risk compliance.

A general customer-service chatbot: likelihood 3, impact 2 (inconvenience, easily corrected, no financial hit), base 6. Customer feedback and conversation logs make it easily detectable, 0.75×. Final score 4.5, low. Standard guardrails, AI disclosure, an escalation path to humans.

A medical-diagnosis support tool for rare diseases: likelihood 3, impact 5 (an incorrect suggestion could contribute to patient harm), base 15. Physician review provides moderate detectability, 1.0×. Final score 15, high, even though the chatbot and the diagnosis tool have the same likelihood. Impact did the separating.

Recommendations:

• Score every AI system on likelihood and impact before it ships, and anchor each rating to real evidence rather than instinct.
• Take the highest impact dimension, never the average. The worst harm is the one that sets the tier.
• Apply the detectability factor deliberately, and treat lowering it through monitoring as a legitimate way to reduce risk.
• Use the score to triage, then run a full impact assessment on anything scoring high or critical.
• Rescore on material change. Model drift, a new data source, or a regulatory shift can move a system across a band.