Risk management is at the core of good project management.
Or as Tim Lister says “Risk management is project management for adults”.
The standard approach is to use a risk matrix to classify project risks based on their probability and impact, then give each one a ‘risk score’ by multiplying the two numbers. Then you rank the risks by score and address the top ones first.
Risk matrices have been widely praised and adopted as simple but effective approaches to risk management.
And as many risk matrix practitioners and advocates have pointed out, constructing, using, and socializing risk matrices within an organization requires no special expertise in quantitative risk assessment methods or data analysis.
So in terms of “understanding and managing risk”, it seems to work.
Unfortunately it doesn’t.
It is unfit for purpose. It actually may even be doing more harm than good.
Sh!t in, sh!t out
Things go wrong from the very start. Namely with the probability estimates you put into your risk matrix.
Human beings are not very good with non-linear risks. Our instincts evolved to help us deal with immediate physical dangers in our environment. So we can tell whether an oncoming car is likely to hit us, for example.
But the more complex the risk, and the more factors are involved, the less helpful our gut instinct is. And project management risks are some of the most complex risks in the world.
It’s extremely difficult to say how likely it is that an information breach or ransomware incident will actually occur. So most people rely on gut instinct, on the grounds that it’s better than nothing.
But if you ask someone to gauge the likelihood of a project risk — even someone with very deep knowledge — they will be hard pressed to give you an accurate answer. For instance, what’s the likelihood of a key supplier or system integrator going bust? Is it low, medium or high? Why do you say that? How do you know?
It’s a similar story with impact. In theory, it’s easier to get a reasonably good idea of financial impact by thinking about management time, developer hours, lost sales and reputation damage. But people rarely bother, because the risk matrix is only asking for a simple assessment anyway.
Enter the matrix
So the information you put into your risk matrix is hopelessly inaccurate. But then the matrix itself makes things even worse.
Because these matrices have such a low resolution, they make very different risks look alike. For example, in a 3x3 matrix (low, medium, high on both axes), risks with 67% probability and 99% probability are both “high”.
Clearly, you’d want to address the 99% risk first. But when you come to rank your risks, you have no way of knowing which one is worse based on the matrix.
What’s more, the matrix gives equal weight to probability and impact, so an incident with 1% probability and $500,000 impact has the same priority as one with 0.2% probability and $2,500,000 impact.
In fact, in some fairly common situations (mathematically speaking, when probability and impact are negatively correlated), you’d actually be better off choosing the matrix square at random.
Yes, you read that right — pin your matrix to the wall, throw a dart for each risk and you’ve got a better chance of picking up the most important ones.
The risk matrix can be, quite literally, worse than useless.
Dangerous illusion of control
The problem with the risk matrix is that it feels scientific. It promises a quick, simple solution to a wicked problem without taking up loads of time, or asking you to do too many hard computations.
Before, you had no idea about risks. But now, you’ve put them in neat little boxes and given them solid-sounding scores. You “understand and manage your risks”, or so it seems.
But all you’ve really done is creating a story that gives you a dangerous illusion of control.
Not only is there no proof that risk matrices work, there’s actually proof of the opposite.
Using the matrix actively hampers firms’ efforts to deal with risk, absorbing time, money and effort for no benefit at all.
In a nutshell: Don't rely on your risk matrix to understand and manage your risk.
Posted on Monday, June 15, 2020 by Henrico Dolfing