Mistaking Stress Testing for Forecasting

Author: Ron Dembo

The Toronto Star recently reported that “The Canada Mortgage and Housing Corporation (CMHC), which insures high-ratio residential mortgages, said Thursday that Canadian housing prices could plunge by almost 50 per cent if we have a double-dip ‘W-shaped’ COVID-19 recession and governments don’t provide enough financial support to survive the pandemic.”

Hot to jump on the numbers was Phil Soper, president and CEO of realty giant Royal Lepage, who immediately said, “To have forecasts that are not only so wrong, but are done in a way where you can’t even see where they could come up with it? That damages their credibility as a forecaster”.

But CMHC wasn’t making a forecast. The scenario was in fact part of a stress test to its liquidity in the event of a crisis, something every sensible financial institution has been doing since the Global Financial Crisis in 2008.

“Stress tests by their nature are going to be more pessimistic. We need to find out whether we’ve got enough liquidity to withstand default risks, and we do” have enough, said Nadine Leblanc, CMHC’s chief risk officer.

The point is that the scenario was fundamentally not a prediction of what they thought would happen. It was a test of extremes. And their case wasn’t helped by the fact that the Toronto Star ran another of the CMHC’s stress-test scenarios as its headline in a way that made it sounds exactly like a prediction: “New CMHC report says Canadian housing market could see a 14% plunge”.

This kind of confusion around forecasting is plaguing the risk management industry. There is no choice but to try and describe the possible space in which the future lies — it is an essential function of preparing for the unknown — but it should be clear that the appropriate way to forecast is not to make a singular, deterministic prediction about what you think will happen in the future. Instead, just as CMHC was doing, forecasts must act as a way to generate many possible science- and data-based futures with the best, most-trusted information available. In doing so, they enable us to test whether we are prepared for a range of different outcomes.

The risk thinker admits that there is no “solution” in the face of radical uncertainty — no single answer to the multiple possibilities of tomorrow; there is only a strategy with one or more hedges to protect us from the magnitude of extreme downsides — or, to exploit the benefits of the upside. The form of a solution using risk thinking is an action and a hedging regime. And coming up with that regime requires a range of forecasts about the future with which to stress test a portfolio, not just a single prediction about what one thinks will be most likely to happen.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store

Riskthinking.AI develops new standards, forward-looking data, ratings and algorithms for the latest science-based measurement of climate financial risk.