The pandemic made your business forecasts sick. What’s the cure?

Shifts in global and local economies due to the Covid-19 pandemic and the discrepant, ever-changing national and local responses have rendered the rudimentary analytic models of many organizations useless, or worse—detrimental to the business.

“Everything has changed so we have had to change our models,” says Frank Bernhard, private equity firm chief data officer-in-residence. “The risk profiles of our portfolio companies have changed, their supply chains have changed, and customer experience and buying habits have changed. So the reliance on historical data has to change as well.” Bernhard recommends that companies everywhere need to take an inventory of their analytic models, and adapt. 

Similarly, Freddie Mac’s chief economist, Sam Khater noted that as the economy began to deteriorate so did their models. To compensate, his team started tracking many non-traditional, high-frequency data on a daily and weekly basis. This data provides a more real-time view of the evolving economy. He says, “We did this because many of the traditional monthly indicators were out of date before they were even published.” 

But why exactly are traditional models and analytic techniques broken, and how can they be fixed?

A tale of two analytic modeling techniques

Andrew Duguay, chief economist with the industry insights and predictive analytics company, Prevedere, says the difference is in trend-based versus driver-based models. Trend based models rely on the company’s own historical data, such as sales or production data, and sometimes macro-level industry data. These formulae expect trends to continue on a similar path and at a similar pace. Driver-based models, on the other hand, rely more on leading indicators of performance or business activity. They incorporate external data about situations or observations that highly correlate to and presage one’s own business outcomes. This is like the difference between driving the business by looking in the rearview mirror versus peering ahead at various nearby and distant traffic patterns while noting the weather and even listening to what other drivers and pedestrians are thinking. 

Although forecasts crunching historical data may work fine in normal times when markets are relatively stable, in anomalous times like today these trend-based models falter. In times of turmoil, leading indicators prevail. However, converting to driver-based analytics does come with risks. During a normal, stable market, developing driver-based analytics can be overkill, resulting in over-engineered models that require expensive data sourcing, cleansing and integration (i.e., “data ops”), and difficult to maintain algorithms.

Still, some organizations have decided not to wait for turmoil, but rather have advanced to relying on driver-based models, regardless. “When we worked with oil and gas customers on measuring the cost of inefficiency throughout the supply chain, we didn’t learn much doing trend-based analysis,” said Satyam Priyadarshy, managing director, technology fellow and chief data scientist with Halliburton. “You have to find the root cause of something to determine actionable insights.” 

Even traditional trailing indicators like travel sentiment on social media, which normally indicate actual travel plans, can become leading indicators of individuals getting back to voyaging via planes, trains and automobiles. 

Relying on data from prior recessions

Economists, business analysts and data scientists are finding that historical data from the last several recessions is of little use today. In those instances the bubbles bursting predictably spread from one sector to another. It was easy to discern and establish sets of leading and lagging indicators. Duguay says that “this all gets tossed out the window” because this crisis didn't start as an economic crisis; it started as a pandemic. 

Moreover, it’s unlikely that we will float back naturally to business as normal the way we have following other economic crises. This time there’s a certain volatility driven by a number of complex and regrettable factors including: 

• The current U.S. administration’s propensity to shut-down data collection activities, make data unavailable, cease coordinating with global pandemic tracking and economic organizations, and dismiss the validity of official data sources
• The pace of policy decisions and reversals regarding stimulus payments and tax relief, and the enactment of government emergency powers 
• Differing and ever-developing lockdown durations and reopening criteria across thousands of U.S. municipalities and hundreds of countries 
• Fluctuating and uncertain vaccine funding and approval processes, and 
• The anticipation of U.S. presidential and congressional races themselves.

On the other hand, certain organizations find they have to rely more-so on historical data than before the pandemic. Companies like the connected car solution provider, Wejo, experienced a drop in data volumes due to a reduction in automobile traffic. As Jim Johnson, head of data governance and analytics, explains, “Ironically the lack of current data has increased the value of historic data. Existing methods of data collection and post-Covid traffic prediction are flawed given current reduced data volumes.”

A third pseudo-analytic alternative

Indeed, even the best AI-driven analytic models based on leading indicators can break as the data they rely on become non-indicative or even a set of trailing indicators. When this happens, business leaders must start considering other option-based analytic techniques such as scenario planning. Scenario planning enables an organization to define predetermined responses to certain possible conditions of varying likelihood, thereby enabling it to be more agile, if not completely anticipatory. 

Unfortunately, scenario planning is a skill that few organizations practice with any discipline other than at the highest strategic levels. Rarely do companies apply this technique to shorter-range or more granular business activities. 

What not to do when your models are broken

Yasith Fernando, senior manager, data & analytics, digital strategy & transformation at EY says CIOs he speaks with have realized that their analytics models are so focused on just their own data that they can no longer be trusted. Many company executives, for lack of a better approach, have simply reverted to using their intuition. 

Unfortunately, it takes an economic crisis to get business and IT executives to realize that new analytics techniques and data sources are required. Other executives remain paralyzed. 

“A lot of executives just can’t seem to wrap their heads around what a leading indicator is,” laments Pawan Murthy, head of marketing with Aumni, an automated investment analytics platform. This speaks to another global epidemic, one throughout the business world: a deficiency of data literacy.

This article originally appeared in Forbes.