The quantity of data becoming available to companies has increased enormously. This can be illustrated by the worldwide installed base of data storage capacity, which is growing at a rate of more than 15% every year. One factor is the ongoing digital transformation of businesses, which means that processes are increasingly monitored, producing lots of data that can be analyzed. Companies also have access to public or semi-public data repositories full of information useful for forecasting purposes, such as historical weather data, social media trends, and health statistics.

This huge increase in useful data is playing into the hands of ML, which requires a lot of data to work well.

The essence of Moore’s Law still holds. Gordon Moore formulated the law back in 1965, stating that the number of transistors on an integrated circuit would double every two years. Today, computers are still doubling their computing power every two years, and the cost of this power continues to go down.

All this incredible power is becoming available for ML forecasting, which is crucial because ML models require a lot more computing power than statistical forecasting. In some cases, it can take weeks to train an ML model. Therefore, any increase in computing power improves the business case for ML forecasting at the expense of statistical forecasting.

ML models can yield much richer results than classic statistical analyses such as linear regression. A striking example is the ‘value at risk from customer churn’ problem, a classic from the telecom business mentioned by McKinsey. For this problem, an ML model can provide insights along multiple dimensions, whereas the output of the regression analysis is a simple curve.

5. ML models are more flexible and versatile

ML models are more flexible and versatile than statistical analysis:

• A statistical model is always based on certain presumptions about the patterns that will be found in the data, such as the curve in the ‘value at risk from customer churn’ problem discussed above. ML, on the other hand, automatically figures out any pattern present in the data at hand, and continues to adjust the patterns as new data comes in.
• Statistical forecasting analyzes time series, one by one, independently of each other. ML, on the other hand, applies what we call cross-learning: while working a given time series, it remembers and applies the insights gathered from other time series, leading to superior results. A notable example in forecasting is the ability to cross-learn from the time series of different product families or groups.

ML has definitely made it, and it’s expected to lead the forecasting challenge in the foreseeable future. However, all of the above doesn’t mean that ML is going to magically solve every forecasting problem. There are indeed a number of prerequisites:

• It only make sense to use ML if you have access to large amounts of relevant data.
• ML will not be a viable option if you don’t have the necessary computational power.
• Expert data scientists and a profound knowledge of the business at hand are needed to develop a suitable ML model, and select the right data to properly train it.
• You might encounter some internal opposition when deploying ML because it’s much more difficult to understand how the system works and the ‘criteria’ to be taken into account