How to Better Estimate the Cost of Forecasting Improvement
Executive Summary
- This article covers the costs of forecast inaccuracy and why it is essential to quantify the cost of forecast inaccuracy.
- This gets to creating a model for creating the cost of under and over forecasting.
Introduction to the Quantification of Forecast Inaccuracy
Most companies have not quantified the cost of forecasting inaccuracy, and this is a significant problem because it prevents them from making the necessary investments in forecast improvement. In most companies, the costs of forecasting error are quite high — and the forecast is usually far less accurate than it could be.
See our references for this article and related articles at this link.
How to Perform Forecast Testing
I have performed linear prediction analysis projects where I have turned back the clock and using the same demand history I can quite significantly improve forecast accuracy — and without doing that much work. Of course, companies that contact me know they have predicted problems — so my sample is admittedly skewed. However, even on projects where I am not there to improve forecast accuracy (for instance I am there for supply planning or production planning), I hear the same types of issues and the same dissatisfaction with the projection.
Example of Cost of Forecasting
For example, there is a list of things that all companies should be doing on forecasting, which is frequently not done. Examples of these things are listed below:
- Substitution of demand history too from expired products to new products
- Incorporation of promotion values in the order history
- Forecasting with best-fit functionality
- Top-down forecasting (where applicable)
- Developing a forecasting measurement dashboard which creates a history of different forecast input (i.e., from sales, customers, etc..) and is then is used to adjust future judgment forecasting information.
The Importance of Quantification of Forecast Inaccuracy
Companies make their decisions regarding investments in demand planning initiatives and software on the logic that forecasting accuracy has a value to a company. However, it is quite apparent from my projects that companies invest far too little in the forecasting infrastructure. This is a problem because many businesses are unwilling to invest the necessary amounts into forecasting to receive the benefits that all businesses desire.
Companies pay the price for prediction inaccuracy, but they do not get a “bill” where forecast error is outlined — unfortunately.
Few companies employ anywhere close to the number of techniques that are necessary for attaining a reasonable level of forecasting accuracy. Therefore, what is apparent is that the value of the investment that is evident to me — is not apparent to many companies. However, how would a company know the appropriate level of money to invest in forecasting? What if a company could know – at least roughly speaking what a certain degree of forecast inaccuracy cost versus another prediction error level? As the error increases, naturally, the expenses of the company increased.
These costs are incurred in multiple cost buckets.
Below you can see the major buckets I tend to place forecast inaccuracy.
Many of the costs of forecast error of implicit, not explicit. For example, of the three costs buckets, service level, and inventory is the easiest to quantify. But if only these areas are quantified, the overall cost of forecasting inaccuracy will be significantly underestimated.
Getting Into the Details of the Cost of Forecasting
Talking about the expenses is easy, and not too many people disagree that these are the typical cost areas. But this does not answer the question of how much various levels of forecast inaccuracy costs.
For that, it is necessary to develop models, each customized for different areas of the business. The following sectors have been listed based on what is traditionally the most costs of forecasting can be found in the least costs.
Category #1: Forecasting Accuracy and Inventory Costs
This is the most obvious place to look for the expenses of forecast inaccuracy. When I first approached this problem, it made sense to me to choose chose these costs that would be easy to quantify. Inventory costs reduction from higher forecast accuracy can be emulated through the use of the dynamic safety stock formula.
The Calculators for This Cost Category
- Two versions (one from Brightwork and one adapted from Lokad) of the lost sales calculator are available at this link.
Category #2: Forecasting Accuracy and Lost Sales
This requires an estimation of the cost of a lost sale and then multiplied by the number of lost sales that are attributable to forecast error.
The Calculators for This Cost of Forecasting Category
- Our reduced safety stock level costs calculator is available at this link.
Category #3: Forecasting Accuracy and Production Costs
Unfortunately (for the ease of obtaining data) there are other reasons for procurement price changes aside from last-minute requests. Another major reason for price variability – volume discounts — is just one of the other possible reasons. Short lead-time orders must be differentiated from orders that are unable to qualify for volume discounts. The trick is in distinguishing these grounds for price increases — which again will only in very rare circumstances be coded with some reason code.
Without accurate data, it can make sense to go off of estimates by people that work in the business as to the percentage that should be allocated to forecast error for upward purchasing variances.
The Calculators for This Cost of Forecasting Category
Our production-related expenses due to forecast inaccuracy calculator are available at this link.
Category #4: Forecasting Accuracy and Transportation Costs
Transportation costs are not a commonplace where companies look for savings from forecasting. If you spend time talking to directors of transportation, one of their number one complaints is that they are asked to deliver materials quickly, but are not given enough time to do so. This is because the shipping patterns are far less stable than they could be — and a big part of the reason for this is forecast inaccuracy. It should be understood that any prediction accuracy will lead to things like expedited shipments, production inefficiencies, and extra stock — but it is a matter of degree.
Companies pay more for transportation due to forecast errors as they have to expedite both inbound and outbound traffic. However, unless a different mode of transportation is used, it is often difficult to attribute a percentage of all expedite costs related to transportation which can be traced back to forecasting.
The Calculators for This Cost of Forecasting Category
Our transportation-related expenses due to the forecast inaccuracy calculator are available at this link.
Category #5: Forecasting Accuracy and Procurement Costs
Companies that provide last-minute requests often pay a price premium. The higher the forecast error, the more of this premium charge will be applied. Therefore, just as there is not a single price for an item by volume (volume discounts are of course quite common), there is often not a single price based on the lead time provided by the customer from the supplier. Lots of emphases are placed upon collaborating with suppliers — with demand collaboration being one of the most common types of cooperation.
However, poor forecast accuracy significantly undermines demand collaboration — after all — who wants to use an estimate if it lacks sufficient precision? I have witnessed many companies talk about forecast collaboration — but without once bringing up the issue of the prediction accuracy of their supply chain partners. There is a common belief that the more parties that add to a forecast, the better quality the forecast.
This is not true.
The best approach is to measure the input to the forecasting process of those that contribute and to allow in those with the highest forecast accuracy to continue to provide information and to have their contribution weighed more heavily than other inputs.
Interestingly, there is little written on the increased procurements costs, which are directly traceable to forecast accuracy. On the other hand, the price reduction, which can be obtained from volume discounts, is frequently covered in books on procurement. However, suppliers most often publish these volume prices. Short lead-time orders, on the other hand, are not published. If one thinks about negotiation, if products are required as soon as possible, it is tough to negotiate on price — it is also more difficult to check with multiple vendors. Short lead time procurement removes all of the flexibility from the purchasing department that they ordinarily have under normal supply circumstances.
Separating the Reasons for Procurement Cost Variability
Unfortunately (for the ease of obtaining data) there are other grounds for procurement price changes aside from last-minute requests. Another primary reason for price variability – volume discounts — is just one of the other possible reasons.
Apparently, short lead-time orders must be differentiated from orders that are unable to qualify for volume discounts. The trick is in distinguishing these grounds of price increases — which again will only in very rare circumstances be coded with some reason code. Without accurate data, it can make sense to go off of estimates by people that work in the business as to the percentage that should be allocated to forecast error for upward purchasing variances.
The Calculators for This Cost Category
Our forecast error related procurement premium calculator is available at this article.
Conclusion
Estimating the total cost of forecasting improvement is tricky because the cost of forecasting is hidden within the company. Secondly, most companies are set up to “get a product out the door,” and not to record the reasons why things happen. Therefore executive decision-makers often would like to have access to information that their company does not register.
An Important Related Issue
Even after the costs of forecasting have been quantified, there can be a further issue because of the following:
- The cost of forecasting is often incomplete because quantifying all of the expenses related to forecasting inaccuracy is relatively rare (it requires that the company has excellent record keeping — and reason codes for why things in the business occur.
- The cost of forecasting can end up being unbalanced between the expenses of over forecasting versus under-forecasting. This is related to a common problem regarding how lost sales are estimated versus the other “explicit” cost of forecasting within a company. This topic is explained in more detail in this article.
A Better Approach
A primary reason that companies don’t know the costs of forecasting inaccuracy is that they do not repeatedly test how different forecasts compare against one another. And this is partially related to the problem of measuring comparative forecast accuracy.
Observing ineffective forecast error measurements at so many companies, we developed the Brightwork Explorer to, in part, have a purpose-built application that can measure any forecast. The application has a very straightforward file format where your company’s data can be entered, and the forecast error calculation is exceptionally straightforward. Any forecast can be measured against the baseline statistical forecast — and then the product location combinations can be sorted to show which product locations lost or gain forecast accuracy from other forecasts.
This is the fastest and most accurate way of measuring multiple forecasts that we have seen.