How RMSE is Calculated for Forecast Error Measurement

Executive Summary

• RMSE is a universally accepted forecast error measurement.
• RMSE is generally low in effectiveness in providing feedback to improve the forecast.

Introduction

RMSE, or Root Mean Square Error, is listed as one of the significant forecast error measurements. However, its complexity in calculation and difficulty in explanation make it a distant third to the far more common MAD and MAPE forecast error calculations. Nonproportionality makes RMSE nonintuitive. You will learn about how RMSE is calculated and the limitations of RMSE for improving forecast accuracy, which is similar to but more extensive than other standard forecast error measurements.

Our References

If you want to see our references for this article and related Brightwork articles, visit this link.

How RMSE is Calculated

How RMSE is calculated is one of the most common questions we get.

RMSE is calculated as follows.

1. Take the absolute forecast minus the actual for each period that is being measured.
2. Square the result
3. Obtain the square root of the previous result.

The formula is..

= Square Root(Squared(F – A))

Similar to MAD, RMSE uses squared values. Because RMSE is squared, the error is not proportional — meaning that more significant errors become much more prominent as they are squared.

Any error measurement that is not proportional makes it intuitively more difficult to understand than a proportionate measure. How is a non-forecasting-focused person supposed to interpret nonproportional forecast errors? Our brain is designed to understand proportionality.

For example, the Richter Scale is a logarithmic scale, which allows a relatively small number of values to represent large magnitudes in difference. However, it also means humans cannot calculate the difference between different earthquake values.

This video explains in detail how RMSE is calculated.

The Same Problem With All the Standard Forecast Error Measurements

Due to its lack of proportionality and high complexity in the calculation, RMSE was never a forecast error method before developing our own. However, in addition to this lack of proportionality, RMSE shares the same problems with all standard forecast error measurements.

These include the following:

1. They are difficult to weigh. Any forecast error aggregated beyond the product location combination must be weighed to make any sense.
2. They make comparisons between different forecasting methods overly complicated.
3. They lack the context of the volume of the demand history or the price of the forecasted product, meaning that the forecast errors must be provided with context through another formula.
4. They are difficult to explain, making them less intuitive than a different approach.

Why Do the Standard Forecast Error Calculations Make Forecast Improvement So Complicated and Difficult?

It is important to understand forecasting error, but the problem is that the standard forecast error calculation methods do not provide this good understanding. In part, they don't let tell companies that forecast how to make improvements. If the standard forecast measurement calculations did, it would be far more straightforward and companies would have a far easier time performing forecast error measurement calculation.

What the Forecast Error Calculation and System Should Be Able to Do

One would be able to for example:

1. Measure forecast error
2. Compare forecast error (For all the forecasts at the company)
3. To sort the product location combinations based on which product locations lost or gained forecast accuracy from other forecasts. 
4. To be able to measure any forecast against the baseline statistical forecast.
5. To weigh the forecast error (so progress for the overall product database can be tracked)

Getting to a Better Forecast Error Measurement Capability

A primary reason these things can not be accomplished with the standard forecast error measurements is that they are unnecessarily complicated, and forecasting applications that companies buy are focused on generating forecasts, not on measuring forecast error outside of one product location combination at a time. After observing ineffective and non-comparative forecast error measurements at so many companies, we developed, in part, a purpose-built forecast error application called the Brightwork Explorer to meet these requirements.

Few companies will ever use our Brightwork Explorer or have us use it for them. However, the lessons from the approach followed in requirements development for forecast error measurement are important for anyone who wants to improve forecast accuracy.