How to Understand an Incorrect Forecasting Article by CIO on Nike

Executive Summary

• There are problems with the interpretation of forecasting concepts that are exemplified by Nike’s understanding of forecasting, as explained by CIO magazine.
• In a CIO article, Nike and AMR make several incorrect statements around forecasting harkening back to common forecasting myths.

Introduction to the Problems with Interpretation of Forecasting

One of the widespread problems in decision-making, in general, is that of the misinterpretation of events that came before. What is true of studying broader history (that there can be many interpretations) is also true in the study of previous business experiments. It is primarily publications and institutions that have a luxury or writing history.

See our references for this article and related articles at this link.

However, as institutions and media outlets have particular internal incentives, they often chose to tell stories that are substantially different from reality.

Interpreting the Narrative

I have noticed a storyline in several articles in the past several years that relate to the effectiveness of quantitative methods vs consensus or and collaborative methods. As an individual who found that overly mathematical methods were not as effective as proposed (the opposite of the conventional wisdom ten years ago), it is interesting that quantitative methods are now generally out of favor, in favor of consensus-based approaches.

However, as I will describe in this analysis of an article in CIO magazine, the contemporary interpretation I am seeing is entirely one-sided towards collaboration.

It leaves the challenging political battles of what group will have the most significant say in the forecast number unaddressed. It also misrepresents the actual (not hypothetical) complexity of math that was implemented at companies.

The CIO Article on Nike

CIO magazine decided to write an article about Nike which focused on their i2 implementation. Still, in doing so, they explain the philosophy of forecasting of several of the highest-ranking members of Nike’s supply chain organization. The article was surprising because it shows that several things that these leaders think about forecasting are not true. This leads to the discussion of this article.

The Quotes from the Nike CIO Article

I will begin by going to the direct quotes from this article. This quotation below is about a question asked of Roland Wolfram, Nike’s Vice President of Global Operations and Technology regarding the Nike i2 implementation.

“It drives Wolfram crazy that while the rest of the world knows his company for its swooshbuckling marketing and its association with the world’s most famous athletes, the IT world thinks of Nike as the company that screwed up its supply chain? Specifically, the i2 demand-planning engine that, in 2000, spat out orders for thousands more Air Garnett sneakers than the market had appetite for and called for thousands fewer Air Jordans than were needed. But there was a lesson too for people who do, in fact, follow “this sort of thing,” specifically CIOs. The lesson of Nike’s failure and subsequent rebound lies in the fact that it had a business plan that was widely understood and accepted at every level of the company. Given that, and the resiliency it afforded the company, in the end the i2 failure turned out to be, indeed, just a “speed bump.””

So why did the demand planning engine spit out an incorrect forecast? Was it purely a problem with i2’s software or were their specific design decision taken that can be addressed that other companies can be made aware of that can prevent this in the future?

Systems Implementation Question

Also, why was the previous system not run in parallel before the production system was cut-over? Nike must bear responsibility for how they cut over new systems to production. No vendor can make you do this incorrectly as it is under the client’s control.

“If there was a strategic failure in Nike’s supply chain project, it was that Nike had bought in to software designed to crystal ball demand. Throwing a bunch of historical sales numbers into a program and waiting for a magic number to emerge from the algorithm? The basic concept behind demand-planning software? It doesn’t work well anywhere, and in this case didn’t even support Nike’s business model. Nike depends upon tightly controlling the athletic footwear supply chain and getting retailers to commit to orders far in advance. There’s not much room for a crystal ball in that scenario.”

The beginning of this quotation describes quantitative forecasting, albeit in derogatory terms. Typically a “magic” number is not expected, but a proposed forecast. I have been on a lot of supply chain projects, and I don’t recall anyone ever using the term “magic number” or “crystal ball.” The assumptions are that the system will provide its best guess as to the forecast and that this best guess will have a percentage of error.

Statistical Forecasting Not a Fit for Nike?

The excerpt goes on to say that what is called quantitative or statistical forecasting “does not fit with the Nike business process.” Firstly, something very similar to the process described above is used at most companies globally. The forecasting process does fits with Nike’s business process because, as with other companies, there is a lead-time between when a product is demanded (in the store) and when it can be supplied.

Secondly, Nike sells a large number of shoes. Therefore it has the law of large numbers on its size, improving the potential forecast accuracy.

Nike must forecast its demand because it cannot instantaneously produce the shoe in the store. And one of the forecasts that it uses would be a statistical forecast (but not necessarily the only type of forecasting).

The statement that demand planning does not work anywhere is strange and is almost hard to comment on except to say that for a concept that does not work anywhere, it seems to be implemented pretty much everywhere.

Using Sales Orders Instead of Forecasting?

The next excerpt proposes that instead of forecasting, companies should use orders and invoices in SAP ERP.

“Indeed, Nike confirms that it stopped using i2’s demand planner for its short- and medium-range sneaker planning (it’s still used for Nike’s small but growing apparel business) in the spring of 2001, moving those functions into its SAP ERP system, which is grounded more in orders and invoices than in predictive algorithms. This allows us to simplify some of our integration requirements,” – says Nike CIO Gordon Steele

This also makes no sense. It is a false dichotomy.

How Orders Serve as the Basis for Forecasts

Orders (although it is unclear why invoices are needed as they are financial documents) are always used in forecasting. They are called the demand history. However, a “predictive algorithm” better known as a forecasting method (seasonal, trend, linear, etc…) is used on this demand history to produce the forecast, which may then be manually adjusted based upon domain expertise. This statement by Gordon Steele demonstrates a great deal of confusion as to what the forecasting processes are and what it isn’t.

Furthermore, the idea that one would remove forecasting from the forecasting engine (i2 Demand Planner) to “reduce integration issues” is absolute nonsense. Nike is a large company with significant resources to place in demand planning. I2 may or may not be the right solution for them, but some forecasting application undoubtedly is. Or statistical forecasts can be created entirely effectively with spreadsheets.

Within SAP ERP, only an insufficient set of functionality for forecasting is available, which would be unable to meet Nike’s extensive forecasting requirements.

Wolfram Repeats a Myth

Wolfram goes on to repeat an often-repeated myth about forecasting relating to its “art.”

“Wolfram says Nike’s demand-planning strategy was and continues to be a mixture of art and technology. Nike sells too many products (120,000) in too many cycles (four per year) to do things by intuition alone. “We’ve tuned our system so we do our runs against [historical models], and then people look at it to make sure it makes sense.”

The art that Wolfram describes is also known as domain expertise. The planning with domain expertise applies what the planners think the forecast will be, and then adjusts the forecast. Nike is a major apparel and footwear design, marketing, and distribution company. (they manufacture no product). It would be strange if Nike used only manual methods to perform forecasting. Thus it is a peculiar thing for Wolfram to point out that Nike uses a computerized forecasting system. The “run” that Wolfram refers to is most likely a forecasting run. This calls into question his earlier statements about not performing a forecast. The second part of what Wolfram describes is part of the standard forecasting process.

This also describes the forecasting process at most companies I have consulted with. Wolfram implies that this standard process that is used at every company I have ever worked with is somehow unique to Nike.

How AMR Misunderstands Forecasting

In the next excerpt from the same article, a Vice President from an analyst firm offers his views that crystal ball forecasting is no longer fashionable.

“There’s been a change in the technology for demand planning,” says AMR Research Vice President Bill Swanton, who declined to address the Nike case specifically. “In the late ’90s, companies said all we need is the data and we can plan everything perfectly. Today, companies are trying to do consensus planning rather than demand planning.” That means moving away from the crystal ball and toward sharing information up and down the supply chain with customers, retailers, distributors and manufacturers. “If you can share information faster and more accurately among a lot of people, you will see trends a lot sooner, and that’s where the true value of supply chain projects are, Swanton says.”

Bill Swanston should declare who said that if provided with data, “everything could be planned perfectly.”

Possibly irresponsible and low integrity consultants said this, or software companies proposed it, but its unlikely many forecasting experts believed this. It’s doubtful that either consultants or software companies ever suggested this in the terms described by Bill Swanton. Furthermore, the use of collaborative forecasting does not preclude the use of statistical forecasting and vice versa. The fact is companies need both types of forecasting categories. And finally, there is plenty of evidence at this point that consensus methods are even more challenging to implement than statistical methods and have a higher bias.

Bill Swanston’s Straw Man Argument

Bill Swanton is not accurately representing what was said about forecasting. This is called creating a straw man. That is you present an argument as ludicrous so you can contradict it.

“A straw man argument is an informal fallacy based on misrepresentation of an opponent’s position. To “attack a straw man” is to create the illusion of having refuted a proposition by substituting a superficially similar proposition (the “straw man”), and refuting it, without ever having actually refuted the original position.” – Wikipedia

It is not considered an honest argumentative approach. Secondly, he is presenting an all or nothing approach to performing forecasting, which can be roughly translated into “statistical forecasting bad, consensus forecasting good.” It may make a good bumper sticker, but it is in no way true. In reality, statistical and consensus methods work together to result in a final forecast.

What is Make to Stock…….Again?

In the next excerpt misdescribes what make-to-stock is.

“Nike’s supply chain project is supposed to drive the manufacturing cycle for a sneaker down from nine months to six. Cutting out that three months would match Nike’s manufacturing cycle to its retailers’ ordering schedule? they order 90 percent of their sneakers six months in advance of delivery. This means Nike could begin manufacturing its sneakers to order rather than three months in advance and then hoping they can sell them. Converting the supply chain from make-to-sell to make-to-order is the dream of any company desirous of gaining competitive advantage through its supply chain. Dell has done it, famously, with PCs; Nike wants to do it just as famously with sneakers.”Nike hasn’t gotten there yet. And its business case relies on a nearly 30-year-old model that some analysts and retailers grumble is out of touch with the reality of today’s market. But it’s a business case Nike’s leaders believe in. This is how CIOs keep their jobs when a project goes off track and it’s how they keep getting funding to keep it going.”

This entire paragraph is incorrect. The paragraph starts by declaring that Nike would like to move to make to order environment, but then at the end admits that Nike does not have one yet. The article neglects the cost differences between a make to stock vs makes to order shoe. Nike has designed its entire manufacturing strategy around inexpensive and controllable overseas factories that have extended lead times to pay the minimum in labor and environmental costs for shoes.

Shortening Lead Times?

Shortening lead times the way that is described in the paragraph above would mean moving production closer to consumption, which would significantly reduce Nike’s enormous margins on its products. It has been estimated that seamstresses that work in overseas factories receive roughly 9 cents for a product that retails for $30 to $35 at a retailer in the US or Europe.

Not declaring the profit margin on their products may lead one to come to different conclusions regarding how Nike’s supply chain should be planned. As for the Dell reference, this is also incorrect. Dell does not purely make to order; it also sells pre-configured computers through normal retail channels. However, even on the part of the business where the customer orders online, Dell does not make to order; it is assembled to order.

Secondly, and Nike should know this, Dell makes a different product than Nike and has a different labor model than Nike. Most of the difficult work in a computer is performed by the component manufacturers not by Dell (a trend which is happening in the automotive industry as well as is described in this article). Dell manufacturing consists of placing hard drives, memory, and motherboards into computer cases. In this environment where the complexity is external, assemble to order makes sense. However, Nike shoes have individual styling for which materials must be custom ordered, are mutually exclusive and which have limited duration production runs. You could not design a more make to stock environment than Nike’s.

The Make to Order Definition

The definition of make to order is the following:

“…a production approach where once a confirmed order for products is received, products are built. BTO is the oldest style of order fulfillment and is the most appropriate approach used for highly customized or low-volume products. This approach is considered good for highly configured products, e.g. automobiles (Holweg and Pil, 2004; Parry and Graves, 2008), computer servers, or for products where holding inventories is very expensive, e.g. aircraft.” – Wikipedia

Commingling Data Acquisition with Forecasting Methods

The next excerpt goes on to blend data acquisition with forecasting methods.

“Nike also is behind its rivals in direct point-of-sale (POS) integration with retailers, says Shanley. Supply chain experts agree that actual data from stores, rather than software algorithms, are the best predictors of demand. But Nike’s SAP system cannot yet accept POS data, though the company says it’s working on it.”

The statement, as with Wolfram’s on “orders and invoices” confuses the demand history with the forecasting method. These are two entirely different things. Both are needed, but one does not replace the other. Also, which supply chain experts “say this.” I have supply chain expertise, and I first don’t agree with the statement. Still, secondly, I don’t even agree that the statement is even internally consistent, or that the dichotomy it is attempting to create makes any sense. POS data is preferable over shipment data; however, that does not follow that POS acquired demand history can be used exclusively to create a forecast without a software forecast method. How could it, unless it was a constant forecast.

Forecasting Myths

This overall presentation on by this article is highly misleading, and Nike’s position is just not believable. i2 Demand Planner was a reasonably capable forecasting engine with a multidimensional cube as a backend and average interface. It could not possibly have been responsible for all Nike’s problems unless Nike pushed the vendor to implement it incorrectly. Secondly, the knowledge level demonstrated about forecasting in comments by Nike leadership indicates that it would be challenging for any demand planning applications team to have success at Nike.

This article repeats several of the most common myths about forecasting, that CIO magazine published in previous articles.

<h3 “>Forecasting Myths from Tim Berry

More myths are listed below and are from Tim Berry, head of Palo Alto forecasting:

• Myth 1: It’s About Accurate Forecasting: Not really. We’re all just human, so we don’t predict the future all that well. What’s much more critical is structuring a forecast so we can track results—make it match your accounting input—and then following up. Structure it for tracking, then track it, and then, after review, make the management decisions. The significant loss is how many forecasts never produce the second step. No wonder people disrespect forecasting. Without the following up, there’s very little value. Think of a forecast as a route on an electronic map, and the process of comparing actual results with the original forecast as the GPS technology that places your exact location onto that map. The map alone isn’t nearly as useful as the combination.
• Myth 2: It’s for Experts: Again, not really. In the real world, forecasting is a matter of well educated guessing in rows and columns on a spreadsheet. Real people, the ones who run the business, think about what they can realistically expect. I paid my dues with more than three years as a professional market researcher and more than 20 years in business planning. I’ve written about sophisticated techniques, including smoothing and weighted moving averages and econometrics, in some books that are old now. Technical analysis isn’t what makes forecasting work. Common sense and frequent review make forecasting work.
• Myth 3: You Can Manage Without it: Managing a company without sales forecasting—the forecast, the actual results, and the management that follows—is about as smart as driving a car without a steering wheel, or maybe I should say with your windshield covered in black paint. Of course, there are always exceptions, but most of the small business right now is dealing with sales less than planned. And planned sales usually drive planned spending. So this is not a good thing

(more on forecasting myths is covered in this article)

Overall Conclusion

The people here interviewed by CIO either do not understand or are not being forthcome about why their demand planning implementation ran into problems. This is most likely for political reasons; because Gordon and Wolfram cannot possibly know as little about forecasting as they seem to in this article. It was not because they were sold a crystal ball, or because magic algorithms did not work (if Nike believed this, should they hire new people to evaluate software), but because they made an incorrect decision regarding the most essential of a planning decision. That is, at what level is the appropriate level to model.

The CIO author has no background in what he is writing about, and the article was not checked with anyone who did. They are also taking Nike’s rather ridiculous statements at face value and not providing any objective analysis. It takes about a minute of reading the article for a person experienced in forecasting to figure out something is wrong with Nike’s statements.

Foundational Concepts in Forecasting

An excellent article can both inform about a particular situation and clarify foundational concepts. This article does the opposite, it both misinforms about the situation at Nike, and it also confuses the reader on the foundational concepts. However, the article can be read as an example of the period cycle that the forecasting market goes through in general. This quote from George Plossl’s book written in 1985 describes this cycle.

“Some years prior to World War II, many companies began to recognize the potential benefits of preparing forecasts in a formal manner. They set up a separate group responsible for preparing the data, outlined the procedures for approval of the forecast by those in the organization who were vitally concerned and frequently restricted distribution to those eligible to receive such confidential information. In many cases, large sums of money were spent to develop forecasts using statistical techniques, market research or other sophisticated methods. During this period, the basic assumption seemed to be that forecasting problems could be solved if only enough money, effort and intelligence were put into making them. This could well be termed the rose-colored-glasses era of forecasting. When perfect forecasts were not forthcoming, the inevitable period of disheartened with their failures with the explanation that all would have been well if only there had been a good forecast. As a result of this reaction, many companies ceased all organized attempts to forecast and returned to intuitive guesses, stung by the failures md high costs of the methods that had let them down. This could be called the backlash era. Most companies have now recovered completely from this reaction. Naive attitudes toward forecasting are rare in industry; only a very few companies till believe they should expect more accurate forecasts and could get them if only they could find the right technique.” – George Plossl

Indeed, if this cycle has been documented at least twice, it must be a repeating cycle. However, rather than being reactionary, Nike would have been better off if they had segmented their database into forecastable and unforecastable products, and applied different methods to each. Articles like this help lead to a lot of bad decisions and incorrect solutions being selected. This article describes why companies so frequently choose the wrong software for their forecast needs.