Disregarding Gartner’s Deeper Technology Insights and Predictions

Table of Contents: Select a Link to be Taken to That Section

Executive Summary

Gartner has an abysmal history of technology prediction.
Part of the reason for this accuracy is not Gartner’s primary concern.

Introduction (Skip if You Watched the Video)

In their reports, Gartner will sometimes make technology projections or try to describe the technology. At Brightwork Research & Analysis, we have analyzed these projections and have found quite a few of these statements to be unreliable. Gartner shows no evidence of being predictive; there is no reason to pay attention to their predictions. Furthermore, Gartner’s technology prediction opinions have been parodied in numerous articles on the Internet by experts in that particular category of technology. You will learn about how Gartner makes predictions and how they are designed to be used in marketing collateral by software vendors and consulting firms.

Our References for This Article

If you want to see our references for this article and other related Brightwork articles, see this link.

Notice of Lack of Financial Bias: You are reading one of the only independent sources on Gartner. If you look at the information software vendors or consulting firms provide about Gartner, it is exclusively about using Gartner to help them sell software or consulting services. None of these sources care that Gartner is a faux research entity that makes up its findings and has massive financial conflicts. The IT industry is generally petrified of Gartner and only publishes complementary information about them. The article below is very different.

First, it is published by a research entity, not an unreliable software vendor or consulting firm that has no idea what research is.
Second, no one paid for this article to be written, and it is not pretending to inform you while being rigged to sell you software or consulting services as a vendor or consulting firm that shares their ranking in some Gartner report. Unlike nearly every other article you will find from Google on this topic, it has had no input from any company's marketing or sales department.

Is Gartner Focused on Forecast Accuracy?

Many of the technology predictions I have read seem to have been designed to create a splash rather than stand the test of time. These are called “nuggets” and are explicitly designed to be picked up and repeated by vendors and consulting firms. The nugget is always positive, and Gartner designs the nugget to be repeatable, not to be accurate.

Gartner’s Error in Forecasting Windows Phone Market Share

One that sticks out as ludicrous was Gartner’s prediction that the Windows Phone would become the second most popular smartphone platform between 2012 and 2015 and account for 19.5% market share. 2015 is not yet upon us, but the Windows Phone has only a 3.2% market share.

Gartner is well known to take a large amount of money from Microsoft and routinely publishes pro-Mozilla and anti-Linux articles. Android predictions. Microsoft has a tremendous amount of money to spend with Gartner. Open-source projects have very little, and at Gartner, money talks.

Gartner’s SaaS Prediction

Another example of Gartner’s inaccuracy in long-term technology forecasting is taken from their prediction on SaaS.

Up until now, the unique nature of the software market has meant that buyers had very little negotiating power after the initial purchase of a software license,’ Gartner Vice President William Snyder said in a research note. ‘We expect those dynamics to change considerably over the next 5 to 10 years, giving CIOs and software procurement officers more bargaining power while potentially reducing software vendor profit margins.

Gartner also predicts that a fourth of all new business software will be delivered by software as a service by 2011.

Software costs have not significantly declined, as predicted by Gartner. Gartner also overshot their second prediction that a fourth of all software would be delivered as SaaS by 2011. As of 2012, software provided by SaaS represents only four percent of total software sales. Granted, SaaS software is less expensive than on-premises software, so the number of seats served is undoubtedly higher than its sales percentage. But it isn’t easy to see how the delivery of SaaS as a percentage of all enterprise software would be anywhere close to the twenty-five percent value of software delivery estimated by Gartner.

Was this a bad analysis on the part of Gartner? Based on the enormous advantages that SaaS has in terms of cost and time to deploy and the efficiency of central administration by the vendor, I believe Gartner’s projections were reasonable at the time. However, many people who predicted faster growth failed to consider issues ranging from security concerns to the lower customizability of SaaS solutions and the perception of integration issues that prevented SaaS from taking off as initially expected.

Other IT Analysts and Prediction

Prediction is a problem not only for Gartner but also for most other IT analysts.

There are several examples of this, and the largest analyst firms predicted positive things for the following:

i2 Technologies TradeMatrix (which was not a product)
The development of online marketplaces generally.
That MDM applications were going to be significant factors in software projects.
They predicted that “ERP was dead” (in the late 1990s) and that ERP companies would have to move into new products or be left out in the cold.
SAP Solution Manager (which wasted a tremendous amount of the company’s time and is probably the worst content management system ever broadly sold, although it has now been repurposed as is discussed in this post)
SAP’s acquisition of Business Objects (even at this late date) is challenging to see one beneficial thing brought by this merger). Business Objects was not integrated into the SAP Suite — BTW, the SAP Business Objects Explorer, while having some valuable capabilities, is so buggy that most companies have stopped using it. Business Objects’ customer service has also declined while the price has increased. Furthermore, business objects are no longer seen as innovative applications, a common problem for acquisitions.
SAP PLM (which was never an actual “product” as much as it was merely life-cycle functionality distributed in different applications.)
SAP Netweaver (never actually a product, but rather a marketing construct, as can be read in this post)*

This list of analyst misses was put together very quickly from memory. There are many more, and we will add to the list as time passes and more of their errors come to us.

The Similarity Between Analysts and Political Lobbying

There are several problems with software analyst firms. First, they have a bias towards the larger software firms. Industry insiders know this, but the payment from vendors to analysts is mentioned nowhere on their websites and is not discussed by them generally. Analyst firms solicit vendors in a similar way that lobbyists explain the benefits of political lobbying. Lobbyists cannot openly declare that their money buys changes to legislation (Although it does. In fact, the return on investment is estimated to be 22,000% {if you win and the legislation goes the lobbyists’ way}, which is higher than any other competitive investment.) Therefore, lobbyists state that their money only buys “access.” However, as with analyst firms, the people who pay to lobby know what they are doing and what they are getting for their money. However, analyst firms receive money from both directions, both from the suppliers (vendors) and the consumers, that is, companies that buy enterprise software.

The Ridiculous Excuse Offered by Analyst Firms for Clear Conflicts of Interest

If pressed on the issue, Gartner representatives will say they have an “ombudsmen” that mediates disputes with vendors and prevents their influence. Therefore, they have no bias. However, that does not show in the positions that they take. Gartner does an excellent job of appearing unbiased, and they employ good writers. The effect of reading them is very similar to reading the Economist. At first blush, the Economist seems centrist and reasonable. However, after many years of review, it is clear that the Economists represent the significant monetary interests that read the periodical and advertise in it. These interests are against the common interests, but the Economist cannot say they represent wealthy bankers and opposed workers because they would lose credibility, and fewer people would read their magazine, as the bulk of their readership are essentially “want-to-bees.” (Interestingly, the Economist’s coverage of the enterprise software market is similarly overly focused on the most prominent firms) Likewise, if analyst firms only openly stated that they sell their opinions to the highest bidder, they would not be worth much.

Why Analysts Firms Errors Are Also Due to Information Gaps

Therefore, the analysts’ firms have a clear bias toward vendors that pay them the most (and the major monopoly vendors – SAP, Oracle, IBM, can afford to pay the most, so they get the rosiest coverage). The second part of the problem is that when I review articles written by analysts in software that I work with and configure, it’s clear that they have not used the software they are writing about. One cannot expect an analyst to have implemented or used every application they cover. However, I also do not think they are even reading the software manuals (a very time-consuming process but also necessary). Instead, they are working off of vendor presentations and marketing material.

The Problem with Gartner’s High-Level Coverage of Technology

One industry colleague once confided in me that when they worked for a significant vendor, an analyst expected to see all of that vendor’s supply chain products by spending three days at the company headquarters and meeting with different people. Analysts seem to overestimate their ability to understand applications from these superficial materials and presentations that they rely upon. Many analysts seem to think they can understand an application without getting into the details, and of course, they also have deadlines to meet. The scarcity of actual product knowledge was apparent in a recent review of SAP APO. I was reading from an analyst who had been “future sold” by SAP describing the functionality that did not work for me. I know this because I both tested it, and multiple OSS notes are open on the functionality described.

The Importance of a Low Embarrassment Threshold

I have since realized that to be a successful analyst. It is essential to have a high embarrassment threshold. You must write Pollyanish and uncritical articles for the vendors contributing the most cash to your employer. However, as the articles will be around for a while, you will necessarily be wrong. You do not have to feel bad about the incorrect projection. You have the enthusiasm to keep making projections. Although, perhaps not, I never see anyone bring up inaccurate projections of software firms or hold them up for criticism.

Forecast Accuracy and Bias

Interestingly, in supply chain forecasting, bias is frequently discussed in academic papers (although rarely acted upon to help reduce it). The issue of forecast bias of software analysts is entirely unaddressed. However, software analyst firms are biased towards new functionalities and prominent vendors. Therefore, they are not good sources of information on the likelihood of success using various applications. They do have value in synthesizing the marketing messages of many vendors into one place or one article. However, that is not what they are supposedly selling. They are allegedly selling critically thought out analysis of true vendor capabilities.

How Gartner Overestimates the Dynamic Nature of Various Software Categories

Using the software areas in which I have many years of implementing as a sample (supply chain), Gartner describes this software’s business environment as much more innovative and dynamic than it is in reality. This is interesting because Gartner developed a popular analytical product called the Hype Cycle, built on the premise that new technologies have an initial “Peak of Inflated Expectations Phase,” often followed by a “Trough of Disillusionment.” This is an acknowledgment by Gartner that it is much easier for a software vendor to develop new innovative technology than it is for that technology to be implemented.

But the strange thing about this is that while Gartner is famous for discussing hype cycles, it also promotes them!

How Gartner Promotes Hype Cycles

However, while the Hype Cycle points this out (and buyers use it to help them time their investments in software or other technology), I do not see it acknowledged in other Gartner writings.

Gartner itself seems to drive hype cycles, which they, in turn, criticize in their Hype Cycle analytical product. Consulting companies use This technique frequently to make their clients feel insecure about their progress. The more things are seen to change, the more demand for consulting and research services to keep up with that change.

Gartner and the ERP Hype Cycle

One of the most significant hype cycles in the history of enterprise software was ERP systems. Gartner coined the term ERP by taking the term MRP, removing the “M” and replacing it with an “E,” which stood for the enterprise. The ideas promoted by Gartner about ERP systems back in the 1980s were the following:

ERP systems would be the last system that any company would ever need. It would replace every legacy system (all systems that were not ERP then were called legacy by Gartner).
Companies implementing ERP would require minimal customization because so much of the standard functionality would match the customer’s requirements.
ERP systems would transform business by vastly reducing waste and improving standardization by following best practices built into ERP software.
Companies that invest in ERP systems will enjoy a substantial ROI.
Companies that did not invest in ERP systems would be left behind by their competitors who invested in ERP systems.

The exciting feature of these promises is that, in hindsight, none came true. ERP did become the largest category of enterprise software. Still, they became much more expensive than anticipated, required extensive customization, and became a far better deal for software vendors and consulting companies than for the companies that implemented them.

Now, virtually no one considers their ERP system an enabler for competitive advantage. The ERP system is simply a large aggregation of often mediocre functionality that seems to rise in TCO as time passes.

Why Gartner Promotes Hype Cycles

The question is, why? Why does Gartner promote hype cycles?

Lack of Reality Basis: One reason is that Gartner never actually implements software. Few people who work at Gartner have any experience with implementing software. Gartner’s lack of authentic knowledge about the software is evident from reading its media output.
Business Model: Gartner wants to appear as a leading edge. Supporting the newest things is an excellent way to look at the leading edge. Of course, the problem with this is that the latest items are often not ready to be implemented.
Paid by Software Vendors: Gartner will promote trends that come to them from software vendors. Gartner likes to say that they lead the trend, but they report on the trend. This is one of the problems of roughly 1/3 funded (of overall revenues) by software vendors. It means they bring across those things vendors want to be promoted.
Audience: Gartner’s audience is at the top of companies. This is also not a realistic group of people, by and large. Many are attempting to look at the leading edge as possible themselves. If you run into an IT executive at conferences, they are not very likely to tell you the real story of what they have accomplished within their companies. They want to make a good impression, and most are aware of their position and appearance. Therefore, they often have resumes filled with the newest technologies, whether they have been implemented successfully or not.

Go to top

Conclusion

The fact of the matter is that technology prediction is a tricky business. However, Gartner is not correct enough the time for it to be taken seriously. Gartner has not demonstrated any particular aptitude for technology prediction. On the other hand, if your client base does not notice Gartner’s prediction track record, and if the bold predictions make for good marketing copy, then there is no reason to stop.