How to Understand Whether Gartner is Worth the Investment

Table of Contents: Select a Link to be Taken to That Section

Executive Summary

The starting cost for Gartner rises very rapidly the more Gartner is used.
Should buyers be selecting software-based on Gartner’s MQ ratings?
Is Gartner a good value for vendors? If so, for which type of vendors?

Introduction: Is Gartner Worth the Investment

Introduction

Gartner has a significant influence on enterprise software, but its value should be questioned far more frequently than it is. In interviewing IT decision-makers, we found that those relying on Gartner most often know little about how Gartner works behind the scenes. Procurement teams, for instance, will purchase Gartner reports. This is because they are told they are the industry standard. But they often know nothing about Gartner. You will gain insights into Gartner’s value to three different markets that Gartner sells to. This includes software buyers software vendors, and software investors. Some of the questions we will answer include: should buyers be selecting software based on Gartner’s MQ ratings? Is Gartner a good value for vendors? If so, for which type of vendors? And much more.

Notice of Lack of Financial Bias: You are reading one of the only independent sources on Gartner. If you look at the information software vendors or consulting firms provide about Gartner, it is exclusively about using Gartner to help them sell software or consulting services. None of these sources care that Gartner is a faux research entity that makes up its findings and has massive financial conflicts. The IT industry is generally petrified of Gartner and only publishes complementary information about them. The article below is very different.

First, it is published by a research entity, not an unreliable software vendor or consulting firm that has no idea what research is.
Second, no one paid for this article to be written, and it is not pretending to inform you while being rigged to sell you software or consulting services as a vendor or consulting firm that shares their ranking in some Gartner report. Unlike nearly every other article you will find from Google on this topic, it has had no input from any company's marketing or sales department.

The Best Value of Gartner

I would not be the first to argue that Gartner’s research is more relevant for investors than software buyers. But this also depends upon what the investor is using the analytical product.

Investors can be confident that right or wrong, Gartner’s opinions will be considered definitive by a large segment that can have a powerful effect on a vendor’s future condition. Because buyers have to implement the application and gain value from its operation, buyers cannot be confident. So while perception is a reality for investors, this is not true for buyers. For example, when I see a poor quality or high overhead application or database rated highly in a Gartner Magic Quadrant, investors can be confident that the product will be included in the shortlists of many software selections. Hence, perception, in effect, becomes a reality. Strangely, while Gartner’s research seems skewed towards investors, according to Gartner, investors only represent roughly five percent of all businesses, at least by number (not by revenue).
If investors seek to use Gartner for its technology or market predictions, there is no evidence that Gartner can forecast better than anyone else. Due to famous misses, as we covered in How Gartner Got IBM Watson so Wrong, How Gartner Got HANA So Wrong, How Gartner Got Fiori So Wrong, and Disregarding Gartner’s Deeper Technology Insights Predictions, it seems that in some cases (at least according to anecdotal evidence) Gartner is not even attempting to achieve forecast accuracy. Instead, Gartner is trying to either make a splash with a prediction or satisfy a large vendor with its projections, as we cover in the article How Gartner Promotes Hype Cycles. Gartner seems so focused on trends that they, at times, seem to be running a fashion magazine, as we cover in the article Gartner and the Devil Wears Prada. This is because vendors rely on Gartner to help promote their offerings, and they typically have some new hot thing they promote. One reason for paying Gartner is to have these items promoted as legitimate, allowing the software vendor to sell more software. The current craze in 2019 is artificial intelligence. A few years ago, it was analytics and Big Data. In a few years, it will be something else.

What is the Starting Cost for Accessing Gartner?

A standard and fundamental subscription to Gartner begins at roughly $30,000. Still, there are rumors that a subscription can be had for less if online access to the article database is required and minimal questions for analysts. The less a company pays, the less access to Gartner analysts. However, getting advice on negotiation with SAP will cost $100,000. It will mostly combine a few interactions with Gartner and some negotiation reports, as we covered in What Do You Get for $100,000 from Gartner?

Overall, it takes a lot of money before Gartner will do much for a client. This lack of effort on Gartner’s part is part of Gartner’s sales strategy. As we cover in the article, Gartner Runs a Boiler Room Operation in Fort Meyers, Gartner uses the lack of value at earlier levels to upsell customers to higher-level packages. Gartner is instrumental in meeting information, so the customer always feels incomplete and open to being further upsold. Companies that hire Gartner must understand this game.

It should be noted that legitimate research entities do not use sales tactics taken from boiler room operations.

Investing Based Upon Gartner’s Ratings

I am convinced that several studies of Gartner’s Magic Quadrant’s effect on the stock performance of software vendors have been performed and that a positive correlation has been found. These studies will never be revealed and will be used to improve the performance of the technology fund that performed the analysis.

Most likely, the private studies have not only charted the increased stock price. A favorable rating brings this about but addresses the conditions under which a good rating has the most effect (most likely when the rating of a medium-sized vendor increases significantly from the previous year’s Magic Quadrant).

The ideal vendor would match up well with Gartner’s Magic Quadrant methodology; however, they should be concentrated in it.
A large vendor that competes in multiple Magic Quadrants would be more challenging to track because they may have improved their position in some Magic Quadrants, stayed the same, or even declined in others.

What To Do With This Database?

Once a database of this nature was known, the investing company would need to review the Magic Quadrant as soon as it is published and make their buying decision as soon as they can before others can do the same. They would need to determine their investment strategy and be ready to trade before the fact. There are at least two “plays” available to the investment company.

One is the longer-term play that the vendor will receive more sales because of their enhanced Gartner rating,
The second is the short-term play that other investors (and possibly insiders at Gartner) are also savvy to the relationship and are buying the same stock.

According to my interviews with Gartner, investors are not all focused on the Magic Quadrant (which we cover in the article How to Understand Gartner’s Magic Quadrant.

Often investors are looking for the next big thing and want to know who is innovating. In addition to reading Gartner’s research and talking to analysts, investors can also attend various networking events.

Is Gartner Worth the Investment for Software Buyers?

Whether Gartner is worth the investment for buyers depends upon several factors. For instance, one important variable is how much is spent. Some buyers only purchase a subscription from Gartner, while other buyers spend upwards of $10 million on Gartner, at least for some years.

My observation, supported by many others who have analyzed this issue, is that software buyers often take Gartner’s reports at face value and focus too much on the results without reviewing the context. (There is also an open question about how thoroughly the executive read the Gartner reports.)

Gartner’s Magic Quadrants show a strong bias and complex logic once analyzed. This MQ for databases is one example, which we cover in the article Can Anyone Make Sense of Gartner’s Insane ODMS Magic Quadrant?

Something any Gartner customer should question is why Gartner so aggressively opposes open source and how this is driven by Gartner’s revenue model, which we cover in the article How Gartner Makes its Money, and why open source can never fit into their revenue model as we cover in the article How Gartner Opposes Open Source for its Benefit.

Gartner Wants Its Reports and MQs Explained by Analysts at More $$$

Gartner does not want buyers to use their analytical products without paying to discuss the products further with an analyst. They also say that only 10 to 15% of their reports’ information is in the report, and the rest is kept in the analyst’s mind. Imagining what would happen if a legitimate research entity wrote its reports this way should be straightforward. We ranked Gartner against real research entities, and the results can be found in the article How Gartner’s Research Compares to Real Research Entities.

As a Software Buyer Getting Value Out of Gartner

To get the best value for the time spent with a Gartner analyst, I would recommend merely telling the analyst your high-level requirements and then letting them recommend a shortlist of vendors. The less well-defined the requirements, the more this will cost the analyst’s time, and a full Gartner-guided software selection will be quite expensive and not entirely necessary.

Gartner for Software Selection?

The Gartner analyst is not the best person to tell if a vendor can meet your requirements. That is why I recommend communicating your high-level requirements to the analyst. Analysts at Gartner do not get into that level of detail in their vendor briefings. Standard types of information to have ready before such a call with a Gartner analyst would be:

How much do you want to spend
Your general preferences:
1. Are you very focused on having leading functionality or on having a low-risk implementation?
2. How important is software support to you?
3. Etc.
Where is the software being implemented now and in the future globally?

These are just examples; the Gartner analyst will have the complete list of items they will need to provide you with the required feedback. In addition to detailed surveys, the Gartner analyst will interact with vendors and software demonstrations to work. They will not release all of the survey information to you, but if you tell the analyst the type of information above, they can offer you a shortlist of vendors that may be a good fit for you.

Overall, Gartner’s insights are a lot more shallow than most assume. Gartner does not use the software they rate and has many examples of being completely incorrect on their projections. One example we cover is in the article Gartner’s Disastrous Advice on Mainframes.

Staying Away from Short Cuts

Many buyers will take shortcuts during the software selection process, and many are far too easily led through the process by skillful salespeople and presales people.[1] Under these circumstances, Gartner can do more harm than good. Over-relying on Gartner analytical products—without getting context from Gartner—can cost a company far more than their subscription and has cost several companies precisely in this way.

While we are on the subject of costs, a software buyer with just a few purchases to make in a year will often need to spend at least $60,000 once the analyst services are included, but this buys minimal analyst time costs escalate rapidly. This is not very much money for most enterprise software buyers. It would quickly pay for itself if spending this money enabled better decision-making on just a single application selection.

Every new software (or hardware, consulting, or telecommunications) purchase requires at least some type of advisement session with a Gartner analyst. The more products to be purchased, the higher the cost.

However, this price must also be compared to other alternatives in the market. Gartner may provide actionable intelligence to buyers, but it is certainly not the only source of information. In addition to asking whether or not Gartner’s research and discussions with analysts can improve software selection, the question must also be asked as to how Gartner’s costs compare to alternative sources of information. More than likely, Gartner is the most expensive information source a company can purchase, so a little bit of Gartner buys a lot of information from other sources.

Is Gartner Worth the Investment for Software Vendors?

The first question of whether it is worth having a subscription to Gartner if one is a software vendor is generally a no-brainer, and the significant vendors all know this and spend quite a bit of money on Gartner. Gartner likes to pretend that this does not influence their rankings. As we cover in the article Is Gartner Correct, they are Unbiased. They state they have an ombudsman where vendors can bring complaints which we analyzed in the article How to Best Understand Gartner’s’s Ombudsman. Gartner created this fake office to keep from disclosing who its vendor funders are. As we cover in the article, it has never disclosed this information: What is the Difference Between the Gartner Ombudsman and Disclosure? Gartner is so biased in its coverage that we caught them distributing straight PR material from vendors as their work, as we cover in the article How Gartner Distributes HANA Press Releases. This disclosure problem is industry-wide, as covered in the article G2Crowds Problem with Financial Disclosure.

Smaller vendors must see where they rank and the information buyers and investors read about them. Vendors don’t have a way of getting out of the Gartner system of tribute. If vendors do not pay, Gartner will retaliate against those vendors by lowering their rankings, as we cover in the article How Gartner Controls Vendors.

The financial bias in the Magic Quadrants is entirely traceable to who pays the most to Gartner. Not long after, new software companies raise tremendous money; they seem to perform exceptionally well in Gartner’s Magic Quadrants very quickly. Two notable examples of this are the vendors Birst and Snowflake.

In addition to vendor ratings, Gartner offers a great deal of analysis regarding current topics in the software industry, with mergers or new directions for software, hardware, and consulting companies, which we cover in the Analytical Products Offered by Gartner. Even if a vendor disagreed with all of Gartner’s ratings (and their rating with Gartner), a subscription to Gartner would be valuable from the market intelligence perspective.

Vendors want to know the answer to one central question: whether Gartner is worth participating in a more meaningful and expensive way (educating, buying consulting services from, attending Gartner events, etc.).

The Expense of Gartner

Any vendor who has dealt with Gartner knows how expensive it is to participate with them.

How much it benefits a software vendor is based on the vendor’s nature and product. Of course, it’s more than that. A software vendor dedicated to improving its rating with Gartner must also invest time and energy into educating Gartner. It is estimated to cost between $50,000 and $100,000 in direct fees to keep up Gartner on one’s products in a way that can put the vendor in the best possible position in the Magic Quadrant and Gartner’s other analytical products. This is sometimes referred to as the “Gartner Tax.” There are numerous approximations of how much it costs to get your software rated with Gartner, and so much of it is situational. There is also the question of how many products a vendor offers. This initial estimate is for one or two products. The price does not scale directly with each new product that the vendor needs to educate Gartner about, but generally, the price does go up.

Because many of the cost estimates are not specific to the number of products for which Gartner is providing technical advisory services, the amount of money a vendor would need to budget is unknown.

Quotes on Gartner’s Costs to Vendors

Some interesting quotes on this topic are listed below:

“All in (SAS Costs, Travel, Prep Time an optionally become a client) the cost of creating a good relationship with a Gartner Group analyst seems to average around $75k.” – Anonymous on Quora.com

“Being a client enables you to get ‘face time’ with the Gartner analyst team which is critical to getting their mindshare and more favorable positioning for you. This means attending their events and scheduling 1-on-1 time with them regularly, responding to all their requests for information and even booking a few days of paid analyst consulting time.

So I’d say you can in the box for $25K but if you want to really get value out of it you will want to spend more. Factoring in travel costs and extra fees it’s probably closer to $50-100K/year for a full effort.” – Kris Tuttle, Director of Research, Soundview Technology

These quotations are instructive regarding vendors’ costs to hire Gartner, and their cost estimates are similar. However, I want to note that these are only the explicit or direct costs of hiring Gartner. There are also internal costs that a vendor must tabulate to calculate their total costs.

The Internal Costs of Hiring Gartner

For example, the vendor’s labor cost is not included in these cost estimates to comply with Gartner’s requirements. There are forms to be filled out and communication with Gartner to keep up. A software vendor might make a similar investment when making a software deal. The difference is that there is no software sale at the end of the process, only the potential that future customers will contact the software vendor because of the better ranking provided by Gartner. A vendor can quickly recover that much in software sales and consulting fees resulting from an improved Gartner rating.

Gartner has a lot of overhead to be placed in their MQ. Gartner analysts have huge egos, and vendors report that they score better if they pay and make the analyst feel good about themselves. Some people can be hired to look after alliances that disingenuously compliment Gartner analysts and tell them how much they value their advice.

There is no reason why dealing with Gartner should cost this much, except that Gartner is in a dominant position in the IT analyst space, and therefore vendors can expect to have to play by their rules. Gartner can demand a premium payment from software vendors, and many are obligated to pay it.

In addition to consulting services, there is a cost to participate in Gartner conferences, which starts at roughly $35,000 for the basic package (booth at a conference, Gartner-arranged private get-together with potential clients, etc.) and can move up rapidly in price.

I have spoken with several vendors who mentioned that buying their technology advisement services helps them get a better ranking with Gartner. However, these technology advisor services are enormously expensive per consulting hour. Furthermore, the question of how much to spend depends on the company’s characteristics, with size being the most important determining factor, which is why I broke down the benefits by vendor size in the following paragraphs.

For Large Vendors

The primary software vendors’ participation is a godsend, and deciding to participate is easy. I am certainly not telling prominent vendors anything they don’t already know. Furthermore, large companies like SAP, Oracle, and Microsoft have so much money; what is several million handed over to Gartner? Gartner’s methodologies are designed to make these large vendors look as good as possible, so the value to these vendors is excellent, and they all sign up for Gartner’s services.

I have read Gartner’s statements, where they say that software buyers are increasingly interested in future solutions under development by major vendors. As a result, major vendors are listed in quadrants, for which Gartner readily admits the vendor has no product—yet. The value in doing so is to get software buyers to postpone their software selection until a significant software vendor they are familiar with comes out with the application.

Here, Gartner does the bidding of their largest vendor customers.

While not the most extreme example, neither SAP nor Oracle should be listed in this MQ. However, they were placed where they are due to the money paid by SAP and Oracle to Gartner every year.

For the significant vendors, Gartner is just one of several strategies that allow them not to compete for as much on their product, including partnering with major consulting companies, promoting the integration of applications within their suites, etc.

How Outspending Smaller Vendors Improves Ratings

SAP, Oracle, IBM, and Microsoft can outspend (in time and money) smaller vendors and get higher ratings that are especially valuable for marginal or immature products. For instance, any new product that SAP comes out with, no matter how buggy, will be rated decently merely because of SAP’s size. Microsoft SharePoint, the worst content management system I have ever seen, but one I am forced to use because it is installed at every one of my clients, is well-rated by Gartner.

Gartner seems to rate large software vendors high on “ability to execute,” even when I have seen many implementation problems, so being big is no guarantee of implementation success.

Gartner surveys quality measurements, but the methodology—just being large—increases how Gartner views a vendor’s “ability to execute.” This category of vendors doesn’t need to invest much time impressing Gartner; for prominent vendors, Gartner is an automatic purchase. For all these reasons, the large vendors are perpetual Gartner customers in the realm of multi-million dollars per year. In this way, Gartner can be viewed as a primary anticompetitive impediment that continually pushes buyers to the most expensive solutions, as we cover in the article Gartner and the Patent Software System.

For Medium-sized Vendors

For medium vendors, the marketing benefit to improving or maintaining one’s quadrant position/rating is typically worth the yearly cost and associated costs of working with Gartner. This is particularly true of vendors with international offices and broader software suites, which fit with Gartner’s methodologies.

I believe the benefits are fewer for medium-sized vendors, but the return on investment can still be positive. There are not just three vendors (large, medium, small) but rather a continuum of vendors. The bigger you are as a vendor, the more incentive there is to participate with Gartner.

For Small Vendors

The smaller the vendor, the more difficult it is to score well. Small vendors may have the best products in their class. Still, the best they can hope for is to make it into the Visionary quadrant. That is if they are lucky enough to compete in a Magic Quadrant directed explicitly towards their product rather than a Magic Quadrant where their product must compete against software suites.

Gartner salespeople will pitch that buying their consulting services can improve a vendor’s ranking. Still, given their criteria, it’s more sales talk than reality because it does not fit Gartner’s methodology. I have pointed this out previously, but it’s worth repeating: I do not think Gartner will adjust the numbers for a vendor that pays them money and purchases their technology advisor services. Instead, the methodology, combined with higher exposure to products from vendors paid to get to know better, is where the improvements in rankings come from. For instance, I score well in my age category for physical strength because of my genetics and exercise. However, I am not and have never been a fast runner. I could never perform very well in any ranking system for athletic performance, focusing more on speed than strength (i.e., its methodology is tilted towards speed). The same principle applies here.

Sweetening the Pot for Smaller Vendors

To give smaller vendors that don’t have a chance of doing well in the Magic Quadrant the incentive to buy consulting services, Gartner may profile them as a “Cool Vendor,” which is another report that they create. This prevents Gartner from losing revenue due to smaller vendors dropping out.

“First of all there are so many different quadrants at this point (so that competing firms can each claim a top spot) that virtually anyone can claim that they are in the upper right. Secondly, what CFO will base their decision on what Gartner says? Every firm has a unique “Magic Quadrant” based on their history, infrastructure, existing portfolio, etc. To suggest that there is a universal winner is simply marketing/revenue generation and a way for them to drive additional consulting services.”- Jon Carrow

However, in the final analysis, no other Gartner product is close to as influential as the Magic Quadrant. (I asked this question of people I interviewed for this book, and they could not recall other Gartner analytical products as being that influential.) I do not recall people adding the “Cool Vendors” winners to a software selection shortlist.

The Necessity of Placing Well-Known Applications High (For Credibility)

If a vendor’s product is well-known in its space, Gartner will want to include it in the Magic Quadrant regardless of whether the vendor pays Gartner. However, they won’t rank the vendor where it should be ranked because of their criteria. Many small vendors who repeatedly see more significant vendors with worse products than theirs ranked significantly above them in the Magic Quadrant typically opt out of the Gartner system once they find out how much money Gartner wants for their services. There is a distinct relationship between how large a vendor is and how fair the Gartner rankings are. The smallest vendors with the most innovative products tend to take the dimmest view of Gartner’s research.

Secondly, smaller software vendors have less money to spend than more significant software vendors, and therefore, participation and consulting services purchased from Gartner are much less of a sure thing. However, Gartner can begin to make sense to a small vendor depending upon the timing, where that vendor is in their growth stage, and how neatly their products fit into the Magic Quadrant. For example, smaller companies that are ready to make the next move and have the organizational and consulting capabilities to handle larger deals may feel it is time to engage Gartner.

How Gartner Functions

The following anonymous quote is instructive.

The Gartner and Forresters of these worlds have a simple, yet effective model: they talk to businesses, observe what is common, figure out some new terms and write about what is for 80% already present, so they find a willing ear with the audience because it suits the narrative instead of saying you doing it all wrong. Gartner doesn’t innovate, they replicate. You will never hear a Gartner person contradict someone and they assume authority by bombarding you with ‘facts’.

This video shows how an analytics solution can show the Gartner MQ changing over time. This video illustrates the common problem with analytics: it creates the analytics without analyzing whether Gartner’s Magic Quadrant has any validity. This is common in enterprise software, where no financial biases or claims are fact-checked. If you use an advanced analytics tool but don’t apply any critical thinking or know the domain, the result is false insight.

There is virtually no analysis of Gartner in the enterprise software space. As we cover in the article How to Understand Why IT Lacks Functioning Research Entities Covering SAP, virtually no entities perform research or have independence from vendors and consulting firms in the enterprise software space.

The enterprise software space is brainless from a research perspective. Companies do not bother to check the accuracy of firms like Gartner. All that is usually necessary is for an established brand to say something is true. That is, it is not only Gartner. Deloitte, WiPro, or IDC can propose something, and because these are established brands, even though the information is usually false, it tends to be believed.

Go to top

Conclusion

This article did not analyze the value of every of Gartner’s products but instead focused on the matter related to software and vendor rankings. Several additional services can be purchased from Gartner. For example, some companies hire Gartner to help them raise money from the investment community. I cannot say how much Gartner charges for this service or their success ratio for assisting companies in raising capital. The quality of Gartners is poor, and they are riddled with financial conflicts that are entirely undisclosed. However, the standards for research in the IT sector are extremely low, and decisions to purchase research are based on groupthink and the entity’s reputation. This is in part because companies that buy IT research have no idea how research works and are easily impressed with a brand name.

Throughout this article, I have discussed whether or not Gartner’s services are worth the expense. I have explained that in some cases, they are, and in some cases, they probably are not and that the determination changes depending upon “who you are.”

Comparing Gartner to Alternatives

Comparing Gartner’s research to the alternatives, such as other IT analyst firms, must also be addressed. For instance, even though Gartner is much larger and more prominent than Forrester, there are some things that Forrester does better. If the focus is content management and collaboration software (and more), The Real Story has far less financial bias than Gartner because they do not take money from vendors. Depending on the specific software category, a smaller IT analyst firm may cover it better. As usual, the best approach is not to commit everyone to one IT analyst firm but to use different ones for different purposes. One strategy might be to purchase multiple-seat Gartner subscriptions but only use Gartner analysts for the software categories. Gartner is the preferred source, and to augment Gartner with other IT analyst firms for specific software categories where these firms have better offerings. Unlike Gartner, many IT analyst firms allow their customers to buy a single article, making for a cost-effective approach and allowing the company to compare and contrast different reports.

Academic Research (for Some IT Topics)

However, other IT analyst firms are not the only alternative to Gartner. For instance, I often rely on academic research when writing books and articles. Academic research will not tell you if you should buy vendor A or B, but it provides a good overview of software categories. Another reason to look for a diversity of opinion beyond IT analysts is that writing orientation is entirely different. For instance, as with consultants, all IT analysts tend to be promoted because they make money when companies buy and implement new software. Academic researchers want more funding to perform more research; they are not attempting to sell software or consulting services. As a result, academic research is not promotional and may provide a historical context, often missing from commercial information sources.

Aside from the writing orientation, the depth of analysis is another reason to expand outside IT analysts and general IT publications. Academics are trained to research the nature of cause and effect thoroughly. Individuals with PhDs perform all academic research, so more highly-trained researchers in academia than exist in IT analyst firms (although some of those who work in IT analyst firms do have PhDs). Academic researchers perform literature reviews and ensure that their research conclusions are consistent or that any discrepancies can be explained within the context of other research in the same area. Academic research is, of course, less up-to-date than IT analyst research, so one cannot expect the latest happenings in IT from academia. All this to say, IT analysts should only be seen as one potential information source for a company that wishes to inform its decision-making.

The Overall Expense Level of IT Analysts

The final point is that IT analyst research is the most expensive of the various written information sources available to a company. The money spent on a certain level of IT analyst research can purchase enormous amounts of information from other sources. Of course, this scenario changes a bit depending on the party using the information. Vendors wanting to improve their ranking with Gartner will need to spend money on Gartner’s technology advisory services and make the adjustments Gartner recommends. But buyers and investors should first determine how much time and money will be allocated to software research. IT analysts should be allocated some percentage of that total, and Gartner would be a certain percentage within that category.

Sharing This Article

Share this article with someone you know by copying this article link and pasting it into your email so they can read the same information.