The Brightwork Research & Analysis Rating of IT Analysts Like Gartner, Forrester, and More

Executive Summary

• After many years of reviewing IT analysts’ work, I wondered if any IT analyst firm was producing non-corrupt work.
• I review and rate some of the best-known IT analysts in this article.

Video Introduction: The Brightwork Research & Analysis Rating of IT Analysts Like Gartner, Forrester, and More

Text Introduction (Skip if You Watched the Video)

In 2013, I found it shocking when I began researching Gartner to prepare a book on the company. It became apparent that Gartner was not only not performing research, but that Gartner was extremely corrupt. At this time, I had not spent much time analyzing the other IT analysts. However, as time passed, and after reviewing more and more material from IT analysts, large problems at all of these IT analysts came into focus. The studies by IT analysts referenced by software vendors were rigged to meet the software vendor’s specifications.

In this article, I review other IT analysts along with Gartner.

Our References for This Article

If you want to see our references for this article and other related Brightwork articles, see this link.

Notice of Lack of Financial Bias: You are reading one of the only independent sources on Gartner. If you look at the information software vendors or consulting firms provide about Gartner, it is exclusively about using Gartner to help them sell software or consulting services. None of these sources care that Gartner is a faux research entity that makes up its findings and has massive financial conflicts. The IT industry is generally petrified of Gartner and only publishes complementary information about them. The article below is very different.

• First, it is published by a research entity, not an unreliable software vendor or consulting firm that has no idea what research is. 
• Second, no one paid for this article to be written, and it is not pretending to inform you while being rigged to sell you software or consulting services as a vendor or consulting firm that shares their ranking in some Gartner report. Unlike nearly every other article you will find from Google on this topic, it has had no input from any company's marketing or sales department. 

IT Analyst #1: Gartner

The Death Star of IT analysts. I call Gartner a “Rubix Cube of Corruption.”

Why a Rubix Cube?

Because every time I analyze Gartner, I find another dimension to their corruption. It’s difficult to keep “all of the sides of the cube” in one’s head. Hence I came up with the term “Rubix Cube.” I have documented many areas of Gartner’s corruption, but they are so numerous that if you ask me to list them all extemporaneously, there is no way I can do so without leaving some out.

Gartner’s best-known output is the Magic Quadrant. This item is entirely fake—Gartner rigs their Magic Quadrants backward from which vendors pay them the most.

One of the strongest pieces of evidence for this is that they leave out open source options from their Magic Quadrants, as we cover in the article Can Anyone Make Sense of the ODMS Magic Quadrant? This database MQ leaves out all open source options, even though they are some of the most popular and best respected. However, to Gartner, open source either does not exist or is a dirty word. Gartner’s excuse is that they only include vendors, not products.

IT Analyst #2: IDC

IDC is the second largest IT analyst firm in the world. They cover many topics, including projecting the number of laptops manufactured in a given year.

Like the next analyst to be discussed, Forrester specializes in sponsored studies where they write something very complimentary about a vendor or category of software. How Gartner and IDC Help Vendors Co-Opt Things, They are Unrelated To.

IDC has zero integrity and can be hired to write just about anything.

Owned by a Chinese Company and Then Blackstone?

IDC has an additional problem, and it is curious to never see it discussed.

The combined conglomerate (IDC/IDG, the IDG side controls some of the world’s largest IT media outlets) was acquired by a Chinese company. As every Chinese company eventually reports up to the CCP in China, IDC’s databases are certainly being used for espionage against US and European firms.

How the US FTC approved IDC’s acquisitions makes one wonder if US authorities either understand how China is continually performing espionage against the US and Europe or if all of the money China distributes throughout Washington DC is providing them with the ROI that they had predicted.

There are plenty of books on the topic of Chinese industrial espionage. This one above is quite good. The Chinese’s position is that the IP of the world is theirs to steal by any means necessary. This is why it is not a great idea to allow any Chinese company to purchase a company like IDC/IDG. One, because IDC has large amounts of information about US and European companies, and two, because the IDG part of the company publishes 8 of the top 20 IT media websites.

When owned by a Chinese company which is in a country ranked fourth from the bottom in press freedom in the world. There is little doubt that the Chinese government had full access to IDC/IDG’s database and that any information submitted to IDC for analysis is also used for other purposes. However, after being owned by China Oceanwide Holdings Group, Co. Ltd, in 2021, it was purchased by Blackstone. So that is better?

IDC produces low-quality work; I can’t recall reading any IDC report without seeing it rigged. Being owned by a Chinese company or now Blackstone will not have a company focused on integrity in research.

IT Analyst #3: Forrester

Forrester used to be a true competitor with Gartner. However, with Gartner’s growth, and continual acquisitions, Forrester is now tiny compared to Gartner in revenues. Forrester produces comparisons like Gartner called Forrester Waves. But, they are not very popular, and it is rare to hear anyone refer to them, while the Gartner Magic Quadrant is constantly referred to by buyers and vendors alike.

As noted in the IDC entry, Forrester is another entity that can be relied upon to publish anything that a vendor pays them to publish. Forrester will absolutely humiliate itself in its sponsored studies, which you can check in the article How to Understand Forrester Fake S/4HANA TCO Study.

How Forrester Likes to Keep it at a 30,000 Feet Level

A final point about Forrester is they function at a very high level. This is even apparent from how they market their services.

Forrester’s marketing is often very “head in the clouds.” I don’t know how long they have marketed this way. But they appear to overlap very strongly with strategy-type consulting. 

With Forrester’s descriptions, they could very easily transition into the self-help field.

I believe most of Forrester’s revenue comes from either scamming companies on their soft serve future predictions, or selling vendor-sponsored studies, that the vendors use as marketing collateral.

IT Analyst #4: G2Crowd

G2Crowd is not technically an IT analyst. However, they are prevalent and cover and rate enterprise software, so I have them included here.

G2Crowd pretends to be an objective user review-based website for software comparison. However, they are, in fact, a lead generation site for software vendors. Vendors are paying Their business model to showcase their products.

I only found this out through reading a G2Crowd press release that was not intended for users of G2Crowd and is something I covered in the article The Problem with G2Crowd’s Financial Disclosure.

G2Crowd manipulates the reviews but allows consultants to write reviews, and consultants only are interested in promoting their preferred applications.

There is some decent information available at G2Crowd. Though you often read a consultant’s review, the company and the website are completely dishonest, and they are lying to the people who use their website.

IT Analyst #5: Nucleus Research

Nucleus Research is a smaller IT analyst that once had a good reputation. However, sometime after 2007, they became corrupt. One strong piece of evidence is how they began estimating the ROI of applications after this time. There has been a huge increase in ROI in their studies/white papers.

The following is a tabular form of the studies/white papers produced by Nucleus Research on CRM that I could find.

Something that should immediately come across is that the ROI estimate from the studies after 2007 is extremely high. Let us chart the ROI estimates.

Nucleus Research noted in their later studies the software in the CRM category was getting better. However, this massive increase strains credulity. Most likely, this increase in ROI is more related to Nucleus Research’s ROI from accepting money from CRM vendors. That is, as they wrote more pollyanish articles about CRM, they got more money from the industry.

IT Analyst #6: G2Crowd

Reviewing A Table of the IT Analysts

Now that we have described each entry, let us review a table where some characteristics are applied.

IT Analyst #7: Constellation Research

Constellation Research was founded by Ray Wang, who is a bit of a conference celebrity due to his apparently fantastic speaking ability. However, when it comes to Constellation Research’s research follows the standard model of trying to keep vendor relationships and maximize financial opportunities rather than telling the truth and following a high-integrity research model.

If we look at one specific area of coverage, say of the SAP by Constellation Research, I find that it tends to repeat SAP’s assertions, even when the claims are false, and comes across as highly subordinate to SAP. I understand that SAP has very persuasive relationship managers, but they don’t know anything about the reality of SAP’s products, nor do they care what is true.

Ray Wang began his career at Gartner, a problem, as Gartner (as we covered) is highly corrupt.

I would place Constellation Research’s output significantly above that of Gartner or Forrester. And Constellation Research should be commended for not publishing the completely transparent “pay for study/marketing white paper,” as do Gartner, IDC, Forrester, and Nucleus Research. However, when facing down the mega-vendors, Constellation appears to buckle and give the politically correct and low-risk analysis.

Conclusion

This analysis concludes that none of these IT analysts is uncorrupt. I could add more entities to this study, but the results are the same no matter where I look. This means that these companies produce unreliable information for IT decision-makers. It is a damning statement, but that is where the evidence points.

The Broader Problem with Private Research

This says something broader about how poor to non-existent real research entities are in the private space. Private research mostly consists of pretending to be a research entity and then functioning as a marketing front end for the industry you cover. There is little difference between any of these companies and JD Power and Associates, which performs the same function but for consumer products.

These entities do not follow research rules and can be paid to present pretty much any view. And Forrester and IDC play the same game.

This is why the research entities and the entities on this list are very much about profit maximization, which can never lead to good research. It is also why private research has such poor outcomes.

Brightwork Research & Analysis is a research entity. And we have tried to get funding for real research in various technology areas. We have found there is no interest in this research, or that is where the conclusion is not predetermined. This is why nearly all of the research on the Brightwork website is self-funded.

What is generally called research in the IT space is predetermined outcome reports used as marketing collateral for vendors and consulting firms? People who reach out to us are often sales directors who want to know if they can have a written report that helps them in upcoming sales cycles.

That is where the market really is.

Are Only These Companies to Blame?

Here is a problem with only blaming IT analysts. Vendors often point out that IT analysts are corrupt (at least in private). However, how can IT analysts make money without in some way selling out? The vendors want positive coverage and eventually promote IT analysts to provide that coverage. Over time, even though the IT analyst may have started with good intentions (although not all, G2Crowd began as exactly what it is today), eventually, the market forces push it towards selling its brand and ratings.