G2Crowd’s Problem with Financial Disclosure

Executive Summary

• G2Crowd promised a new way to review software.
• However, G2Crowd has major disclosure issues that we cover.

Introduction

Recently, during an interaction on LinkedIn where the head of CrowdReviews brought up Gartner’s bias in being a pay-to-play system. During that interaction, the topic of G2Crowd came up, which is another applications ranking entity that the head of CrowdReviews was also critical of. In my view at the time, G2Crowd was far more transparent than Gartner, and G2Crowd aggregated reviews from people that seemed to have application experience, unlike Gartner analysts. However, the discussion got me thinking about how transparent G2Crowd is, which brought up an investigation into G2Crowd and a test I performed on G2Crowd.

Inflated Application Scores

When investigating the G2Crowd site, the biggest thing that is not making sense for me is that G2Crowd’s scores for bad applications were just too high. That looks fishy to me. I know quite a few bad applications because I see them either fail at clients or add very little value. These happen to be applications that are most recommended by consulting companies because they maximize the consulting companies’ profits. SAP has many applications like this, and they are like a vacuum cleaner, hoovering up the budgets at IT departments with their long-term maintenance costs. But it is not only SAP. Tableau, an impressive application in many ways, does not implement anywhere close to its advertised level. Tableau has a lot to recommend, but users will not be developing the self-generated reports that look anything like the beautiful reports that Tableau creates for demos or you can view on their website. And one of the problems with G2Crowd is that if most of the people that write Tableau reviews are making money from their Tableau skills, you won’t get real in the reviews. Do they know more about Tableau than almost anyone? Yes. Do they care about the benefits of Tableau?

No.

Why not?

There is a big difference between what Tableau does for them (gives them a nice job) versus what it does for users.

Performing a Test

So I performed a test. I selected one of the worst applications I have ever tested. An application that adds incredibly little value but which IT departments love spending money on. That is the SAP Business Warehouse or BW. So the first thing to test was whether SAP BW was rated low on G2Crowd. To my surprise, SAP BW was rated at four stars, with only one low review (2 stars) before I began this experiment.

Here is a tabulation of the reviews of SAP BW. There are four five-star reviews and five four-star reviews. How could such a weak application receive such high reviews? The one-star review is mine. Only one other review at G2Crowd is low. 

The Test

Therefore the test I intended to perform was to see if G2Crowd would accept an accurate but low review of SAP BW and see if G2Crowd would publish it. After submitting the review, it went into the review queue. And G2Crowd did publish my review around a day later.

Therefore G2Crowd passed the test of publishing the negative (but accurate) review. I was asked by another person on LinkedIn why I had even questioned why G2Crowd would not publish the review.

Why should they censor? Their business model is expert review. As to your other question: Maybe there are other sides to BW that you do not look at? Everybody has a blind spot, I too (perhaps more than one in my case …)”

To me, this question is merely naive, and I responded with the following:

“That is simple, so that they don’t offend SAP, if SAP happens to be a funding entity. SAP funds all manner of entities such as Diginomica, Gartner to promote false information about SAP. But to Jeev Trika’s point, I have no idea if SAP even sponsors G2Crowd. As with Gartner, they do not explain who their funders are on their website, preferring to not have their readers know this information. The idea that BW is a good application, for anyone but people that make money from it, is simply untenable.

On repeated SAP projects I have found BW simply adds very little value. Reports take forever to develop, and it unsurprising because the BW Workbench is difficult to use and the overall design of the BW is extremely dated. I use a forecasting application to do exactly what the BW does for SAP DP, and I don’t have any of BW’s problems. This illustrates how dated BW is. So what you are proposing would not at all match with these observations, observations which I have collected over a decade. If BW cannot be called a bad application, then no application can be called a bad application.”

One might be tempted to say that G2Crowd has passed the test, and in a way, they had. This sentiment would be echoed by G2Crowd’s press release of May 24, 2017.

“There is a paradigm shift occurring as B2B buyers increasingly behave like consumers, relying on authentic online reviews and social media to make purchase decisions and no other company is better suited to catalyze this market opportunity than G2 Crowd.”

I have no reason to believe that G2Crowd’s reviews are not authentic, and many have the person’s LinkedIn profile and show as verified. But that is not the end of the matter. Once again, if bad applications can receive good ratings, then “authenticity” does not appear to be the magic elixir that it is presented to be. Secondly, many consultants in the particular applications are leaving reviews — and surprise, surprise, those reviews are nearly always glowing. That alone tips the review average due to this bias. The more consultants can pump up applications they work in, the more money they stand to make.

From the bigger picture, G2Crowd appears to have some bias. I say this because if reduced quality applications get excellent ratings, then the rating system is not effective. And the questions arise, why this application received good scores and 4 cases a perfect score (5 stars). Again, this may be the bias of who decides to write reviews.

I found a similar overestimation in other poor quality applications on G2Crowd. I do not know the answer to that question, but I have a few ideas that I will discuss at the end of this article, leading to the next topic, which is G2Crowd’s vendor-funded business model.

G2Crowd’s Vendor Funded Business Model

The next topic that arose was G2Crowd’s funding. I noticed that G2Crowd has nowhere on their website where they list who their vendor funders are or how much they receive from each funding entity or each software vendor. This should be fairly obvious that this is something that G2Crowd should list. But G2Crowd does not seem to be interested in anyone knowing that they take money from vendors on their website. I would go as far as to state that G2Crowd is hiding this fact. So the question is, why? And the answer is obvious.

Once again, the person I was interacting with on LinkedIn stated.

“Re G2Crowd – while they are not required to publish investors there seems to be quite some info available, from them as well as from other sources.”

G2Crowd may not be required by law to declare their funders, but why is it considered acceptable for them not to be? Is this simply a svelt excusing of corruption? I could ask why G2Crowd seems so little interested in being honest with its readers and would ask readers whether they would question the motivations of someone who manages the reviews of applications but then was not involved in disclosing that they receive income from the vendors whose applications are rated.

The Obligation to Disclose

I should also point out that any company performing research is obligated to disclose its funding that may create a conflict of interest, and this has been well established in research for a great deal of time. Therefore, G2Crowd, like Gartner, cannot be considered an entity interested in research, as they do not follow the established rules of research.

I then looked up several of the links sent to me by the person I was corresponding with (and which I have copied in the reference section) and found the following quotations.

“The 2016 fiscal year saw more than 300 percent growth for the company as over 800 technology leaders, including Salesforce, Marketo and IBM have signed on to leverage G2 Crowd’s unique user generated content in their marketing programs.” – G2Crowd

So if we look at this, it is intended to impress people. It is not designed to expose their funding for the users of G2Crowd. And, of course, no specifics are mentioned. Three companies that were mentioned without any other details are not real disclosure.

G2Crowd as a Marketing Extension of IBM

And the press release had another fascinating piece of information, something I don’t think the person to whom the quotation is attributed knew the implication of what they were saying.

“G2 Crowd is an important partner in expanding online advocacy for IBM. The user review platform is giving us new ways to connect to our customers, hear their feedback about our products and services, and strengthen our culture of customer centricity.” – Kevin Eagan, Chief Digital Experience Officer, IBM

Kevin Eagan essentially gave up the game with that comment. IBM sees G2Crowd as an “advocate” for IBM. And advocate as a biased party. An advocate who a party pays is, of course, a super biased party.

Secondly, IBM is not entirely a customer-centric player. So this statement is ridiculous. IBM may want to become more customer-centric, but they are not now, and they have not been in recent memory. But the part about seeing G2Crowd as a way to advocate for IBM applications is undoubtedly true. But it also brings up several questions.

• Will IBM force-feed reviews from its employees into G2Crowd to improve its ranking?
• Will G2Crowd effectively police this? Will, they put resources into police it?
• If G2Crowd does police, will this cause a problem with IBM’s funding?

These things are all quite relevant questions because IBM lies all the time. They lie at conferences. They lie on consulting engagements (I have worked with them and have seen them lie and lie egregiously upfront). They lie in their contracts (something covered in great detail in the excellent blog by I Cringley, one of IBM’s foremost experts. Especially at the higher levels in IBM, it is virtually impossible to find honest people. They repeatedly lied to get to their positions. I do not doubt that if given the opportunity, IBM will encourage its consultants to write reviews on G2Crowd.

G2Crowd hiding its vendor funding makes it more difficult for those outside the organization to catch the upward bias on IBM’s applications.

Conclusion

This non-disclosure is becoming more and more evident that these private entities only don’t disclose their income. Gartner offers their ombudsman, which I have analyzed in detail and have found to be entire without merit. I have covered the ombudsman in these articles is The Gartner Ombudsman for Real? And Differentiating Between Gartner’s Ombudsman and Disclosure Requirements. But be it G2Crowd or Gartner or Forrester or other entities, the perspective is the same. The entity hides its funding sources, and if asked about funding, it indicates that who is funding it is not significant and would never impact what they write. Honestly, this isn’t easy to swallow. I have researched the effect of funding on media output, and the conclusion is that funding does influence media output. I cover this in the article The History of Media Entities and Financial Bias.

If readers don’t expect more from rating entities, the rating entities are happy to continue hiding funding sources from their readers. It isn’t very ethical, but the people that own these entities generally don’t care. So unless they are pressured to change, they won’t.

The Problem: Thinking that Gartner is Focused on What is True

Gartner is hired by companies who fundamentally don’t understand how Gartner functions. Gartner has virtually no first-hand experience in the technologies they evaluate and get most of their information from executives at buyers or executives at vendors and consulting firms. Gartner is also not a research entity. They compare very poorly to real research entities once you dig into the details, as we did in the article How Gartner’s Research Compares to Real Research Entities. Gartner serves to direct IT spending to the most expensive solutions as these are the companies that can afford to pay Gartner the most money. Gartner has enormously aggressive internal sales goals that place accuracy far below revenue growth in importance.

References

*https://www.g2crowd.com/press-release/accel-leads-30m-series-b-g2-crowd-accelerate-growth-market-leading-b2b-review-platform/

*https://www.g2crowd.com/press-release/g2-crowd-series-a-funding/