How to Understand Why IT Lacks Functioning Research Entities Covering SAP

Executive Summary

• The IT industry lacks functioning research entities.
• Much of this has to do with how little money there is in performing actual research.

Introduction

Something occurred to us recently after we published the first independent TCO study on HANA. This is what we stated.

“And apparently, IT lacks functioning research bodies that focus on SAP. We say this because years after SAP made its TCO claims, no research entity, several with infinitely larger resources than Brightwork investigated this issue. In fact, the two most prominent IT analysts (Gartner and Forrester) released SAP supportive and inaccurate information on this topic.

The overall story on HANA never made any sense. The research shows there are foundational issues that make SAP’s assertion impossible. For one, how can an immature product that is priced higher than a mature product have a lower TCO? Before one gets into the calculations, the just first approximation would seem off to anyone experienced in system implementation.”

Before we go any further, we want to call out one research entity that does an excellent job covering SAP. One is Bloor Research, which you can see in this article How to Understanding Bloor Research on HANA.

But there are very few research entities to point to that cover SAP accurately, and that is what we will cover in the following article.

The Present Day Rules of IT Research

The rules of IT research are simple. IT analysts (I speak only of those that cover SAP and other vendors) try to maximize their revenues and profits. To do this, they get as much money as they can from vendors. This means both the largest vendors have the most money to spend to get the most preferential treatment. Further, aside from giving analysts money to buy better positioning, the larger vendors can afford the best analyst relationship teams. Constantly updating analysts, who are often switching positions or leaving the analyst companies, is a lot of work for the software vendors. And smaller vendors have less money for doing it. The more care and feeding, and flattery the vendor can give the analysts, the better your outcome. That is how good the vendor can make the vendor feel about himself or herself. That is a sign of respect for their opinion.

So, the research entity will censor what they write about the largest vendors right off the bat. And things bring up the question of HANA and its TCO claims. The one TCO study that was published happened to be published by Forrester. It was paid for by SAP. The study is completely unreliable and should be disregarded as it was engineered backward from the desired outcome. You can read our analysis in the article How Accurate was Forrester’s TCO Study for SAP HANA.

Academic Research into IT

Academic research into IT does exist, and we very frequently review the academic research and weave it into our own research as we did for the book The Real Story Behind ERP. However, academic research is very rarely referenced or read in IT.

How Consulting Companies Undermine Research

Consulting companies are supposedly knowledge-based entities. So what is their view and presentation of research?

We can recall working for a consulting company, and when we presented a reference to academic work, we were told the following.

“The client paid for our advice on the topic, not for some academic’s view on the topic.”

This comment was particularly ironic because we subcontracted to this consulting company and offered them the knowledge they did not possess. They obtained the project contract, in part by claiming to possess capabilities that they did not. When on the project, we were instructed to use the business cards they gave us and pretended to be part of the consulting company.

And this is the norm, where subcontractors are kept as secret as possible (note for customers, check each of the consultants on LinkedIn, and find out how many works for the consulting company and how many works for another company or are independent consultants).

On the topic we were offering to the consulting company, the consulting company’s background and experience were literally zero. All of it resided with us. And we have learned from project experience, researched by others, and books by others. In this area in question, we have written some books, but much of the information comes from other sources.

That is not the answer the consulting companies want to tell their customers. This is because many consulting companies approach their customers to offer customers something they can’t get anywhere else. Deloitte says their “methodology” distinguishes them from anyone else. Capgemini claims the same.

I have actually evaluated methodologies for SAP implementation from many consulting companies, as I covered in the article The Real Story on SAP Implementation Methodologies and did not find anything particularly different in any of them.

Consulting companies adopt concepts and methods from outside but tend to stamp their name on them. Consulting companies will tend, to the degree that they can, hide the origin of what they offer to customers.

Example of Consulting Appropriation

One well-known vendor in application integration has consulting companies use its software without telling the customer thinks the consulting company was created.

When I worked for one consulting company, I took an inventory formula from Plossl’s Inventory Production and Inventory Control book and built an inventory calculator. When I showed it to my manager, I was asked where I figured out the calculator’s math. I told him.

And he told me,

“No matter what you do don’t tell the client where you got this from.”

Consulting companies operate not on transparency but secrecy and keeping things proprietary. The benefits tend to be “stupendous.” But of course, not validated.

Once again, the presentation was to be that the formula was somehow proprietary to the consulting company. On the subject of SAP indirect access, several SAP negotiation companies have declared that they have the answer to managing indirect access issues. All the outcomes for clients are “fantastic. There is no way of validating any of these claims. This is all hidden and unpublished.

Therefore, there is a sizable contingent of entities in the IT space that are not only not positively disposed to research, but quite the contrary is actually hostile to research. Consulting companies, by in large, don’t do research (if any reader disagrees, I am happy to learn from you, please provide an example). For example, Deloitte’s research group reports directly to marketing. Not a single word is released without being evaluated by marketing regarding whether it will help increase sales.

Consulting companies create the idea that they create all of this fantastic knowledge internally (to the degree that they can maintain the illusion).

What Would IT Look Like with Functioning Research Entities?

A functioning research entity that was not trying to maximize its income from vendors would be free to perform fact-checking. And if it existed, SAP’s HANA TCO claim would have been falsified many years ago.

None of the entities that cover SAP would meet any accepted definitions of research entities. They are profit-maximizing firms that sell faux research to people who do not know the research rules.

Recently someone proposed something to me that I found quite interesting about Gartner.

I quote:

“A company like Gartner can make of break other companies. Taking money from parties looking for the truth and parties that want to become the new default in the market is a perverse incentive.

To conduct objective research and sell the outcomes, like you do, certainly sound like something that the market could use.

But is it really what they want? Are you sure that this market is about objective advice rather than buying off responsibility for decisions? Maybe both Gartners and their clients both know full well it is a scam, but since it serves both interests, everyone plays along.”

Know They are Buying a Sheild

And this brings up the question, do Gartner’s buyers also know that Gartner is corrupt and that they are not buying any real research but instead simply repurchasing a shield, being held culpable for bad decision making?

I have thought about this, and when I have discussed Gartner with a wide variety of software buyers, I have often found that they have no idea what research rules Gartner follows. That was a reason I wrote the book on Gartner to get the word out. The people that generally know that Gartner is a scam are the software vendors. Although they normally also do not know how Gartner’s research works specifically. But they know something is fishy. This is because they have much more exposure to how the “sausage is made.” Gartner is very close to universally unpopular among all but the largest vendors.

All of this leads to remarkably uninformed decision-making in IT. It also emboldens many software vendors’ marketing departments to make exaggerated claims knowing that these IT analysts will not expose them. What entity can be expected to contradict software vendors’ claims if they rely upon these same vendors for a sizable portion of their overall revenues?

Conclusion

Brightwork Research & Analysis is a real functioning research entity. How can we prove it? Well, for one, we challenge prevailing concepts presented by software vendors, consulting companies, Wall Street analysts.

Research means challenging and testing hypotheses.

But our impact is tiny compared to the major brands in the analyst space. Furthermore, outside or Bloor and ourselves, none of the analyst firms either get into sufficient detail to validate what SAP proposes from the technology. For example, Gartner tried to cover HANA from a technical perspective and fell right on their face.

The peculiar story of Gartner’s support for HANA and how the lead researcher for HANA was also the relationship manager for SAP is covered in the article How Gartner Got SAP so Wrong.

Verifying Statements on Technology

These analyst firms do not hire people who can verify technology statements from SAP or other vendors. Instead, their method is to talk to vendors and talk to (primarily) executives without any hands-on or technical verification.

The state of private (non-academic) IT research is not what anyone would expect from the outside. We are essentially saying that, much like what we are going through politically in the US with Trump’s rise, what is true does not matter. What matters is what illusion can be created. And we are back to one of my least favorite phrases that “perception is the reality.” And it follows that we have faux research entities that follow no research standards and do not and are not set up to fact check but serve as tools to repeat the messaging for the highest bidders.

New Entrants, to Improve this Situation?

And perhaps most interestingly, there is no pathway out of this. New entities like G2Crowd have risen in the marketplace., but G2Crowd is a review aggregator; they are not a research entity — and they serve as lead generation for vendors and are paid by vendors. None of this is declared by G2Crowd. So they follow the “Gartner” model of disclosure. Furthermore, G2Crowd has an exploitive model where the owners of G2Crowd get rich while the people that write the reviews get nothing. And they don’t actually do any research into SAP. They have reviews of SAP products.

So much for the “new-new” model.