How Credible is Brightwork Research & Analysis in SAP?

Executive Summary

• Credibility is critical to getting research accepted.
• We address the issue of our credibility in SAP.

Introduction

In this article, we explore our credibility in the SAP space.

Notice of Lack of Financial Bias: We have no financial ties to SAP or any other entity mentioned in this article.

• This is published by a research entity, not some lowbrow entity that is part of the SAP ecosystem. 
• Second, no one paid for this article to be written, and it is not pretending to inform you while being rigged to sell you software or consulting services. Unlike nearly every other article you will find from Google on this topic, it has had no input from any company's marketing or sales department. As you are reading this article, consider how rare this is. The vast majority of information on the Internet on SAP is provided by SAP, which is filled with false claims and sleazy consulting companies and SAP consultants who will tell any lie for personal benefit. Furthermore, SAP pays off all IT analysts -- who have the same concern for accuracy as SAP. Not one of these entities will disclose their pro-SAP financial bias to their readers. 

A Small, But Highly Efficient Research Entity

While small, we have an outsized reach. We have a higher website volume than another IT research entity with 700 employees, which we cover in the article How Does Brightwork’s Web Influence Compare With Similar Sites?

A significant reason is that the available information on SAP is nearly universally financially biased.

Some SAP resources who dislike the conclusions in our articles, which do not help them sell SAP projects or bill SAP hours, have seized Brightwork Research & Analysis’s small size to declare that it means BR&A is less credible. However, this assumes that the size of an entity is correlated with research accuracy.

This is demonstrably untrue. In the next section, we will get into measured accuracy on SAP by some of the largest entities.

The Research Approach

The articles at Brightwork are written without concern for hedging or trying to develop a politically correct result. We write exactly what we conclude without going through a filtration step. Our view is that when you begin to filter a research conclusion, you aren’t doing research but have moved into a different area closer to marketing. 

How Faux Research is Written by Entities that Cover SAP

Analysis of articles written about SAP by Gartner, Forrester, and Constellation Research (as examples), demonstrate clear signs of the authors censoring themselves. The traditional IT analyst writes in a very neutral way. More specifically, the IT analyst writes their conclusions based upon, in significant part, triangulating backward from what is acceptable to say rather than what is true. 

We cover in the book Gartner and the Magic Quadrant a Guide for Buyers, Vendors, and Investors that the IT analyst space does not stack up very well against real research entities, such as academic research, Consumer Reports, or The RAND Corporation. There are specific rules to research, and they mainly originate in the scientific method, which declares that.

1. One works from the evidence to a conclusion rather than vice versa.
2. Second, that hypothesis is tested and not merely accepted (the concept of skepticism)
3. ..and that the researcher removes himself or herself from conflict.

These rules are enumerated in several places. However, private research has a generally poor history of following these rules. In this sense, we refer to research that requires independence, not research BMW might perform to improve its automobiles.

SAP Loyalists With Financial Bias Are Angry At Us

SAP loyalists have previously attacked this lack of neutral tone as reducing the credibility of Brightwork Research & Analysis. One SAP resource we debated and who subsequently wrote an article critical of Brightwork Research & Analysis declared that the material presented in an article HANA was “fake news” — based on it having:

“Not the right tone.” 

However, when asked to support the claim that our research conclusion was false, they admitted that they did not know themselves but used the tone to conclude.

Brightwork does not have a significant analyst firm staff that can employ six people to review articles and ensure every corner is rounded off. Gartner and Forrester do this, and they are not doing research — instead, they are marketing front ends for SAP and other vendors that pay them. They are reverse engineering conclusions based upon what is profit-maximizing. But even if every corner is rounded off, SAP resources tend to live in an echo chamber and believe several biased assumptions such as SAP has the most significant technology in software, Fiori is the best user interface in software, HANA is the fastest database in the world (running 100,000x faster than any competing technology according to Bill McDermot) and that Hasso Plattner mainly designed it. 

Brightwork’s Unmatched Accuracy in Predicting SAP

Curiously, it is rare for the research entity’s track record to discuss credibility in the conversation. For example, due to large sums of money paid to Gartner by client-server vendors, they provided inferior quality advice on mainframes, as we cover in the article How to Understand Gartner’s Disasterous Advice on Mainframes.

How Companies like Gartner and Forrester Continue to be Successful

A primary reason that Gartner and others can get away with offering such financially rigged and poor-quality advice is that the IT analyst market is very shallow. Gartner has made many acquisitions, which have significantly reduced competition in the space and provided a monoculture type to the IT analyst market. IDC is another foremost IT analyst, but they don’t cover the same things as Gartner. Forrester is now so small about Gartner that Gartner has, in effect, a monopoly, particularly on the lucrative vendor comparison market. Websites like G2Crowd are popular, but they are a lead generation front end for vendors and are not in the same space as Gartner.

Neither Gartner, Forrester, nor Constellation Research has anything approaching Brightwork’s accuracy in predicting SAP. Rather than challenging SAP’s assertions, Gartner, Forrester, and Constellation Research tend to align themselves with SAP’s statements. Any IT analyst that does this will have poor accuracy.

Our accuracy can be reviewed in the article A Study Into SAP’s Accuracy.

What is clear by reviewing the list is that if companies followed Brightwork’s advice, they would vastly improve their SAP spend.

How We Attained the Best Accuracy in Enterprise Software on Predicting SAP

Brightwork can attain this accuracy because we use our experience of being on projects, not spending a lot of time talking to executives, and we are focused on doing good research rather than profit-maximizing the business.

We believe that as soon as any research entity attempts to follow a profit maximization route, research is dead at that entity. Anyone wanting to leverage Brightwork’s research output should reference our accuracy. Our statement in response to “SAP homers” is as follows: 

“If Brightwork Research & Analysis has an ax to grind with SAP, has a problem with Hasso Plattner and Bill McDermott, and (fill in the blank), why is Brightwork so increadibly accurate about SAP?

In the forecasting world, a positive or negative bias results in the forecast being inaccurate — and consistently inaccurate.

A perfectly accurate or nearly perfectly accurate forecast cannot be claimed to be biased.

Allow us to illustrate.

This forecast has a negative bias. It has repeatedly produced a lower forecast than the actuals. 

This forecast is consistently aligned with the actuals. How can an accurate forecast be referred to as biased? It cannot. 

Moving From the Data to the Conclusion – Rather Than the Other Way Around

We are not afraid to write whatever the data drives regarding a conclusion. This is not done in the IT analyst community, which tends to do the reverse. The senior members tell IT, analysts, the conclusion they need to arrive at, and then the analysts rig the text to match that conclusion. 

None of the major or nearly any of the smaller analyst firms are willing to do this. One analyst firm was known for writing material critical of SAP, as they had Oracle as a client. However, as soon as SAP bought them off, the SAP material was removed from their website. 

Brightwork claims the most accurate track record in predicting HANA, and we can guarantee no other IT analyst firm will produce evidence to contest this assertion. All the major IT analyst firms’ track records are so poor that they don’t want the subject broached.

Furthermore, Brightwork is the only IT research entity that we are aware of to track accuracy. We have many articles on the site that compare the accuracy of IT analysts and IT media. 

Another article where we compared accuracy was our retrospective evaluation of Who Was Right and Who was Wrong about SAP HANA?

And we don’t only cover SAP or make predictions on SAP. We have predicted a coming bust in AI/ML.

Why?

Because the statements by vendors and consulting companies do not match these projects’ reality or capabilities.

Will Sharing Brightwork TCO Calculators be Interpreted as Flinging Mud at SAP?

This is a question we have received from several vendors. So, we need to answer it here and provide the correct context, which may not be known by those who ask this question. Any material provided to a prospect raises the question of the entity’s independence that produced it.

Our domain expertise around SAP is complex to undermine. But ordinarily, SAP will respond that Brightwork is not an approved source of information. The only approved sources of information by SAP are IT analysts SAP pays. Explaining that to a client puts SAP in a problematic spot in illustrating how self-centered the policy of “approved analysts” is. Imagine if Consumer Reports were declared “not an approved analyst by General Motors.” And that JD Powers & Associates, which GM does pay, is an approved analyst. How well would that go over? Furthermore, SAP’s history of accuracy on TCO regarding their products can be called into question as this article does on The Problem with Forrester’s TCO Study on S/4HANA.

This was an SAP-funded study, naturally, and when we walked groups through the study, it resulted in hearty laughs from the audience. The study is embarrassing, but neither Forrester nor SAP seem to appreciate why. SAP’s own funded studies can be used against them.

Why SAP Wants to Assert Lower TCO, Not Get Into the Details

SAP usually does not try to get into details because they know they will lose. Thus, the correct response from a vendor would be,

Let us get into the calculations and the assumptions.

SAP Will Attempt to Move for Immediate Dismissal

If SAP can push the account to dismiss the calculation immediately, then, well, the calculation does not matter. But SAP can’t survive any scrutiny of their TCO claims. So, outright dismissal is what needs to be stopped. A response can be,

SAP makes very large claims for its lower TCO, how are these claims supported. And what independent entity agrees with SAP on these claims?

The subject being broached is positive for any vendor competing against SAP because TCO for SAP is a house of cards. No IT analyst will tell the truth about SAP TCO because they seek to keep SAP or gain SAP as a customer. This is where Brightwork can offer remote presentation/support for the calculators, answer questions, etc. Or there can be remote questions answered by email, where Brightwork then points to supporting articles and can even create new ones to answer questions.  Brightwork does many TCO studies. See this article for the complete list of The Public Brightwork TCO Calculators.

• All of the studies are done the same way.
• They are based on regression formulas.
• They are adjustable, but only with a range. For example, a very large implementation requires a separate calculator from just a large implementation.

Credibility Versus the Vendor

Some vendors will ask about the credibility of Brightwork Research & Analysis without asking the question..

“Compared to what.”

Vendors sometimes reach out to us and ask us to help them create their value estimation—this is something we never do. But what is curious is that vendors who ask us to do this don’t appear to ask themselves how credible a TCO study is that a vendor produces. It should be self-evident that a study by an independent party is more credible than an entity trying to sell a product. However, the entity must also be genuinely independent. Forrester will take the vendor’s assumptions and produce a TCO study to the vendor’s specifications. JD Power and Associates create awards that deceive customers about the quality of automobiles. Their primary customers are the lowest-quality auto manufacturers. These are not examples of independent entities that follow research rules.

Conclusion

The credibility of Brightwork Research & Analysis in SAP is complex and dependent on the person in question. Any entity not parrot SAP marketing’s material lacks credibility for dyed-in-the-wool SAP shops. But these aren’t very good potential customers in any case. On the other hand, we enjoy high credibility for many people who work on SAP projects and see the gap between SAP’s marketing department and SAP projects. We are a severe research entity with the highest history of accurately predicting SAP. We have the studies to demonstrate that we are the most accurate source of SAP.