The Results of the Brightwork SAP HANA Survey

Executive Summary

• This is the publication of the results of our survey on SAP HANA.
• These results are quite contrary to what both SAP and SAP’s consulting partners said about HANA.

Introduction

We ran a poll on the Brightwork website, asking detailed questions of those who had experiences with HANA. We are publishing this poll now because we have 100 responses to the survey.

Our References for This Article

If you want to see our references for this article and other related Brightwork articles, see this link.

Notice of Lack of Financial Bias: We have no financial ties to SAP or any other entity mentioned in this article.

• This is published by a research entity, not some lowbrow entity that is part of the SAP ecosystem. 
• Second, no one paid for this article to be written, and it is not pretending to inform you while being rigged to sell you software or consulting services. Unlike nearly every other article you will find from Google on this topic, it has had no input from any company's marketing or sales department. As you are reading this article, consider how rare this is. The vast majority of information on the Internet on SAP is provided by SAP, which is filled with false claims and sleazy consulting companies and SAP consultants who will tell any lie for personal benefit. Furthermore, SAP pays off all IT analysts -- who have the same concern for accuracy as SAP. Not one of these entities will disclose their pro-SAP financial bias to their readers. 

The Maintenance Overhead of HANA Versus Oracle or DB2?

Consistent with our research into HANA, the maintenance overhead is rated higher by roughly 1/2 of the respondents. But we did not expect any respondents to say it was lower than Oracle or DB2, primarily because it is indisputable that HANA is far less mature than either Oracle or DB2. 

HANA and Transaction Processing Performance

35% of the respondents replying that HANA is faster than Oracle or DB2 is not possible. This is covered in the article Why HANA Has Problems with Transaction Processing, John Appleby on HANA Not Working Fast for OLTP, and Issues with SD HANA Benchmarks. This is because SAP only added row oriented tables (which are better for TP) to HANA after it was first designed.

SAP, of course, promised enormous speed improvements, but it is not clear from the responses.

But then again, 35% of the respondents that stated it runs faster are incorrect. We know this because the design of HANA and because of SAP’s unwillingness to allow for any competitive benchmarking, and the fact that they created a new benchmark, the BW EML benchmark, as we cover in the article John Appleby on SAP BW EML Benchmark after HANA failed to perform on the previous transaction processing benchmark (called the SD benchmark).

This makes me wonder how many respondents are HANA consultants trying to promote HANA (hence the problem with reviews on G2Crowd, which is overflowing with consultant reviews.)

HANA Meeting Promises

This answer is amusing. How could HANA have possibly met the promises of SAP that 41% of the respondents stated that it did? Bill McDermott said that HANA ran 100,000 times faster than any competing technology. Steve Lucas stated that HANA would improve worker productivity by an average of 5,000 times! 

I have reviewed the promises made about HANA to many customers at this point, and the promises are so absurd, there is no way for them to be met.

Of the many promises made, there is one promise that could be met. If HANA was used for a pure analytics application, it could outperform previous versions of Oracle or DB2 on old hardware. If that were the only promise that 41% of the respondents replied “Yes” measured, then it would be possible.

Fifty-nine of the 100 respondents stated that HANA did not meet the promises.

Warning Companies About HANA?

Only 13% of the respondents recall if HANA not meeting its claims was ever discussed! However, 59% reported it did not meet its claims after the implementation.

This is quite standard in the SAP space, where both SAP and the SAP consulting partners mislead SAP customers in a unified manner. Gartner and Forrester would also be useless in warning customers, as they are both paid by SAP. Gartner published false information about HANA, as we cover in the article How Gartner Got HANA So Wrong.

Will Your Company Expand the Use of HANA?

The standard modality of HANA implementation is that we have tracked is that it tends to slow quickly after the first implementation. However, 1/2 of the respondents state that HANA will be expanded in use.

Again, as at least 35 of the respondents either deliberately falsified their answer on question one (or answered without knowing) and are therefore unreliable, this 50% value is quite overstated.

Did SAP Pressure You Fill Out the Survey?

No one felt pressured by SAP to fill out the survey. Again, we are trying to get realistic and not rigged answers to the survey. 

Conclusion

The survey results were interesting. But an impossible answer to one of the questions shows the bias in the survey results. Those that work in HANA want to promote HANA and would like it if the survey were inaccurate and positively biased.

There is no doubt that this survey has been positively biased in favor of HANA. Survey bias is a common issue. Political polls, for example, are biased by the method used, which often results in a non-representative sample being used. Confirming identity is more natural in person but far more expensive. Online polls are easier to administer, but the very individuals who visit the site (and therefore take the survey) mean a bias in the sample. A very desirable online method is to send the poll to specific people. But only a small fraction of people who sent a survey will fill out the questionnaire unless there is a strong relationship with the sender or sending organization.

Overall, there has been close to little attempt to get accurate information about HANA to customers or the market. IBM belatedly pushed back on HANA’s claims, as we cover in the article IBM Finally Fighting Back Against HANA. This is also covered in the article John Appleby, Beaten by Chris Eaton in Debate and Required Saving by Hasso Plattner. But competitors have been restricted in critiquing HANA because each of them is a partner with SAP. Overall, SAP’s powerful marketing machine has pushed inaccurate information about HANA into the market.