How Accurate was Forrester’s TCO Study for SAP HANA?

Executive Summary

• Forrester was paid off by SAP to create a false TCO study that is inaccurate.
• SAP fed a passive Forrester TCO information on HANA.

Introduction

SAP paid Forrester to produce a fake TCO study of HANA published in April 2014. This study had no validity and passively used information provided by SAP without questioning this information. The study was used by SAP marketing to try to prove something for which there was never any evidence, which is that HANA has a lower TCO than competing databases. At the time, barely any SAP HANA instances were live. You will learn about the reality of Forrester’s HANA TCO study.

Our References

If you want to see our references for this article and other related Brightwork articles, see this link.

Notice of Lack of Financial Bias: We have no financial ties to SAP or any other entity mentioned in this article.

• This is published by a research entity, not some lowbrow entity that is part of the SAP ecosystem. 
• Second, no one paid for this article to be written, and it is not pretending to inform you while being rigged to sell you software or consulting services. Unlike nearly every other article you will find from Google on this topic, it has had no input from any company's marketing or sales department. As you are reading this article, consider how rare this is. The vast majority of information on the Internet on SAP is provided by SAP, which is filled with false claims and sleazy consulting companies and SAP consultants who will tell any lie for personal benefit. Furthermore, SAP pays off all IT analysts -- who have the same concern for accuracy as SAP. Not one of these entities will disclose their pro-SAP financial bias to their readers. 

Forrester’s Explanation of HANA

Even Forrester’s beginning explanation of HANA appears curious.

“SAP HANA is a distributed in-memory data platform that enterprises can use to support real-time analytics, predictive and text analytics, and extreme transaction volumes. The next-generation data platform demands looking at these new technologies to help deliver the speed, agility, and new insights critical to helping your business grow. For decades, organizations have built the transactional, operational, and analytical layers to support various applications, operational reporting, and analytics. However, with the growing need to support real-time data sharing driven by mobile enterprise, separate transactional, operational, and analytical layers are creating an obstacle in supporting such an initiative. Distributed in-memory data platform offers a new approach to collapse the technology stack that can eliminate redundant hardware, software, and middleware components to save money and reduce complexity through automation and integrated systems that can help developers and DBAs become more productive.”

This sounds like SAP provided it.

SAP HANA Total Cost of Ownership Framework

Here is where Forrester lays out the foundation of the research.

“To better understand the costs and risks associated with a HANA implementation, Forrester interviewed several customers with multiple years of experience using HANA and conducted a survey of 25 additional HANA users. Drawing on the experiences of these customers, as well as Forrester expertise in this area, we constructed a financial model to represent the potential savings associated with using HANA as a replacement for a traditional database platform in several different ways.

Based On Financial Projections

The financial model Forrester created was entirely a projection. At the time of the publication, Forrester would not have had live HANA instances to draw upon.

While the customers had just begun to use HANA in this way, they were able to discuss the projected impact as they continued to expand their use of HANA as a replacement platform. In this financial analysis, we include the impact of using HANA in conjunction with the SAP Business Warehouse (BW) powered by SAP HANA, SAP Enterprise Resource Planning powered by SAP HANA, and a custom-developed application. For each case, we project the costs associated with using a traditional database platform, as well as with using HANA, and calculate the comparative TCO. We project a migration for BW in the first year of the analysis; in Year 2, the composite organization begins using HANA in conjunction with its current ERP deployment; in Year 3, the former ERP environment is retired and it is run on HANA; and, we project the costs and cost avoidance associated with developing a custom application on HANA (as compared with developing it on a traditional database platform) in the third year. The financial analysis is based on this scenario, as shown in Figure 1. The cost projections are based on information gathered through interviews and surveys, pricing information from SAP, and Forrester knowledge of database and application market rates.

Prior to HANA, customers were using traditional database platforms for their applications (both SAP and non-SAP), incurring costs associated with required hardware and software for processing and data storage, as well as the labor required for development and administration. By transitioning these applications to HANA, Forrester projects that customers are able to reduce the hardware and software required for these applications, as well as reduce the efforts for administration and development resources.”

This projection has not held up. HANA’s costs have turned out to be far higher than competing options — higher even than Oracle, which has a very high maintenance cost.

The Benefits of HANA?

This is an integral part of the study. This is a projection. It is not based on what happened to these clients.

“This is due to the way that HANA eliminates or reduces the need for a variety of associated software and hardware, and adds efficiency to the development and system maintenance processes. Many of the customers we interviewed stressed that HANA also provided significant benefit to the business. However, for the purposes of this analysis, we focus entirely on the IT impact. Our interviews with four existing customers, survey of 25 additional customers, and subsequent financial analysis found that a composite organization based on these interviewed organizations could expect to experience the risk-adjusted costs and costs savings shown in Figures 2 and 3 when utilizing HANA for this purpose. 1 (See Appendix A for a description of the composite organization.) As demonstrated by the total cost savings with HANA, there were significant reductions in the hardware, software, and labor required for each application.”

Strange Statements from Forrester

That is undoubtedly a strange statement from Forrester. HANA has more hardware than other databases because it relies upon several components. Regarding hardware, HANA has the highest hardware requirement of any database we track. One reason for this is because the database requires so much memory. The labor is also higher than competing offerings because HANA is still immature. It was incredibly immature when this study was published.

HANA could not be discounted at this time and was the most expensive database one could purchase. HANA hardware is known to be very expensive. HANA labor was and is extremely expensive. HANA eventually became a discounted item, but that was several years later.

Forrester’s Incorrect Cost Savings Proposals

“Savings. Based on the composite total cost model, the sample organization experienced the following risk-adjusted benefits (that represent those experienced and projected by the interviewed companies):

• Reduced hardware cost (simpler data footprint/simpler landscape/simpler setup). Moving to the HANA platform allows customers to reduce the volume of servers and storage required, due to the data compression, as well as efficiencies gained due to HANA’s functionality.

• Lower software costs (simpler data footprint/simpler landscape/simpler setup).”

HANA has a very complex setup and has very high maintenance costs. This is due to several factors, one being the database’s immaturity, and the second being that HANA relies on many supporting components that other databases do not. It simply has a very high TCO. Forrester’s proposals are the exact opposite of what was learned about HANA.

HANA as an ETL, Middleware & Development Platform?

“HANA also allows for a reduction and elimination in many of the software products such as ETL, data replication, management, and other middleware software. Additionally, the HANA platform includes advanced analytics capabilities such as text, geospatial, and predictive analytics as part of its platform offering without additional license.

• Faster development time (simpler application development/simpler user experience). All of the interviewees cited a faster, more efficient application development process through automation and minimizing complexity, which allowed for a reduction in resource time.

• Increased productivity for administrators (simpler processing and operations). Due to the simplification of the environment with HANA, customers are able to reduce the amount of administration time required to tune, optimize, and manage databases and servers.”

Are You Serious, Forrester, Or Pulling Our Legs?

The statement above assumes that HANA can be used as the application and reporting platform, among other things. That is a big assumption and turned out not to be true.

The other parts of this quote sound like they came from SAP marketing. HANA never ends up offering any ETL or middleware functionality. It provided no data replication functionality that was superior to any other database. HANA’s “analytics” never panned out. The HANA environment was not simplified but became well known as the most complex database environment among the competing alternatives (Oracle, IBM, Microsoft). No particular automation has ever appeared in HANA.

Did SAP Make Up Stories Case Studies About HANA from Whole Cloth?

Every single item listed here supposedly contained in the interviews that SAP made with customers and then passed on to Forrester appears to have been made up. There is no other explanation for how the “interviews” could be so different from the reality that later came out about HANA.

“Based on the composite total cost model, the sample organization experienced the following risk-adjusted benefits (that represent those experienced and projected by the interviewed companies):

Reduced hardware cost (simpler data footprint/simpler landscape/simpler setup). Moving to the HANA platform allows customers to reduce the volume of servers and storage required, due to the data compression, as well as efficiencies gained due to HANA’s functionality.”

No.

HANA’s hardware is more expensive, not less expensive.

We have this information from many HANA customers at this point. HANA does not compress anywhere near the predicted 98.5% proposed by SAP. This is covered in the article, How John Appleby Was so Wrong About HANA.

“Faster development time (simpler application development/simpler user experience). All of the interviewees cited a faster, more efficient application development process through automation and minimizing complexity, which allowed for a reduction in resource time.”

Why is any of that true?

HANA’s Environment is Simpler?

“Increased productivity for administrators (simpler processing and operations). Due to the simplification of the environment with HANA, customers are able to reduce the amount of administration time required to tune, optimize, and manage databases and servers.”

HANA’s environment is not simplified. It is far more complex than other databases. SAP’s argument may be hitched to the columnar model’s simpler data model, but there is no real evidence that columnar tables are simpler.

The sample organization experienced the following risk-adjusted costs:

“Software licensing costs. This includes the software cost associated with HANA, as well as the costs of the application software. We include initial costs as well as ongoing maintenance.

The majority of customers surveyed used full use licenses versus runtime licenses for their database software. This study has calculated savings based on runtime software license cost to better compare similar costs between HANA and potential alternatives. Using full use licenses will raise the software cost associated with the non-HANA scenarios for BW and ERP significantly; we encourage the reader to consider which type of license makes sense in the appropriate environment.”

The Logic of Using a Runtime License

Then why did Forrester use a runtime license to calculate the software cost? Runtime licenses are temporary; the full-use license should have been used. The last part of Forrester’s statement is entirely misleading. Customers need to use a full-use license, and it is well known that the runtime license is just for SAP to get their foot in the door.

“The study is commissioned by SAP and delivered by Forrester Consulting. It is not meant to be used as a competitive analysis.”

Yes, and it shows.

“Forrester makes no assumptions as to the potential cost savings that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the report to determine the appropriateness of an investment in SAP HANA.”

This is a dodge. SAP directly used this study to declare that “HANA reduces TCO.”

Conclusion

This study is paid for research, which turned out to be false on every claim. The author of the study does not contradict anything written by SAP. The claims are made by SAP and accepted by Forrester. For the study to be so inaccurate versus what we later discovered about HANA, SAP would have falsified the customer examples given to Forrester. Forrester blindly accepted what SAP told them rather than performing an analysis.

This study demonstrates that if you pay Forrester, you can get the desired result.