Why it is Likely That HANA Underperforms SQL Server
Executive Summary
- We have obtained reports of SQL Server outperforming HANA.
- We analyze the hardware specification differences that lie beneath SQL Server versus HANA.
Why are IT departments not noticing that HANA’s performance is so unimpressive versus dramatically less expensive databases?
This article was updated as of February 2021.
Video Introduction: Why it is Likely That HANA Underperforms SQL Server
Text Introduction (Skip if You Watched the Video)
In the field reports of SQL Server’s performance versus HANA, several customers have been surprised that HANA does not seem to outperform SQL Server for performing similar tasks. But one item is often left out in the conversation in performance comparison, which is the hardware difference between what is provisioned for HANA versus what is normally equipped for other databases. SQL Server will normally have a slim or low-cost hardware provision. However, even with much less hardware, HANA is routinely reported to us as not outperforming SQL Server. You will learn the challenges in communicating the issues with HANA to IT decision-makers who frequently have trouble segmenting the different dimensions that result in various performance benchmarks.
Our References for This Article
If you want to see our references for this article and related Brightwork articles, visit this link.
The Hardware Used by SQL Server Versus HANA
SAP has their customers purchase the most extensive possible hardware specs for HANA. It is impossible that SQL Server had anywhere near the same spec as the box that was used for HANA. It follows logically that HANA significantly underperforms its hardware specification. We addressed the inability (or lack of interest) by IT directors to account for the percentage of the performance boost for hardware as we covered in How Much is Hardware Responsible for S/4HANA’s Benefits?
Ahmed Azmi offers the following take on this topic.
“The only way to compare the performance of 2 DB systems is to run them both on the same hardware. If you run HANA and SQL Server on the exact same hardware, SQL Server will outperform HANA by a wide margin. Give me any database system and I can boost its performance by an order of magnitude simply by upgrading the hardware. If you spend a million dollars on memory and processor acceleration, you can make any database faster. It’s just a matter of throwing more hardware (money) at it.”
As for “in memory” type processing, Microsoft added this to SQL Server back in 2016, as shown in the graphic below.
What is HANA?
HANA is two things, and a third thing that is an emergent property of the first two things:
Item #1: Hardware
HANA always involves new hardware and a much bigger hardware footprint than what preceded it, using both high-end RAM and SSDs to push out any spinning disks.
Item #2: Software
This is a database designed with a column-oriented data table store customized for reading access (i.e., for analytics). There is a lot of false information proposed by SAP, and at one point, SAP claimed the entire database was column-oriented, but HANA is a combination of column and row store.
Item #3: Hardware & Software Interaction
The interaction between the column-oriented data and the memory. This is a well-known feature of the interaction between memory and data stored in columns, which we cover in the article How HANA is Such a Fast Database (For Analytics).
But a major issue, which should be obvious, is that SAP is a software company and makes none of the hardware that HANA uses. However, SAP routinely takes credit for performance increases that don’t have anything to do with SAP’s software. And they have nothing to do with the work or innovation of any other database vendor either.
SAP is the most aggressive in combining the hardware improvement story with the changed software (the database) and allocating all of the credit to SAP or the software vendor.
Yes, Adding More Expensive Hardware Increases Performance
Also, this is not to say that any system or database placed onto more expensive hardware will not improve in speed.
That is always the case.
And here arises the problem in differentiating just what the hardware “bump” is doing versus the overall HANA “bump.” HANA is a combination of hardware (RAM and SSD replacing spinning disks) with a column-oriented table design.
Our Interaction with HANA’s Performance
Recently we received the following message on HANA performance from someone with first-hand experience with HANA.
I rather like HANA. We migrated from Oracle to HANA… and I like it very much, although I think it would be so much better on different hardware than what we have.
There was particular hardware that was purchased that our contact would have preferred to be from another vendor. We won’t get into the hardware as that would be a different article.
Our contact when on to laud HANA’s performance for planning runs. The software that was residing on HANA, in this case, was supply chain planning software. And our contact reported a 30 to 50% improvement in planning runtimes.
Isolating the Performance Benefit of the Hardware from the Performance Benefit of the Software
Upon considering this improvement, I began to think of what we know that HANA improves regarding performance. And its primary benefit is read access.
Read access is very intensively used in analytics, however, with a planning system. The load is to the memory and the processor for a planning run.
Therefore the question is, why would HANA’s column-oriented design improve the performance of this type of processing?
I asked if it was not likely that the performance was not due to the columns oriented design (which only provide a performance benefit in reading operations) but is instead due to the hardware (the extra RAM and SSD?)
Our contact commented that
Performance management in the past consisted of optimizing the data flows between CPU, memory, and disk. With in-memory computing, disk I/O has become irrelevant and things that plagued us in the past like disk-layout, RAID configuration, I/O hot-spots, redo-log I/O performance, concurrent/direct I/O and tricky OS settings like IBM’s I/O queue depth have all gone away now.
But we would point out that these things are real when any system is moved to being entirely in memory.
That is when the more expensive hardware is added.
HANA’s Efficiency Level with Hardware
There is a lot of discussion by SAP on how HANA is so fast. But there is very little information around how well HANA addresses the hardware that it is given. SAP stated that it had developed the fastest database globally and done so on the first version of HANA. But then, HANA has been continually “innovated” since its introduction. HANA is known as one of the most patched databases. This, along with several other factors, has led to HANA being maintenance-heavy, which is accounted for in the Brightwork research A Study into SAP HANA’s TCO.
Information from some sources indicates that HANA requires significantly more hardware than competing databases to return similar performance results. This would suggest that HANA was not designed and competing databases regarding how it addresses hardware resources. And these data points that are available to us are recent, so it can be argued that it was because HANA is in its first version. But let us remember, SAP claimed that HANA was the world’s fastest database in its first release. This performance promoted Jon Appleby of Bluefin Solutions (an SAP consultancy) to state that…
“Oracle if finished on SAP,”
..as we coved in the article How SAP Has Quietly Changed its Strategy on HANA and Oracle.
This fits into comments that SAP uses regarding HANA’s “innovation.”
SAP has converted the term “innovation” into the frequent updates required to patch problem areas, basically to try to catch up to other databases.
Why Hasso Would Like Customers to Upgrade their Hardware
SAP, and Hasso Plattner, in particular, have made a significant point of emphasis that companies need to upgrade their hardware. In four books on in-memory computing, Hasso has stated that companies don’t understand how beneficial it is to have all of this performance. However, it should be remembered that all of this advice regarding adding hardware is within the context of adding expensive hardware to SAP in particular.
However, the observations cataloged up to this point demonstrate that HANA is not capable of using more expensive hardware than competing offerings. Furthermore, the inefficiency with which HANA can do things like utilize processors is relatively weak. Therefore, the question arises whether SAP prompts customers to buy such high hardware specifications to compensate for this issue.
Hasso and SAP Oversimplifying
Furthermore, Hasso and SAP are providing an oversimplified perspective on hardware, speed, and decision making. Hasso has stated that decisions in health care, surgeons must be made instantaneously, and HANA will be part of that. This is untrue, as is covered in the article How Accurate Was Hasso Plattner and Fortune on HANA?. Even in an ER environment, the orientation is to stabilize the patients. Scheduling operating rooms and the right personnel for specialized procedures takes weeks. There is a whole chain of events that take place that the database will not speed up much. I happen to perform extensive forecasting testing, which is very processing intensive. But I can simulate a forecasting scenario for a large company’s data set using just a well-powered laptop. I think it cost me around $1500 at the time. The software I use makes excellent use of relatively inexpensive hardware.
Many examples could be provided. And for those that are less interested in the details per se, the result is that more complexities go into the performance equation than what SAP presents to corporate buyers.
Introduction to HANA Development and Test Environments
Hasso Plattner has routinely discussed how HANA simplifies environments. However, the hardware complexity imposed by HANA is overwhelming to many IT departments. This means that to get the same performance as other databases, HANA requires far more hardware and far more hardware maintenance.
Development and testing environments are always critical, but with HANA, the issue is of particular importance. The following quotation explains the alterations within HANA.
“Taking into account, that SAP HANA is a central infrastructure component, which faced dozens of Support Packages, Patches & Builds within the last two years, and is expected to receive more updates with the same frequency on a mid-term perspective, there is a strong need to ensure testing of SAP HANA after each change.”
The Complications of So Many Clients and Servers
HANA uses a combination of clients and servers that must work in unison for HANA to work. This brings up complications, i.e., a higher testing overhead when testing any new change to any one component. SAP has a process for processing test cases to account for these servers.
“A key challenge with SAP HANA is the fact, which the SAP HANA Server, user clients and application servers are a complex construct of different engines that work in concert. Therefore, SAP HANA Server, corresponding user clients and applications servers need to be tested after each technical upgrade of any of those entities. To do so in an efficient manner, we propose the steps outlined in the subsequent chapters.”
Leveraging AWS
HANA development environments can be acquired through AWS at reasonable rates. This has the extra advantage that because of AWS’s elastic offering, volume testing can be performed on AWS’s hardware without purchasing the hardware. Moreover, the HANA AWS instance can be downscaled as soon as the testing is complete. The HANA One offering allows the HANA licenses to be used on-demand. This is a significant value-add as sizing HANA has proven to be extremely tricky.
Conclusion
One cannot observe HANA’s benefit without taking into account the more expensive and more modern hardware that HANA brings along with it. SAP proposes that its combination of column-oriented tables and more recent and costly hardware leads to several performance benefits. However, adding more expensive equipment to any solution does not require SAP. That is high-speed RAM, and SSD can be purchased without contacting SAP. It can be installed without contacting SAP. And the performance will improve. Furthermore, switching out old hardware for new hardware is one of the fastest ways to improve performance. Not only do you not need SAP, but you also don’t need Deloitte or Accenture and other HANA devotees to enable this.
As a control, one must know the benefits of HANA versus adding more expensive hardware to an existing solution and not employing HANA. Companies that allow SAP to take credit for what is a premium hardware configuration are fooling themselves.
Furthermore, the hardware component is the lowest cost portion of HANA.
And…
- Migrating a current solution to new hardware is the lowest effort of migrations.
- It does not require a reimplementation of software.
- It does not require purchasing a new database license from SAP.
- The fastest migration is the data points we have indicated that a HANA implementation takes on average 1.25 years. But a real hardware upgrade is a small fraction of that time.
Why put off what will be, in many cases, over 1/2 of HANA’s benefits for 1.5 years?
A substantial portion of HANA’s overall benefit can be obtained by merely installing HANA type hardware with the existing database.
It isn’t easy to see how HANA can match the SQL Server’s performance, given HANA’s hardware specification. What is impressive about this is that HANA is the most expensive database of any of the databases it competes against. SQL Server is considered the “budget” commercial RDBMS. SQL Server would further outperform HANA at this large price discrepancy as its low cost would leave more budget to have an even larger hardware specification than HANA.
However, this is not the story; you will hear if you listen to SAP or SAP consulting firms. Yet, in the over nine years since HANA’s introduction (this article was updated in September 2019), SAP has to produce a benchmark that validates SAP’s claims. This is covered in the article Issues with SD HANA Benchmarks. SAP to cover up this issue, a new benchmark was created, as we cover in the article The Hidden Issues with BW EML Benchmarks.
Observe the claims of performance superiority in the following video and our analysis of each point.