How Gartner Got HANA So Wrong
Executive Summary
- Gartner made many false proposals about HANA, but without declaring how much money they take from SAP.
- Forrester also provides pro-HANA information in return for SAP’s money.
Introduction
On January 11, 2013, ComputerWorld published the article Will Oracle’s Customer Base Migrate to SAP Business Suite. In this article, Gartner is quoted on the topic of HANA. Gartner got nearly everything about HANA wrong. A primary reason for this is how much SAP paid Gartner, that the Gartner analysts know very little about databases, and how effectively SAP’s analyst relationship managers presented their marketing information to Gartner. You will find the accuracy of the Gartner article with the benefit of 4.5 years of hindsight.
Our References
If you want to see our references for this article and other related Brightwork articles, see this link.
Notice of Lack of Financial Bias: We have no financial ties to SAP or any other entity mentioned in this article.
Article Quotations
Biased Information Source Says What About HANA?
“SAP’s announcement that its Business Suite applications, which are deeply embedded in many global enterprises, will now be able to run on its in-memory database technology, HANA, leaves the IT community with a very interesting decision on its hands. SAP’s announcement that its Business Suite applications, which are deeply embedded in many global enterprises, will now be able to run on its in-memory database technology, HANA, leaves the IT community with a very interesting decision on its hands. SAP is the first vendor in the market to fully combine transactional applications with analytics in-memory and has two to five years, according to some analysts, to sell to customers before other vendors catch up.”
This is restated in the article from Readwrite.
Oracle and IBM are expected to match SAP in time. Microsoft has announced that it is working on the same technology for the next version of MS SQL. “I do believe that every other vendor is going to go in that direction, but it’s going to take them two to five years to do it, which gives SAP a huge head start,” Feinberg said.
This was false because ComputerWorld and Readwrite relied upon Gartner and Forrester. What Gartner did not know and never investigated was how many areas HANA was behind all of the other database competitors. We cover one area: SAP had to add row-oriented tables to run transaction processing, as we cover in the article John Appleby on HANA Not Working Fast for OLTP.
This is perhaps because ComputerWorld also receives large amounts of money from SAP. At this point, Oracle 12c and IBM DB2 BLU have both exceeded HANA’s performance. This is covered in the article, Which is Faster HANA or Oracle 12c?
Furthermore, there is an open question as to how necessary it is to have RDBMSs with a column store — as this will always both increase the overhead of the database and because applications tend to be focused on specific things or particular types of processing. This topic is covered in our analysis in the article How Accurate with Bloor Research on Oracle In-Memory?
Notice that Gartner accepted the premise presented by SAP that their column orientation was the correct approach.
SAP Was First to Market?
“Computerworld UK spoke to Donald Feinberg, VP and distinguished analyst at Gartner, who explained the significance of the announcement for IT managers.
Until now, this has been impossible to do. The structures that are needed in a disk-based database system are different for online analytical processing (OLAP) and online transactional processing (OLTP). However, what has changed now is that you have got a database in-memory where you can create OLAP structures virtually,” said Feinberg.
“Now SAP can set up a database to do my transactions and analytics with everything virtual. People have wanted to do both in the same database for years, but have not been able to do it because of the discrepancy between the two types of data.
The magazine ReadWrite further quoted Donald Fienberg in a separate article:
“It’s the only in-memory DBMS (database management system) that can do data warehousing and transactions in the same database. That’s where it’s unique.”
Natural Questions
A few questions come to the surface.
- One is if that is a requirement driven by customers.
- Secondly, even years later, SAP customers do not perform analytics as described by Gartner.
- This is because so few companies use HANA for their ERP system. This is covered in the article How SAP Controls Perceptions with Customer Numbers. The vast majority of HANA sales are for SAP BW. BW is a database warehouse. Therefore, even with years of hindsight, the functionality that Gartner promoted does not have applicability. At least not yet. Consequently, the impact of Feinberg’s statement has been very close to zero.
A second question is whether this is all that advantageous. Companies have been using data warehouses for many years and reporting on them. It is preferable to perform some types of reporting in the application. For instance, ECC has many reports that are used. But more complex reports that draw results from many tables are most often pushed to the data warehouse. SAP and Feinberg proposed that the data warehouse and the ERP system should necessarily unify. That might or might not be a good idea, but it is not where the IT industry is headed presently.
How HANA Lags
IBM had something very similar to HANA back in 2008. Therefore, SAP was not the first. SAP was the first to market this capability heavily.
Since the publication of this article, HANA has lagged behind other competitors because SAP’s HANA design is inferior to the alternatives. This is demonstrated in the poor performance of HANA for anything but read performance on actual projects.
“I think this is going to be disruptive to other vendors, as I think it’s going to propel SAP from a database standpoint. It’s going to put them on the map – today they are number four in the market, and a small number four at that,” said Feinberg.
SAP has not changed this position since this time.
HANA for Sales Forecasting?
For SAP customers, HANA-powered applications can speed up the sales process dramatically. For example, today when salespeople for a large manufacturer takes a large order from a customer, they may not be to say on the spot exactly when the order will be fulfilled. That information often comes hours later after the numbers are run separately through forecasting applications.
With HANA running SAP’s enterprise resource planning applications – called Business Suite – salespeople will be able to take the order and get forecasting information in seconds. “This changes the way they do business. It really does,” Feinberg said. “And that’s the kind of value proposition that HANA brings to the table because of the fact that it’s an in-memory database. – Donald Fienberg
Considering the Process Outlined by Fienberg
This entire process, as described by Fienberg, is incorrect. Let us break it into individual parts.
- If a salesperson requires a data quote, that is part of the order promise. Order promising is performed inside SAP SD (Now renamed to Order To Cash) or in GATP, the order promising module in the APO suite. But this does not require the creation of a forecast on the spot. The forecast is not generated in real-time, and there is no reason for it to be. Forecasts should not be regenerated much more than monthly (in most cases) because it reduces forecast accuracy. This is covered in the article Monthly Versus Weekly Forecasting. A new order is generated (or, in this case, a quotation) and does not change the forecast in any way. A sales order decrements the forecast. Forecasts are based on previous sales orders, not on today’s sales orders. Therefore, this explanation of how forecasting works is entirely inaccurate.
- The information does not come hours later. Anyone who has worked with SAP to manage can attest that the order promise date quickly returns. It does this because it is merely checking on pre-calculated information.
- Fienberg states that the salesperson can get the information in seconds with HANA. Yet, they already get the information in seconds.
- Fienberg states that this changes how the company does business, but it doesn’t because Donald Fienberg does not understand how the order promising system works in SAP.
- It does not include the value proposition that HANA brings to the table. It has nothing to do with order promising, as described by Fienberg.
SAP’s Adaptive Server?
“You now have SAP Adaptive Server Enterprise that runs underneath the applications, and they also have HANA. I suspect that you are going to see people moving from Oracle, IBM and Microsoft over to these. They just couldn’t do it before now,” he added.”
What does SAP’s Adaptive Server have to do with HANA? The question is about HANA and Fienberg switches to SAP’s Adaptive Server, a database-oriented design. Is this article about standard RDBMSs? No, it is not.
Furthermore, SAP’s Adaptive Server, an old Sybase product, is in a state of decline.
This shows the decline of the popularity of SAP Adaptive Server as tracked by DB Engines.
Literally, what is Donald Fienberg talking about?
Fienberg’s Problematic Timeline
“Oracle today does not have a transactional, in-memory database. They have TimesTen, but none of the main enterprise application packages (SAP, Infor or Oracle’s e-business suite) run on this. I don’t believe SAP will ever certify an in-memory database for Oracle applications (when one is released) and so what you are asking is: will Oracle lose SAP customers that are running on Oracle databases? The answer is yes.”
Oracle 12c is a database that has a transactional, in-memory database (not the entire database as HANA). Placing all the databases into memory is not something non-SAP vendors do because it does not make sense. Therefore, Oracle had this capability five months after this article with Fienberg’s statement was made. Fienberg must have been in contact with Oracle and knew they were developing this. Why did Fienberg propose it would take HANA competitors “5 years to catch up?”
Now, the Second Financially Biased Source is Used
“Duncan Jones, principal analyst at Forrester, agrees with Feinberg and believes that this announcement ‘sets SAP customers free from the clutches of the other database vendors’.
He said: “If it works, as SAP plans, it will be very damaging for Oracle. Oracle has some great products, but it does a lot of things to annoy its customers. There comes a point when you can only annoy your customer base so much, and if they are given an alternative that works, many of them will embrace it.”
Like Feinberg, Duncan Jones was also wrong. SAP paid Forrester a lot of money to write a lower TCO project for HANA. This exceedingly low-quality paper is covered in How Accurate Was Forrester’s TCO Study on SAP HANA?
Oracle’s Sales Tactics are Bad, But According to Forrester, SAP’s Aren’t?
“He said: “If it works, as SAP plans, it will be very damaging for Oracle. Oracle has some great products, but it does a lot of things to annoy its customers. There comes a point when you can only annoy your customer base so much, and if they are given an alternative that works, many of them will embrace it.
“Oracle overcharges, they are very aggressive in the sales process, they don’t necessarily negotiate, they have obsolete licensing rules, and they sell customers products they don’t necessarily need. If a CIO is desperate to save money, they might see SAP HANA as a way to do that.”
And all of these things apply to SAP as well. I continuously have customers and vendors reach out to me to describe the shady tactics used by SAP in fine detail. I don’t focus on Oracle, so the same thing does not happen for Oracle. However, Forrester is a company with several hundred people. It is simply infeasible to imagine that Forrester does not have similar insight into SAP’s underhanded business practices. Yet, according to Forrester, problematic business practices are with Oracle. SAP gets a clean bill of health.
Gartner’s Feinberg Seems to Get Lost
“Gartner’s Feinberg generally agrees. He believes that for deeply embedded legacy applications, large enterprises will not consider the move – they will just wait for Oracle to come up with something similar. “Changing my DBMS platform underneath SAP is not that disruptive to the company, but changing my applications from Oracle to SAP, that’s expensive. I have to retrain everybody, I have to change all my data – it’s time consuming and disruptive.”
Right. That is true, but what is the relevance of this comment? The article’s question is whether companies will switch from Oracle to HANA.
“If I was a small company running on Oracle today and I grow and I need to replace my applications, that’s a risk for Oracle. If I have got a business case for making the transition from one set of applications to another, that will then mitigate the expense and the disruption,” he said.
This is another utterly unrelated comment from Feinberg.
Gartner Distributes Their Bad Prediction Broadly
On January 10, 2013, ReadWrite published the article SAP’s HANA Deployment Leapfrogs Oracle, IBM, and Microsoft.
Article Quotations
SAP has taken a big step ahead of rivals IBM, Microsoft and Oracle with the announcement on Thursday that its in-memory database called HANA is now ready to power the German software maker’s business applications. The pronouncement is sure to darken the mood of competitors, who one analyst says will need several years to match what SAP has accomplished.
What SAP has done is to provide one database that can perform both business analysis and transactions, something its rivals are able to provide only by using two databases, according to Gartner analyst Donald Feinberg. “It’s the only in-memory DBMS (database management system) that can do data warehousing and transactions in the same database. That’s where it’s unique.”
This was proposed repeatedly by Donald Feinberg at around this time. However, the following graphic attests that Donald’s forecasting was quite off.
It is essential to place the events in a timeline to see the prediction graphically. In this view, Fienberg’s prediction seems entirely inaccurate. Furthermore, Fienberg vastly oversold the effect of SAP’s supposed advantage with HANA.
This statement in the IBM technical documentation for DB2 BLU illustrates what Brightwork has been saying is the primary benefit of what HANA does.
Columnar organization favors analytic queries that access a large number of values from a subset of the columns and make heavy use of aggregations and joins. Only the column data needed to process the query is loaded into memory from disk.
Taking SAP’s Word for It?
Donald Feinberg continues…
Those meetings are likely to go on for a while as Oracle and other vendors race to catch up.
Does ReadWrite check anything, or do they allow themselves to be used as a propaganda apparatus by whomever they interview?
There are so many problems with this statement. The first is that it is not established that what HANA does (an attempt to get both OLTP and OLTP from the same database) is needed. Other vendors have added the same capability, but the evidence is not that this is all that necessary. But even if it were true, ReadWrite, taking Donald Fienberg’s and Gartner’s statements to mean that SAP was far ahead of other DB vendors on this capability when they were not,
Gartner’s Conflicts of Interest
Donald Fienberg was an analyst at Gartner while being a vendor relations manager with SAP, a conflict of interest.
Checking with Experienced Database Resources
Having conversations with multiple DB resources, people who have focused 100% of their careers on databases for three decades. The consensus is that no single entity verifies the claims of database vendors and performs benchmarking, etc. The vendors themselves perform all benchmarking. SAP has a single benchmark that they performed, which is only one type of database processing. We analyzed this benchmark in the article What is the Actual Performance of HANA?
The Typical Coverage Available
Examples of entities that provide database coverage include DB-Engines, which tracks the popularity of databases.
Gartner has a Magic Quadrant for databases but does not differentiate the database types in any way, as the following graphic indicates.
Gartner creates a fictitious category called ODMS operational database management systems, as it is too lazy to analyze the different categories of databases.
It places Hadoop, a Big Data database, in the same category as relational databases and the same category as every other database type.
Gartner has no lab, no testing, and very few people who even understand databases, much less touch a database, as we covered in the article How Gartner Got HANA So Wrong.
Gartner places non-relational databases into a relational database magic quadrant and does not differentiate the database in question from the vendor. Instead, they note the vendor on the Magic Quadrant; the database goes unmentioned.
Conclusion
Feinberg did not disclose his position as a relationship manager with SAP. His accuracy when competitors would come up with something similar to HANA (which ended up being better) was off by the years. His quotations at the end of the article are not related to the question in the article. He did not understand SAP’s order promising and forecasting process to explain how HANA would change it.
Overall, Fienberg and Gartner do not come across as independent entities. They come across as paid spokesmen for SAP. In the articles where Fienberg is quoted, there is no mention that Gartner is not independent of SAP but receives probably somewhere around $100 million to $150 million from SAP annually. Secondly, Fienberg’s statements on HANA are undifferentiated from the SAP talking points. Therefore, it looks like Feinberg was coached on what points to make by SAP and that there was no processing of the information or attempt to validate whether it was true. These historical analyses are essential because Gartner routinely faces plants in their predictions. They face plant the same way in each case, mispredicting due to the entities that pay them.
The database category of software is filled with vendors making all claims, but no entity verifies any of these claims. This is a problem because it means buyers in the database market must perform their testing.
This would mean accessing the databases and creating a laboratory environment including all the skill sets. Very few companies do this.