Our Evaluation of the Information Quality Level on Tableau ROI from Google Search Results

Executive Summary

• We review some of the highest-ranking sources on Tableau ROI to answer the Tableau ROI’s quality level.

Introduction

We review some of the most popular articles on Tableau ROI to understand what is generally available to those searching on this topic. Our own site has several articles that rank high for a search on “Tableau ROI,” however, we will leave our own articles out of this analysis.

See our references for this article and related articles at this link.

Article #1: Tableau’s Total Economic Impact Report Tableau

The first thing that became immediately apparent is that Tableau itself dominates the ROI search results for “Tableau ROI.” This is rare and also not a good sign as vendors cannot be trusted to communicate accurate information about the ROI of their application.

The first article moves into a study by Forrester.

Article #2: Forrester’s Tableau Total Economic Impact of Tableau

On the second page, Forrester states the following.

Forrester Consulting provides independent and objective research-based consulting to help leaders succeed in their organizations.

Forrester is constantly producing rigged sponsored white papers. So how is Forrester independent or objective? This is a constant issue that IT analysts have eliminated the validity of the terms independent and objective.

Forrester’s Study Into Tableau ROI

Tableau commissioned Forrester Consulting to conduct a Total Economic Impact™ (TEI) study and examine the potential return on investment (ROI) enterprises may realize.

The purpose of this study is to provide readers with a framework to evaluate the potential financial impact of Tableau on their organizations.

Tableau Marketing?

Salesforce owns tableau, and Forrester has been Salesforce’s “go-to” analyst for concocting fake ROI benefits. In these reports, I have seen constant marketing phrasing placed in Forrester studies/white papers. A good example of this is the following.

Tableau customers experience a shift to a data-centered culture and increased data literacy companywide.

Employees increasingly make decisions regarding both major strategic choices and minor operational changes using data to inform their choices.

Prior to using Tableau, the customers struggled to put timely and relevant data into the hands of business users, present dashboards that enabled quality decision making, and allow managers to sort out salient data points from mounds of reports. In Forrester’s interviews, one executive said:

“As we looked at the massive amounts of data that we had across the enterprise, we had to face the questions: ‘How can we get a look across our entire company? What are we going to use? Are the current tools we are using capable of providing what we need? How do we help decision makers gain insights to manage the business rather than just process routine data?’”

This is not the type of paragraph an IT analyst would write. And this is Forrester speaking to a customer provided to them by Tableau that Tableau selected for them.

Enormous Claims

Forrester goes on to make a series of enormous claims about Tableau.

Accelerated speed for evaluating and making decisions worth $2.2 million. Using Tableau reduced the effort for data collection, organization, and evaluation by an average of 30 minutes per day per user. Each customer Forrester interviewed reached thousands of users over several years, making a productivity impact across their organizations.

Increased efficiency of business operations valued at $1.9 million. Almost every customer Forrester interviewed experienced an impact on the effectiveness of business. Some customers realized increased revenue, greater customer retention, reduced cost of fuel for a transportation company, or (as in the example on which this benefit is modeled) the reduced cost of depreciation for a company managing a complex fleet of vehicles. One executive said, “Shining the data visualization spotlight on existing problems can easily highlight issues that result in huge financial results.”

Avoided $65,308 in costs of alternate software products. As adoption of Tableau increased, the companies were able to retire subscriptions to software tools that became redundant or were superseded by capabilities within Tableau.

Tableau is just another of many visualization analytics applications. At Brightwork Research & Analysis, we tested Tableau and would not recommend using it. First, Tableau is one of the more expensive analytics applications to use. Tableau was hot and sort of made the analytics market several years ago but lost its differentiation, and low-cost competitors came in that could do most of what Tableau could do, but at a much lower price. One is Microsoft’s PowerBI. But our preferred product is QuickSights by AWS, which is very low in price and far easier to use than Tableau. In fact, this loss of differentiation is why Salesforce could buy Tableau at a discounted price.  

Knowing Tableau and knowing the competitors to Tableau, it isn’t easy to believe anything Forrester is writing.

This is Forrester’s projection of the ROI of Tableau. Here it is less than 3 months. This is not credible.

Forrester Disclosure

Observe this disclosure by Forrester.

DISCLOSURES
Readers should be aware of the following:
This study is commissioned by Tableau and delivered by Forrester Consulting. It is not meant to be used as a competitive analysis. Forrester makes no assumptions as to the potential ROI that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the report to determine the appropriateness of an investment in Tableau. Tableau reviewed and provided feedback to Forrester, but Forrester maintains editorial control over the study and its findings and does not accept changes that contradict Forrester’s findings or obscure the meaning of the study. Tableau provided the customer names for the interviews but did not participate
in the interviews.

This is where Forrester is saying, “yes, Tableau/Salesforce paid us,” but had absolutely no outcome on their report, which is not possible. Forrester knows that it has to produce a strong ROI and projected benefits or not have Tableau/Salesforce as a customer.

In the last part of the quote, customer names were provided by Tableau. What is the problem with this? Tableau may have 2500 customers. They can select just a few of their happiest accounts and provide them to Forrester. In fact, Forrester states in the report that they only spoke to 8 customers. This is an infinitesimal sample of Tableau’s customer base. Observe the companies listed by Forrester.

1. Internet commerce
2. Supply chain services
3. Technology products
4. Financial services
5. Travel and entertainment
6. Transportation
7. Financial services
8. Internet commerce

So they did 8 interviews.

This selection or cherry-picking entirely undermines the representativeness of the typical results of Tableau. Forrester states in the green highlighted part of the quote above that other companies could not expect the same ROI. Yes, well, how could it? The vendor cherry-picks the sample!

Article #3: Tableau’s 7 Ways Finance Teams Drive ROI With Tableau

This article is less of an article than a main page on Tableau’s website. It lists a series of disjointed benefits.

Which segments are most profitable? Where is high growth occurring? What are your challenges and opportunities? By placing all product segments in dynamic quadrants, CFOs can define winners, losers, and areas of opportunity with Tableau.

This CFO dashboard provides an overall picture of yearly performance over the past three years, as well as key financial metrics such as net sales, net profit, and net profit margin.

This implies that Tableau can only do this. Again, I do not see Tableau as differentiated from other visualization applications. And at many Tableau customers, I have noticed Excel used far more frequently.

Kimpton Hotels and Restaurants, operator of 65 unique boutique properties across the U.S., needed a better way to report and forecast future revenue opportunities and risks. Within Kimpton, there were historic issues with fragmented data and reporting structures, so managers and leaders across the organization struggled to report and glean insights from forecasts in real-time.

Is it true that Tableau is a better application to use for forecasting than a forecasting application? That isn’t easy to believe.

Article #4: Tableau’s Value of Tableau

There is not much to this article. This is just another Tableau marketing page.

Article #5: Tableau’s Forrester ROI Report

This is another Forrester ROI report from 2010. Here are some quotes.

Tableau Software announced today that an independent study of its customers’ ROI by Forrester Consulting showed a 127% ROI using Tableau’s business intelligence products, an exceptional result.

“Tableau decreases BI support costs and drives end-user empowerment.” The complete Forrester study, titled “The Total Economic Impact of Tableau Business Intelligence Solution” (November, Increased user adoption of business intelligence

Increased user adoption of business intelligence
Increased standardized reports by 400%
Reduced report creation times by 87.5%
Reduced report distribution, storage and duplication costs and simplified publishing of reports
Incurred no training costs to adopt Tableau
Freed up BI analysts to perform higher-value tasks
Realized process improvements from use of visualization in root-cause analysis

And none of this means anything. The study was paid for by Tableau, and Tableau provided all the samples.

Article #6: TeknionUSA‘s The Case for Tableau Training How ROI Justifies the Expense

This is not an article on ROI. It is about training for Tableau.

Article #7: Visualize ROI The Only Way to Win Deals, Crisis or No Crisis, is to Communicate Value

This is also not an article on ROI. It is a sale page for some company called VisualizeROI.

Article #8: Better Buys ROI Business Intelligence

This is a content-free article. Here is a quote.

Your company’s data has gotten to a point where Excel spreadsheets simply aren’t cutting it anymore.

You need something more to help organize and analyze the information you’re accumulating.

This is amusing because spreadsheets don’t go away when analytics software is implemented. This is a repeated claim made by analytics vendors.

Article #9: Accenture The ROI of AI

This is not an article about the ROI of Tableau; it has nothing to do with Tableau and is about the ROI of AI. The word Tableau does not appear one time in this article.

How this article came up this high in the search results is a mystery.

Article #10: Forbes’s Determining The Return On Investment On A New Software Purchase

This is not an article about the ROI of Tableau; it has nothing to do with Tableau and is about the ROI of any software purchase. The word Tableau does not appear one time in this article.

The same comment applies to this article as the previous article. However, this article was terrible.

The Logic for Stopping at This Point

There was really nothing new coming from any of these articles. Not a single one scored above 1 out of 10 and the results were coming back as not even related to Tableau. And as we looked at more of the articles down the list in the search results. It didn’t seem as if there would be anything else coming down the pike if we continued to read more articles. So at this point, we decided that this would be where we would stop.

So now, let us get into the scores.

Honorable Mention: Datanami’s Companies Struggle to Find an ROI on Analytics

This article is not included in the scoring. However, I include it because of how distinct from the 100% financially biased articles that I just reviewed.

Executives across every industry are looking to big data analytics for a competitive advantage. But all too often, analytic investments fail to deliver the goods as promised, according to a recent survey of return on investment (ROI) of sales and marketing analytics.

Last month, ZS Associates released the results of a study of about 450 senior sales and marketing executives for U.S. companies. The study, which was conducted by the Economist Intelligence Unit, found that, while 70 percent of respondents find analytics is either “very” or “extremely important” to their own business, only 2 percent say the analytics have resulted in “broad, positive impact.”

This is important because both Tableau and Forrester are financially biased. However, this study is conducted by a group that does not have a financial bias to promote analytics products. Observe how distinct this study’s outcome is from some rigged Forrester study.

Other studies have found more widespread success with analytics. In its “Betting on Big Data” study released last September, Teradata (NYSE: TDC) reported that about two-thirds of companies reported solid results from their investments, while about one-third had yet to find a satisfactory return on their investment in analytics.

However, even in the Teradata study, the average return gained from analytic investment was on the order of 1 to 3 percent. Those may not be the type of earth-shaking numbers that one would expect from a technology that receives so much attention. But the Teradata numbers do reflect the fact that most of the companies involved with the study were large corporations, where even small fractions can translate into big dollars.

That is not how percentages work. If the percentage is low, the size of the base is not germane. Remember that the company most likely has other investments that return more than an average of 2%.

The question that should be asked is why the return is so low.

The Table of Our Scores

The following is a tabular form of the articles we just covered.

Conclusion

Brightwork reviews the top ROI search results for multiple products and multiple software categories. This was the worst scored ROI analysis we have ever seen.

The average score of the 6 articles reviewed was 1. Three of the articles were not relevant, so they have a score of N/A.

What Did We Expect?

We did not have great expectations of what we would find before performing this analysis. However, the results are far worse than we would have guessed.

It is difficult to say that any of these articles is worth the time to read. The only way it would be is if you were entirely new to Tableau ROI.