Executive Summary

• This article explains the advantages of a unique independent research product that is more focused on real-world usage.
• You will learn the advantages versus case studies.

Introduction

Application ratings are the most widely used research in the software space. There is Gartner Magic Quadrant, the Forrester Wave, G2Crowd, and many others.

One might say the area of software rating is saturated, with the Gartner MQ being dominant, and it is expensive and time-consuming to get into the Leader Quadrant.

On IT Analyst Overpromsing

This topic of overpromising is one of the most interesting in how IT analysts to market to software vendors. It is not specific to the topic of this article. However, we cover how Gartner overpromises what its MQs can do for companies in the leader quadrant in the article How IT Analyst Firms Like Gartner and Forrester Overpromise Marketing Exposure to Vendors.

• There is a lot of debate as to how influential the other application rating products are. For example, when vendors reach out to us, they don’t tend to talk about other application ratings.
• However, there are other aspects outside of just a rating that interests prospects. And one opportunity for vendors to differentiate themselves is how their software is implemented.

Normally, this is considered to be appropriately covered by case studies written up by the vendor and hosted on their websites. Yet, that is only one way of presenting implementation experiences. When we canvased the available offerings at all the IT analysts and related companies, we did not find a research product that provided the type of coverage of the research product that is about to be discussed in this article.

• This article makes the case for a new research product. However, we don’t cover our product until the end of this article. The first part of this article is our research into case studies.
• You will also learn a lot about case studies, their effectiveness, the guidance available on writing them.

This article is broken into the following sections.

• Section 1: Important Features of Vendor Case Studies
• Section 2: How Case Studies Tend to Be Developed at Vendors
• Section 3: What Are Views of Some Decision Makers on Case Studies?
• Section 4: Reviewing The Common Advice on How to Write Case Studies
• Section 5: Our View on Vendor Case Studies
• Section 6: How This Brightwork Research Product Differs from Vendor Published Case Studies

Section 1: Important Features of Vendor Case Studies

Now is an appropriate time to highlight some of the basic features of case studies. Everything that follows is with respect to vendor case studies, an important distinction, as case studies can apply to any area, so this list is only for vendor case studies.

• Case studies are vendor-biased accounts of previous project implementations and outcomes presented in a one-off format.
• Case studies have two perspectives, the vendor and the subject or customer. There is no outside analysis.
• Vendor case studies almost always are written from one point of view.
• Case studies are plentiful on vendor websites, and most vendors have them.
• Case studies are not a research product, they are interpreted as marketing output.

Let us review a few examples of vendor case studies.

See the following screenshot.

This is a standard vendor case study, and it is on the vendor’s website and is written by the vendor. 

You can read the AWS Case Study to garner your own impressions. Indeed, you will have seen this format many times.

Reviewing One of the Best Case Study Websites

Google Cloud provides the best case study websites that we have seen.

So let us review their case study approach.

Google Cloud’s Case Study Overview Page

Google Cloud presents its case studies in a very appealing format. This is consistent with other Google pages we have reviewed which we think have an excellent UI and give a very clean and positive impression to the website visitor.

Google Cloud has a very large number of case studies at their site. They are well written and the case study area has an appealing design.

Now, let us take a look at a sample Google Cloud case study.

Google Cloud has enormous resources to produce what looked to be around 1,300 case studies on their website.

Google Cloud also has a powerful Google brand working for it, which makes the case studies seem more credible than a typical vendor’s website. Google has lost substantial credibility with users due to how they censor their search engine and YouTube, but the brand for corporate services is still probably one of the best out there.

Google Cloud’s case studies are strongly differentiated from other case studies on the websites of other vendors in its number of case studies and in its design and presentation.

However, what is not differentiated is the individual case studies themselves, as they are essentially similar to the other case studies we reviewed at the websites of many other vendors. That is, Google Cloud uses the same fundamental approach as the case studies at other vendors.

Back in 2008, Formpipe’s ambitious product expansion plans brought the company to a point where its development capacity had to be drastically increased. Thus, they started seeking out a long-term nearshore supplier.

We started working on Platina with product technical review and competitive analysis providing a list of recommendations for improving the information system. Gradually the team has taken over the full-cycle development of the Enterprise Content Management product and now delivers a new major version of the product and a new service pack every year. – Sigma Software

And this excerpt is from a case study at CERN by Google Cloud.

Straddling the border between France and Switzerland, thousands of researchers are using some of the biggest, most complex scientific instruments in the world to examine the smallest particles in our universe. The European Organization for Nuclear Research (CERN), based in Geneva, is one of the world’s largest and most respected research centers, funded by 23 member states.

CERN has already redesigned some of its architecture to run on Kubernetes, the open source system developed at Google. Container-based systems such as Kubernetes, which focus on portability of services, allow IT infrastructures to deploy more quickly and scale more easily than traditional architectures. This made it relatively simple to incorporate Google Cloud for these investigations.

The core of the cloud deployment being evaluated through this project is Google Kubernetes Engine (GKE). One of the main features of GKE is the ability to scale up its clusters to very large sizes very quickly. GKE can automate administrative work, reducing administrative overhead and autoscaling clusters up and down as needed. Meanwhile, the addition of Preemptible VMs to GKE can provide flexibility in the way that workloads are handled. Preemptible instances are short lived and easily repeatable, making them ideal for small, repetitive tasks. – Google Cloud

Topic Transition

And this leads to the topic of how vendors develop their case studies.

Section 2: How Case Studies Tend to Be Developed at Vendors

Many vendors create their case studies without spending much time evaluating the case studies of other companies or their competitors. What follows are some observations about the many case studies we read before developing this research product and before writing this article.

Observation #1: The Overriding Similarity of Case Studies

We just discussed the similarity between Google Cloud’s case studies and the case studies of other vendors. However, we found that the vast majority of the case studies at each of the vendors we evaluated were essentially similar.

Even case studies from an extremely well-funded company, like Google Cloud, appear undifferentiated in terms of just the written content than some of the smallest vendors.

Observation #2: How Video Enhances Credibility

Videos embedded in the case studies were very effective, and they made the case study seem more real. For one reason it showed a higher commitment on the part of the customers of the case study as getting them to appear on a video set is a higher level of commitment than just offering some quotes.

Observation #3: The Problems in Comparing Case Studies

We constantly ran into problems with comparing the details of the different case studies. And beyond just reading different types of case studies, we reviewed multiple case studies within the same category of solution at Google Cloud, and still could not find many important points of comparison. The way the Google Cloud case studies were written placed all of the analytical efforts onto the prospect, who would have had to extract information and create their own matrix from the case studies. This observation was generalized to other vendors.

We have concluded this is a universal feature of case studies. The case studies we reviewed were unstructured, with some information appearing in one area of one case study and that same type of information being in another part of a different case study even for the same vendor. There is no attempt to have a consistent location of similar data or to even obtain similar data between case studies.

• All of this makes case studies difficult to evaluate as part of a coherent whole with other case studies from the vendor, restricting pattern recognition.
• This is because each case study is only designed to be evaluated individually. This is a wasted opportunity.

Topic Transition

This topic of case study design led to the question of how vendors educate themselves to write case studies.

How Prevalent are Books or Academic Research on Writing Case Studies Used by Vendors?

One of the important topics is how vendors become educated on how to design case studies. And this comes down to the available published material on the topic.

However, something that we found surprising is that there are few books written on how to write case studies or how they work.

Source Check #1: Information From Books

The book above is one of the few we were able to find on Amazon.com on the topic of case studies. And it is a book on using case studies for research it is not a book that resources in marketing organizations would tend to read.

Source Check #2: Academic Research

After reviewing the available books on case study design, we reviewed the academic literature. You can see the screenshot of our search results from an academic search engine.

When searching on the topic of case studies in academic search engines, we had no problem finding a number of results.

However, what we found is that the results either dealt with research that was itself filled with case studies or were articles about how to use case studies for research purposes.

A quote from a representative article in this second category…

…the tendency to value global theories and practices that transcend context over those that embrace contextual effects remains. By this, we refer to positivism and post-positivism, their ontological and epistemological assumptions of objectivity, use of qualitative methods to test hypothesis and focus on replicability and generalization. – The Curious Case of Case Study Research

The vast majority of academic research articles we found on using case studies are not particularly useful to vendors. Case studies are used in every field, and in only a minority of articles did they have anything to do with helping to market a product or service. For example, we found a lot of articles on the use of case studies in the field of nursing.

Due to all of the aforementioned factors, it is unlikely vendors are getting much guidance on what works and what doesn’t from academic papers on case studies.

Source Check #3: Online Videos

Normally we like to include videos in our articles when appropriate. Yet, after reviewing the YouTube videos on case studies, we could not find anything related to software case studies that were of sufficient quality to want to include.

This video is excellent. In fact, it was the best one we could find on case studies. But, it is not about software or business but is a series of case studies on intelligently designed neighborhoods. But it does show the power of case studies. 

A Summary Of Our Research into Case Study Material

The lack of available material indicates that vendors are mostly merely copying the way that other case studies they see are written. That means that without a firm research basis of what works, the case studies writing is based upon other case studies. This might explain why so many case studies seem to be so similar?.

Topic Transition

And the question of the availability of research into case study design introduces the next topic which is determining the effect of case studies on readers.

Section 3: What Are Views of Some Decision Makers on Case Studies?

Another area of case studies that could use much more research is how case studies are the following…

• The interpretation of case studies by readers.
• The impact case studies have on executive decision-makers.

If we wanted to, we could easily add very significantly to the research in this area. And a major reason is there is so little existing research published on the topic. While the research in this area is puny, there are a very large number of articles on how to develop good case studies.

The following quote is one example of this.

To get the attention of prospects today, marketers have to demonstrate their product or service’s value — not just talk about it. Industrial and other B2B buyers today are cynical about sales hype, and they don’t have time to interpret marketing messages that aren’t direct and relevant. That means that instead of aimlessly “banging the drum” about how great your company is and how terrific your products are, you have to share the real-life experiences customers are having with you and your products. The best way to do that is with case study.

Case studies tell a business success story, walking your audience through a real-life scenario they can relate to. Readers get to “meet” their peers (the customer you’re highlighting), see challenges similar to those they face, understand your company’s approach to solving a problem, get a glimpse of how you applied your expertise and, finally, what your product/solution did for the customer – saved money, improved productivity, reduced downtime, accelerated production…whatever results you have to share. Just like a good novel, there’s a distinct beginning, middle and end, and a hero that’s at the center of the story – your company. – Wiedert

As many articles on the topic of case studies, the writing is confident, but there is little reference in these articles to how the authors know that a particular case study or case study design is effective. So there is no research basis supporting the imparted knowledge.

Performing a Mini Survey on Case Study Interpretation

Because we found very little published material on this topic, we decided to do our own mini-research project on case study effectiveness. It is important that we not confuse you on what we think of the importance or popularity of case studies with prospects. Case studies are popular with prospects. This is a question of how one knows a case study is effective.

What We Wanted to Find Out From Our Mini Survey

We surveyed members of our network who are in decision-making positions that can influence corporate software purchases in companies. We wanted to find out how these people viewed the credibility of vendor case studies. Here is a sampling of some of the responses we received.

Quote #1

Case studies (from vendors) can be helpful, however, they are always positive. I have to say I take them with a grain of salt. I know they are designed to sell me, so negatives end up left out.

I would prefer to see more about some of the challenges they faced on projects. The discuss after the conclusion of the project, but not the path to get there.

Quote #2

I tend to scan them (case studies) rather than read them in full. I find them often overly long.

Quote #3

We read them (case studies), but for whatever reason they don’t seem to end up as a discussion point among the decision makers in our organization. But it is very important to see that the vendor has been successful with their previous customers.

Quote #4

I am interested to see how the software was connected to what we do. But I am often sent case studies by a sales rep that are not a fit for us.

Our Analysis of the Responses

These are quotes and observations we think any vendor should consider when they think through how much their case studies are helping to convert prospects. If you look online, case studies are broadly proposed as effective by sources on the topic.

Yet, some of the things we were able to find in our small survey on case studies were not in the coverage of case studies. This shows that a full spectrum of understanding cannot be the primary focus of those that write articles on case studies.

Topic Transition

And this leads to the topic of the advice given on how to write vendor case studies.

Section 4: Reviewing The Common Advice on How to Write Case Studies

Reading the standard literature on creating software case studies was curious in that none of these observations in our mini-survey on case studies were generally listed as issues in articles on the topic of case studies.

We can’t say for sure, but a potential reason for this is that when the vendor writes the case study, it is difficult to get around the limitations of the standard way vendor case studies are written and presented. The articles we were able to find discussed how to optimize working within the limitations of the construct of a vendor-generated case study. But they did not appear to be interested in addressing the natural limitations of the approach.

Section 5: Our View on Vendor Case Studies

Now that we have provided background, we will cover our view of vendor case studies. Our view is that case studies are most effective if they meet the following criteria.

Effectiveness Criteria #1: The Case Study Appears to be True

Case studies often come across as inauthentic and present an overly one-sided or overly positive description of what occurred on the project. This is one of the biggest dangers of vendor-produced case studies, that they come across as purely marketing collateral.

• Vendors don’t appear to consider having a third party provide coverage of case studies. There is an obvious bias of case studies in both how they are entirely written by the vendor and then presented from the vendor’s own website.
• Vendors often don’t realize that this means the information is “discounted” to whatever degree by the prospect.

Effectiveness Criteria #2: The Case Study is Relatable to Prospects

Vendors are normally trying to sell their solution to more prospects than match their customer base. This means that prospects must be able to see themselves in customer case studies that are not very close to their industry, company challenges, etc. In the example of Google Cloud, they have enormous numbers of case studies, but most vendors don’t have the opportunity to create that many case studies, which can allow a prospect to find a very similar company and implementation themselves.

• We found that vendors generally understand they need to send the most applicable case study they can to a prospect.
• However, the part they miss is that how the case study information is organized is a major determining factor in its perceived applicability to the prospect.

Effectiveness Criteria #3: The Case Study or Studies are Differentiated From Competing Vendors

The following are features of vendor case studies.

1. Ubiquitous Case Studies: Nearly all vendors have case studies, and therefore when a prospect compares different vendors, they generally see the same type of case studies. We addressed this earlier in the article.
2. Interpreted From Who’s Perspective?: From the perspective of the vendors, their case studies seem differentiated. However, this interpretation is frequently because the vendor is not looking at it from the perspective of the prospect who sees the same thing at all the vendor websites that they are researching.

It should be obvious that if your competitors also have very similar case studies, and your case studies are not differentiated, how much of an advantage is one’s case studies?

Effectiveness Criteria #4: The Case Studies Can Be Easily Compared to One Another

Prospects are not looking for just one case study that allows them to envision how they will be successful if they go with you. They are looking for a pattern of success that matches the product they are considering purchasing from you, the industry they are in, and other dimensions of similarity.

When case study information can be compared in an analytical format, the prospects gain more insights from the information. It makes the overall exercise more interesting to them and makes the case study information more credible.

This is one of the great blind spots in the literature on case studies, and also how case studies are configured and presented by software vendors.

Topic Transition

This leads to the topic of some things that reduce the effectiveness of case studies.

Section 6: How Our Product Differs from Vendor Published Case Studies

Distinction #1: Analytical Overview Versus Story-Driven Case Studies

Most vendor case studies are story-driven. A good example is the following quotations from some of the case studies we found online.

Deciding if they can trust you with a huge investment of their time, money, and energy. Can you bring their vision to life? And can you help them find the best way to move their business forward?

Persuading potential clients means telling a story. – Design to Delight

And this quote.

The problem? Marketing departments don’t know how to write a case study that tells a great story. Instead, they crank out data-filled advertisements, without taking the time to craft a strong narrative. So the case study basically amounts to: “This company’s numbers were really terrible, our brand rushed in to save the day, and suddenly the client’s numbers were off the charts.” – Skyword

And this quote.

Stories are all around us. Stories move us, make us feel alive, inspire us to be more than we would be otherwise. Our appetite for stories is a reflection of the basic human need to understand patterns of life — not merely as an intellectual exercise but as a personal, emotional experience. Learning how to tell a story cannot guarantee the reaching of Truth, but it can help you connect with your audience, move your audience, and make your material more memorable. – Stanford

The case study format has advantages, in that it provides a lot of detail about each case study, but the format is actually problematic when it comes to cross-case study analysis as each case study is written as a stand-alone article.

Our case study survey product includes descriptions and percentages and graphics based upon data filled on by successful customers. This link at the whowonnobelprize.com demo website to view how we have produced graphics that tie together the different data points.

Distinction #2: Independently Written Coverage

Vendor case studies are normally alluded to by sales reps, or with individual case studies which tend to be addressed orally in meetings — with occasionally the prospect getting into contact with the reference.

A vendor cannot reasonably expect their case studies to be viewed as objective, nor can they expect to produce a research product on their own case studies due to the implication of bias and due to the fact that vendors are not research entities.

A few important notes about our research product in this regard.

• Neutral Writing: The writing of our survey is neutral.
• Declaration of Compensation and Authorship: The survey states to the readers that while the vendor paid for the survey/study, but that the involvement was limited to providing the survey to the participants and final sign-off. Brightwork Research and Analysis wrote the rest.

How Does the Vendor’s Marketing Department Fit Into The Process?

Marketing provides input to the process. For an effective case study survey, it is necessary that they defer to our experience in research and creating research products rather than try to control the process.

We also have a lot of implementation experience, so know how to design such a survey. Again the survey is its own thing, and separate from the case studies published on the vendor website.

The survey and overall product (which only a portion of which is the information provided from the survey participants) cannot be effective if it is driven by marketing. We drive the process.

Distinction #3: Added Analysis and Connection to Previous Brightwork Research

• The survey includes responses from customers, but we also add our observations to each of these survey responses.
• This creates an end product that provides multiple perspectives to the prospect.
• We also link out to related research articles where appropriate, and we have thousands of IT research articles and have produced significant original work in the area.

This reinforces or signals to the reader of the survey the following.

• That this is real research.
• That it is not controlled by the vendor.
• It is supported by other research which they can read and verify.

This allows us to weave in our previous work to support the survey results and provide more credibility.

Distinction #4: Filterable Survey Results

We offer the survey in a way that allows the information to be easily filtered. The filters can be anything, but we recommend having the industry as one filter and the software area as the other.

A good example or demonstration which you can interact with is in our website whowonnobelprize.com

This is a simplified demo that is more designed to show how filtering works and it contains little in the way of text. The research product we are discussing in this article naturally contains much more text. Once an item is selected, it opens to allow for much more text to be shown.

Distinction #5: A Survey Which is Constantly Updated

New survey participants can be quickly added so that the latest go-lives are updated in the survey.

When the survey is updated, the link to the survey site can be sent out to prospects and other interested parties, and they view the new addition within the context of the overall research product, which helps drive interest.

We produce research products that leverage web capabilities as you can see in the Nobel Prize demo and in another research product, The Brightwork Research Study on S/4HANA Implementation Study.

Topic Transition

This topic broaches how we charge for this service, so that we will address that now.

How Do We Charge for the Survey and How About Customer Support?

We charge a subscription for this service on a yearly renewal year basis. This is designed to be a long-term relationship that compensates us for adding to and supporting the survey website and allows us to provide high-quality customer support.

When the subscription ends and the survey website is thus taken down, the vendor can only take the case study information with them that they provide. The information added by us remains the property of Brightwork Research & Analysis. This is to prevent our survey from being reverse-engineered. Some vendors have a habit of thinking that they can access our capabilities in the short term, and then bring it in-house and have marketing do whatever we are doing.

The item is a Brightwork Research and Analysis product and is not to be published outside of our websites. There is no marketing department in any vendor that does what we do. A major benefit of this research product is the distance it has from the vendor — the fact it is published at a separate entity.

We also offer presentation services where we take prospects through the survey, explain its background, and share the observations on Zoom-type calls. This provides an independent view that is usually desirable in the process of persuading prospects.

The Scalability and Reusability of this Research Product and ROI

Here are some of the benefits of this research product to vendors.

1. Constant Reuse: The product can be constantly reused and constantly referred to by sales reps. In the following way, this might be how sales reps (or other people in the vendor interested in persuading a prospect).
2. ROI: The vendor receives a living and breathing and constantly reusable unique product that will quickly obtain an ROI. If just one prospect who was on the fence goes with the vendor due to the credibility offered by our research product, that more than pays for its cost to the vendor.

The way the survey is referred to in conversations with prospects might look like the following.

Company did XYZ is S&OP, and we had an independent research company produce survey of how our product is used. I will send you the link. – Your Sales Reps

The link is not to the vendor’s website but to a Brightwork Research and Analysis survey site dedicated to that vendor.

• We offer an uptime SLA, so the survey is always available.
• We provide rapid response on our customer service to address any concerns.

Conclusion

• This is a unique research product that has many advantages versus only using case studies to show social proof.
• Your current and future case studies don’t have to change. This is a different presentation and distinct from case studies.

To increase your social proof and increase its credibility contact us to find out how this research product can support your sales teams.