Why Net Promoter Score May Not Align With Business Results
January 27, 2014 7 Comments
I just received a great question: “Why do companies have a very healthy growth although their NPS is low and vice versa why can growth be decreasing although the NPS is very high?” I get asked versions of this question all the time, so I decided to capture my typical answers in this blog post (check out our Net Promoter Score (NPS) Resource Page).
My take: We’ve found a high correlation between NPS and customer loyalty across a large number of industries. But that does not mean that NPS will provide a clear understanding of a company’s business results. There are many reasons why a company’s business might perform differently than its NPS might suggest. Here are some of the common reasons that I’ve seen:
- NPS is not the ultimate question. In many situations, the amounts of promoters and detractors are roughly correlated with customer loyalty and business success, but that’s not always the case. It’s not a universally good metric as it’s not correlated to business success in all situations. For example, NPS may not be at all indicative of business success if customers are trapped because of a high switching cost, limited competition or monopolistic power of the company, unique product or service offerings, etc.
- Comparison NPS trumps absolute NPS. In general, health plans have low NPS scores yet many of them do well financially. Customers may not be likely to recommend their health plan, but if they don’t believe that there are any better options then it will not affect their loyalty.
- B2B roles are under-appreciated. There are different dynamics in B2B situations. If we ask treasury assistants in large companies to provide an NPS for commercial banks, we might believe that it should represent the health of a bank’s business. But what happens if CFOs, who control the banking decisions, give banks a completely different NPS?
- Non-customers are often overlooked. A retailer may have a high NPS, but still lose share if its products and services start appealing to a narrower audience. This type of situation is often missed, because companies tend to get considerably more feedback from existing customers than from prospective non-customers.
- Segmentation can alter the analysis. When an organization looks at its overall NPS, it might miss important trends in different customer groups. What happens if NPS is getting lower for high value customers and getting higher for low value customers? The overall NPS could stay the same or even improve while the company’s results decline.
- Survey design affects results. Many companies have a mismatch between the way they deploy NPS surveys and the insights they attempt to glean from the data. Companies ask the NPS questions at different times and frequencies, which can affect the overall results. If we ask NPS after a customer service event, then the results will likely be different then if we ask it periodically to a random sampling of customers.
The bottom line: NPS can be an effective metric in many situations, but only if used correctly
About Bruce Temkin, CCXP
Customer Experience Matters® 
Great post, Bruce. Keep pushing the envelope on focusing people on impact and results, not just measurement.
It’s not uncommon for companies to under- (or over-) perform their Net Promoter Scores. As you point out, the first thing to check is that you’re looking at an “apples to apples” comparison (what we at Bain call “relative NPS” or “competitive benchmark NPS”).
We wrote an article about how to ensure that your company derives full impact from customer loyalty in December 2013, and you can download the pdf here: http://www.bain.com/publications/articles/converting-loyalty-into-economic-advantage.aspx. There’s a good checklist of 5 questions that can help you think about how to ensure you derive maximum impact from the customer loyalty your company is earning.
Thanks for joining the conversation Rob.
Bruce, completely agree with all your points. One comment I’d like to make, in addition to all the points you mentioned, another influencing factor is often the “sample set”. NPS is usually run in sample of customers, especially for large businesses when the customer base is quite significant. If the sample set of customers identified isn’t representative not just from a segmentation point of view but also in terms of volume, it could really be ‘gaming’ of the NPS.
The number of customers surveyed needs to be of substantial quantity across the various groups and segments to derive a meaningful value.
Here are my responses to your blog post, as republished on CustomerThink:
Well taken points. And, NPS (along with ACSI/CSAT) has long been shown to have further granular interpretation and actionability challenges:
http://customerthink.com/emerging_chinks_and_dents_in_the_universal_application_and_institutionalization_armor_of_popula/
http://customerthink.com/customer_advocacy_behavior_personal_brand_connection/
I’ve got a blog in the CustomerThink queue which summarizes a 2011 article from the International Journal of Market Research, by Professors Robert East and Jennifer Romaniuk of the University of South Australia: “The NPS and the ACSI: A Critique and an Alternative Metric”.(http://www.ijmr.com/AboutIJMR/Samples/Sample5.pdf) Long story short, the authors injected incidence and volume of positive and negative word-of-mouth into a core customer behavior framework, and compared their results (in several retail and consumer products verticals) to both NPS and ACSI.
Here are their Overview and Conclusion statements from the article:
Overview: “As a consequence, metrics based only on current customers, such as the NPS, do not measure negative word-of-mouth effectively. We show that detractors give little of the total negative word-of-mouth on the brand and that, in two out of the three categories that we studied, detractors were responsible for more positive word-of-mouth than negative word-of-mouth. Similar patterns are found in an analysis based on the ACSI measures, which suggests that the NPS and ACSI are closer than their respective proponents are willing to claim.”
Conclusion: “We show that the NPS and the ACSI do not measure negative sentiments about brands effectively.”
So, in addition to the points you’ve articulated, the non-representation of positive and negative word-of-mouth in understanding consumer decision-making is a critical missing element in making both NPS and ACSI more actionable, particularly at a granular (communication, relationships and emotional connection, functional performance, brand positioning, etc.) level.
To build on Rob’s points, relative NPS should be the focus of how companies understand how they are doing. In my experience, most companies do not have solid apples-to-apples metrics to understand how they are doing. At Intuit, our goal was to be be 10 pts greater than the next best competitor in each of the markets we were in. The businesses that were doing this were always growing faster than the overall market. It didn’t matter if the NPS was 70 or -5.
Hi Bruce,
In my company we offer the NPS survey at the end of calls.
Would you agree that if an agent scored poorly on a survey but the comments left were overwhelmingly positive, that particualr survey should not be counted in?
I try not to make specific recommendations like this without more context like what’s the overall measurement framework, how does this feedback fit into the overall incentive system, and how is this NPS administered (which I can only dig into with our clients). However, I often see NPS inappropriately implemented at an interaction level like this. Depending on how it is administered, NPS from a customer can have very little to do with the specific interaction that it is being associated with. A customer may be really unhappy with the company, not the rep or the interaction, but still give a low NPS. And, in general, I believe there’s more to learn from the comments than from the scores.