Net Promoter And Satisfaction Battle For King Of The Ring

Let’s start with a confession: I’m a big professional wrestling fan; so I really enjoy a good battle. One thing that I’ve learned from the WWE, is that it’s the storyline that makes a battle come to life. And the Net Promoter vs. Satisfaction debate has all of the story trappings of a great tag team match!

One one side of the ring in the blue trunks is the tag team of Fred Reicheld, “father” of the Net Promoter System (NPS) concept and Satmetrix Systems, implementor of NPS-based survey systems. On the other side of the ring in the red trunks, we find Claes Fornell, “father” of the American Customer Satisfaction Index (ACSI) and ForeSee Results, implementor of ACSI-based survey systems.

Both of these teams are fighting for their approach to be recognized as “THE” measure for tracking customer relationships. To put this into perspective, this type of measure represents only one of the five levels of a voice of the customer program (see my earlier post on voice of the customer programs).

Let’s start by handing out some awards to the teams:

  • Best marketed: Net Promoter (Reichheld is very good at touting his concept — and in writing compelling books about it)
  • Most mature: Satisfaction (The ACSI has been tracking data since about 1994 and satisfaction has been around as long as I can remember)
  • Most quantitative: Satisfaction
  • Sexiest: Net Promoter (it’s caused a lot of hooplah)

Net Promoter has gained a lot of momentum over the last few years as many large companies have adopted it. The methodology is pretty straightforward: ask people if they’d recommend your firm. Based on their response, they get categorized as a Promoter, Detractor, or neither. You take the percentage of Promoters and subtract the percentage of Detractors and that leaves you with a Net Promoter percentage.

This debate was enhanced by a recent study cited in the Journal Of Marketing which found that…

Using industries Reichheld cites as exemplars of Net Promoter, the research fails to replicate his assertions regarding the “clear superiority” of Net Promoter compared with other measures in those industries.

Well, if you’re wondering what I really think about this Battle Royale, then here it is. Just like wrestling — the storyline is much more exciting than the reality of the battle. Here’s my take on the contest:

  • Net Promoter is not the “ultimate” measure for a customer relationship
  • Then again, neither is satisfaction.
  • But companies are better off when they have more satisfied than dissatisfied customers and more Promoters than Detractors.

My recommendations:

  • Don’t expect any single measure to be eutopia. Both measures are good, but neither one has enough information to fully guage customer relationships and to provide enough diagnostic information to make all of the necessary improvements.
  • Focus on one measure to build alignment. Picking a single measure to focus on (whether or not it’s perfect) can be very valuable in aligning the organization. If you can get your entire company focused on either raising satisfaction or increasing the number of Promoters, then you will likely see some significant improvements in the reallt important metrics: retention, sales, etc. So, if in doubt, pick one and move on.
  • Evolve your metrics over time. The previous two bullets may seem to contradict each other, but they don’t when you look at it over time. The value from locking into a single measure like Net Promoter is as much from aligning the organization as it is around the perfection of the metric. But after the organization gets aligned, firms will need to build out the portfolio of metrics — and find out for themselves which measures are both predictive and diagnostic.
  • Look at Customer Advocacy. The ring was too crowded to add another contestant to the match earlier in this post, but for some industries we’ve found another measure that is a powerful indicator of loyal customer behavior. So, in the purple trunks is Customer Advocacy, the perception that the firm does what’s best for customers, not just what’s best for its own bottom line. We strongly recommend that financial services and healthcare firms take a very close look at this measure.

The bottom line: Don’t get too caught up in determining the winner of this battle. Just make sure that you do something and are prepared to learn and evolve over time.

If you’re a client of Forrester, then I also recommend that you read these two research documents:

About Bruce Temkin
I am a customer experience transformist, helping large organizations improve business results by changing how they deal with customers. As part of this focus, I examine strategy, marketing, interaction design, customer service, and leadership practices. I am also a fanatical student of business, so this blog provides an outlet for sharing insights from my ongoing educational journey. Simply put, I am passionate about spotting emerging best practices and helping companies master them. And, as many people know, I love to speak about these topics in almost any forum. My “title” is Managing Partner of the Temkin Group, a customer experience research and consulting firm that helps organizations become more customer-centric. Our goal is simple: accelerate the path to delighting customers. I am also the co-founder and chair of the Customer Experience Professionals Association (CXPA.org), a non-profit organization dedicated to the success of CX professionals.

10 Responses to Net Promoter And Satisfaction Battle For King Of The Ring

  1. Deborah says:

    Bruce, Nice posting. This is one of the few that gets to the heart of the issue, aligning your organization to build loyalty will drive growth. That’s the point missed in much of the debate taking place around the Journal of Marketing paper.

    Here are my thoughts on the debate:

    1. It appears that the Journal of Marketing research referred to shows that Net Promoter is, at a minimum, equivalent to ACSI in correlating to growth. (Figure 1). That says that a simple metric driven by a single question is at least as accurate at predicting growth than a more complex algorithm driven by multiple questions.
    2. After discussing the claims with Satmetrix, they stand behind the research. Claiming a research bias is a pretty serious allegation. Both Fred and Satmetrix have spent over 10 years studying the correlation between loyalty and growth and helping companies implement programs that improve loyalty and drive growth. The result of this effort is apparent in reviewing the case studies presented at the Net Promoter conferences: http://netpromoter.typepad.com/npc_london_2007/
    3. This conversation is missing the point that is attracting business leaders. The value of Net Promoter is its simplicity. Unlike complex satisfaction indexes, Net Promoter is easy to understand and take action on. Simplified surveys drive higher response rates, a better reflection of the customers that matter, rather than random sampling. Using real-time reporting, leaders can get information in the hands of employees that can address detractors, move the passives and nurture the promoters.

    At the end of the day, it’s what customers do to improve loyalty that drives growth. Net Promoter offers an approach that is understandable by everyone in the business, not just the statisticians. This gives an organization a rallying cry for building customer centric organizations.

    Full Disclosure: During this conversation I was contemplating joining the Satmetrix team. Having experience deploying Net Promoter programs, and researching the market further, I decided to join Satmetrix as CMO: http://www.satmetrix.com/news/pressrelease_2007-07-17.htm

    Let’s continue the conversation.

  2. Bruce Temkin says:

    Deborah:

    Thanks for the post. And, good luck with the new job!

    I’ll stay clear of the ring (at least for this post) when it comes to the battle over the strength of research underlying each team’s claim for supremacy.

    In my discussions with clients, I tell them that they need to spend SIGNIFICANTLY more time figuring out how they will use the metric to drive change in their organization than they spend thinking about which metric to use. So your point about rallying the organization is a good one — firms should not underestimate how important that is.

  3. rshevlin says:

    @Deborah: There are three things I’d like to respond to regarding your comment:

    1) I’m not clear why a question like “on a scale of 1 to 5 (or 6 or 8), how satisfied are you with XYZ company?” is a complex algorithm. Seems pretty simple to me. And if NPS is equivalent to satisfaction as a predictor, then why should companies invest hundreds of thousands of dollars (if not millions) to implement a whole new measurement infrastructure on top of or in replacement of the satisfaction measurement efforts they already have in place?

    2) Who says that understanding and measuring something as complex as customer loyalty should be reduced to one simple question? If that were the case, then Bruce’s definition of customer advocacy (the perception that the firm does what’s best for customers, not just what’s best for its own bottom line) could be a far superior metric than NPS since it too is highly correlated with future purchase intention, and is more actionable. Actionable in the sense that if your company’s score was low on this metric, there’s research that helps firms understand what this definition means to consumers to help drive operational changes. Is a consumer answers no to the question “will you refer us”, what’s a firm supposed to do to fix things? (Granted, the satisfaction measure is no more helpful here).

    3) In addition, NPS, while it may or may not suffer from research bias, does suffer from one other kind of bias: Younger consumers (Gen X/Y) are more likely to refer products/companies they like to family and friends than older (boomer, senior) consumers. So a change in a firm’s NPS might reflect more of a shift in it the demographics of its customer base than an improvement in loyalty building efforts.

  4. Tim Keiningham says:

    Ms. Eastman’s statement that research bias is a serious allegation is absolutely correct. So we are waiting for a good answer. The argument that Net Promoter is “at least” as good as the ASCI is irrelevant. Reichheld claims his research reveals no relationship between the ACSI and growth.

    In the Harvard Business Review article that introduced Net Promoter, in the book, The Ultimate Question, and in presentations regarding Net Promoter, the American Customer Satisfaction Index has been specifically mentioned as not linking to firm growth by Reichheld. The book, The Ultimate Question, argues that the ACSI does not yield much insight into loyalty or growth, noting that “investors rarely waste money on standard satisfaction surveys” as a result (The Ultimate Question, p. 86).

    Similarly, an article in the Harvard Business Review states (p. 49): “Our research indicates that satisfaction lacks a consistently demonstrable connection to actual customer behavior and growth. This finding is borne out by the short shrift that investors give to such reports as the American Customer Satisfaction Index. The ACSI, published quarterly in the Wall Street Journal, reflects customer satisfaction ratings of some 200 U.S. companies. In general, it is difficult to discern a strong correlation between high customer satisfaction scores and outstanding sales growth.”

    Furthermore, in a web-based presentation, Mr. Reichheld states that a “Bain team looked at the correlation between growth and customer satisfaction, and found there is none.” A scatter diagram was shown with the X-axis labeled “American consumer satisfaction index annual growth” and the Y-axis labeled “Sales annual growth.” The R-square reported was 0.00, indicating no correlation whatsoever.

    Given that our findings show that Net Promoter was not superior to the ACSI when using Reichheld’s best-case scenarios, it is virtually impossible to imagine a scenario other than research bias as the cause. This is a VERY SERIOUS problem. We expect research published in our most prestigious journals to be free of bias in management science, just as we do in all other fields of study. We would not consider this kind of problem acceptable had the research been conducted in medical, psychological, or physics research; the same standards apply in management science.

    Managers have adopted Net Promoter based upon the belief that solid science underpinned the claims attributed to the metric. In fact, there would have been no Harvard Business Review paper introducing Net Promoter without the research. This also has serious implications regarding the credibility of Reichheld’s book, The Ultimate Question. Additionally, biased “research” that is published in a prestigious management journal contaminates not only management practice, but also management science, as it will be used by scientists as a basis for future research.

    It is vital that we not be apologists or revisionists when it comes to issues of research bias. Our credibility must never be in question regarding the research we publish in prestigious journals; the truth matters. Therefore, discussions about Net Promoter by researchers (practitioner and academic) must first adequately address this issue. If not, then why do any research at all, as we can simply present the answers we want to believe as supporting evidence and be done with it? Given the evidence we uncovered, however, we seriously doubt that there will be an acceptable answer to the issue of bias in Reichheld’s reported research. [Ironically, in The Ultimate Question, Reichheld emphasizes the importance of eliminating bias from research (pp. 106-111).]

    While we can all agree for the need to have measures that are easily understood and used by managers, that is completely irrelevant to the issue at hand. Regardless of whether or not one chooses to believe in Net Promoter, we all must insist that the evidence used to support the metric be unbiased. This issue is so overridingly important to very core of what we as researchers (academic and practitioner) do and stand for that it must be addressed. Our credibility in science and in practice is based upon honest and fair evaluations of data…if this is contaminated and unchallenged, then there is no reason to believe anything we say.

    Sincerely,

    Tim Keiningham

  5. Katie Paine says:

    Another side of this story is the work that Mark Klein at Loyalty Builders (www.loyaltybuilders.com) has done. He doesn’t think that ANY survey is accurate and instead only looks at customer purchase patterns and uses them to both predict behavior and to diagnosis weaknesses and opportunities. Might be an interesting hook up to see if his data correlates better with NPS or ACSI

  6. Pingback: What Is The Perfect Customer Experience? « Customer Experience Matters

  7. Tyler says:

    Given the roaring debate about NPS and Customer Satisfaction, I think there should be discussion on actual implementation use or abuse. For example, should NPS replalce customer satisfaction at the micro level, especially in a non-consumer product category. Would anyone care to comment on this scenario: Using NPS calls to follow-up on-site technical service events in a non-consumer product? Our company is very NPS drriven, however I fear using this in the microcosm of one service event followed by an NPS survey call is potentially hazaardous. My take on NPS is that it is effective as a macro, over-all firm strength vis-a-vis other similar companies in the same industry.

  8. Bruce Temkin says:

    Tyler: Thanks for the comment. I’ve defined five levels of a voice of the customer program which are important for isolating best practices. You’ve highlighted the difference between two of the levels: Relationship Tracking and Interaction Monitoring. I agree that NPS tends to be much more appropriate for Relationship Tracking than it is for Interaction Monitoring. If people are already an advocate of a company, for instance, then they still may be an advocate even if they have a bad experience. That would not provide appropriate visibility for Interaction Monitoring where you are looking for opportunities to improve specific interactions.

  9. Pingback: Three lessons from the Net Promoter journey « Fredzimny’s Blog

  10. Pingback: My Closing Thoughts On Net Promoter « Customer Experience Matters

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: