Our CX Data Doesn’t Match Industry Benchmarks, Now What?

I am often asked some version of this question:

We just saw the <Temkin Experience Ratings, Temkin Group’s NPS benchmarkForrester’s CXi, JD Powers, The ASCI> and it is completely different from what our internal data is telling us. How should we reconcile the two data points?

Given that I created several of those industry measurements, it’s fair for people to ask me that question. Here’s my answer…

Different Measurement Systems Deliver Different Results

There is no perfect or “ultimate” customer measurement system, since we can never know what every customer is thinking at every moment in time. So all measurement systems are, by definition, somewhat flawed. This is an important point, because we need to let go of the desire to identify which one has the “right” information.

Every customer measurement system can differ along a number of dimensions. In particular, these are often key differences between your internal system and industry benchmark studies:

  1. Who’s being surveyed? Temkin Experience Ratings, for instance, asks questions to randomly selected consumers who have interacted with companies. Internal measurements are often less random, since they may neglect people who haven’t provided contact information or people who are no longer customers.
  2. When are they being surveyed? Temkin Experience Ratings, for instance, asks questions during January. Internal measurements may ask questions throughout the year, after specific interactions, or during specific periods of time.
  3. What’s being asked? Temkin Experience Ratings, for instance, asks three questions, covering Success, Effort and Emotion), on a seven point scale. Internal measurements can be almost anything, including Net Promoter Score that is standardized on an 11-point scale, but we’ve seen companies use 7- and 10-point scales as well.
  4. How is the metric calculated? Temkin Experience Ratings, for instance, is an average of net scores for Success, Effort, and Emotion, which are calculated by taking the percentage of 6s and 7s, and then subtracting the percentage of 1s, 2s, and 3s. Internal measurements may be average scores, they may be segmented by different customer groups, they may be top box or top 2 box, or just about anything.

Given that internal measurement systems are typically different than industry measurements across those four items, it shouldn’t be a surprise that they often deliver different results.

My Take: Rely on Your Internal Data

Instead of trying to find which one of the metrics is correct, I recommend that you:

  • Understand the difference between the internal and external measurement systems (starting with the four items above).
  • Learn what you can from each of them. Maybe the Temkin Experience Ratings shows that you are lower with the general population, but your data shows that you are really doing well with high value customers.
  • Improve your internal measurement system. Most companies we’ve seen have significant opportunities for improving their internal customer measurement systems. Make sure the focus is on building a system that drives improvement, not one that just keeps score.
  • Whenever you can, rely on your internal data. Why? Because you can do more segmentation of the results, track changes to specific customers over time, and go deeper into questions about what’s driving the data. These are all things you may need to drive improvements.

The bottom line: Focus on improving, not on reconciling metrics

About Bruce Temkin, CCXP
I am a customer experience transformist, helping large organizations improve business results by changing how they deal with customers. As part of this focus, I examine strategy, culture, interaction design, customer service, branding and leadership practices. I am also a fanatical student of business, so this blog provides an outlet for sharing insights from my ongoing educational journey. Simply put, I am passionate about spotting emerging best practices and helping companies master them. And, as many people know, I love to speak about these topics in almost any forum. My “title” is Managing Partner of the Temkin Group, a customer experience research and consulting firm that helps organizations become more customer-centric. Our goal is simple: accelerate the path to delighting customers. I am also the co-founder and Emeritus Chair of the Customer Experience Professionals Association (CXPA.org), a non-profit organization dedicated to the success of CX professionals.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: