Net promoter score (NPS®) has become a ubiquitous metric in the business world. NPS was developed in 2003 by Fred Reichheld, a partner at US management consultancy, Bain & Company, as a predictor of how firms would grow. Over the past 20 years, it has seen astonishingly widespread adoption: as many as two-thirds of Fortune 1000 businesses reportedly use NPS or one of its variants. Moreover, the metric has become hugely influential on business-strategy development: NPS is “more than a metric,” enthused one IBM employee in a 2021 article: “One could use the word religion.”
However, there are reasons for dissenting from this new “religion.” Challenges have been raised to the core methodology for calculating NPS, and to how organizations use – or misuse – it in mapping consumer responses. Understanding the inherent limitations of the metric, and the misguided thinking that leads businesses to lean too heavily on a single metric, could be the key to using it effectively. In short, there are three aspects to NPS: the good, the bad, and the ugly. Given the seriousness of these issues, we propose a series of actions that firms using NPS should take to counterbalance its weight.
The first – and perhaps most powerful – reason NPS is so attractive to leaders is its simplicity. It is remarkably easy to understand and use. For those who are unfamiliar with it, NPS generates a single number through a simple technique of asking customers how likely they are to recommend the company to a friend or colleague. Answers are numeric, on a scale from 0 to 10, and responses are divided into three camps: promoters (scoring 9 or 10), neutrals (7 or 8), and detractors (0‒6). Subtract the number of detractors from promoters and, voilà, you have your NPS. (Reicheld has shared the methodology, enabling anyone to apply it.)
In businesses where word of mouth is a key promotional avenue, NPS is particularly important. It is scalable and easily adapted for use at every level, from promoting a business’s overall standing down to the performance of an individual customer service agent. Equally, it can be used across country or regional levels. There is also value in the fact that it has been so successful: as a widely accepted benchmark, it is a common currency that can be employed to make useful comparisons.
These are powerful advantages. What, then, are the drawbacks?
There are four fundamental criticisms to be made of NPS – or, at least, of how it is often used:
The most fundamental challenge comes from academic statisticians who analyzed NPS and failed to establish statistical validity. One early study found that “NPS is not significantly associated with either a firm’s sales growth or gross margin performance in our regressions.” Another, that “NPS has performed poorly as a predictor of brand/company performance.” Another concluded, simply, that: “NPS is neither superior nor inferior to other metrics.” Recent academic research has confirmed these results (“the methodological concerns raised by academics are valid”) to the point that the strong divergence between academics and practitioners on this topic has become itself an area of research. So, while NPS is commonly regarded as a useful metric, it cannot be perceived as a growth predictor or as a holy grail for business strategy.
“The most fundamental challenge comes from academic statisticians who analyzed NPS and failed to establish statistical validity.”
The method of calculating NPS – subtracting the number of detractors from promoters – wastes a lot of information. Neutrals scoring 7 or 8 are disregarded and variation within groups is ignored. For example, scores of 0 or 1 are counted the same as 5 or 6, yet with one group covering such a wide range of scores, they represent very different sentiments. To produce one simple metric, a huge amount of potentially valuable information is ignored. In fact, a simple average would be more diagnostic about the true situation.
A further set of issues which are rarely well understood by NPS users relates to drifts in customer demographics that can occur over time. For one thing, there is a tendency to survey only “stayers.” Truly unhappy customers leave, and consequently cannot be subsequently surveyed. This could give an inflationary bias to NPS as negative opinions are naturally filtered out. Conversely, customer enthusiasm is prone to fading over time. When NPS is used in the early life stages of a new company or product, the customer base is, by definition, made up of early adopters – they have identified the product as something for which they have an immediate need and do not want to wait for the modificatory process of market opinion. Over time, that enthusiasm tends to wane. More broadly, customer expectations inevitably evolve as time passes; what they once perceived to be impressively innovative begins to seem mundane or commonplace, and scores may dip accordingly.
NPS is highly sensitive to cultural differences. Do North Americans, Brits, and the French express satisfaction or delight in the same way? The US grading system is widely perceived as being more generous than, say, the French one. Against that backdrop, the NPS scoring system assumes that these cultural differences are not meaningful. That is a particular issue for multinational businesses, yet many still use NPS to make comparisons across markets.
Secondly, there is a problem in relation to NPS as a tool for influencing remuneration levels. This gives employees a strong incentive to introduce bias into the system for personal benefit. This could be direct, for instance when a salesperson or service provider asks a customer for feedback and pleads, “Please score 9 or 10, or I won’t get my bonus.” It may also be indirect, for example when communications soliciting feedback nudge participants more subtly toward positive scores.
Over the years, I have collected creative and terrifying examples that I show in my classes, to the stupefaction of the participants. In larger samples, there is even the potential for mechanized generation of false returns – gaming the system on an industrial scale and generating data that is completely unconnected to customer opinion and, therefore, of no use at all to analysts. In any of these cases, the bias seriously undermines the value of NPS as an accurate gauge of customer sentiment.
Those issues may not have been evident in the design process of NPS, but they have become significant as its use cases have developed.
Companies will want to retain NPS as a tool in the struggle to understand their markets; however, they will need to work on the “bad” and “ugly” elements in the following areas to ensure that they are using the tool in the most effective way possible.
Start with a simple average and follow this average overtime. Using a different range of scores for the categories may yield a more accurate picture – for example, 8-10 for promoters, not just 9 and 10 , or put the 6 in the passive category.
Companies should avoid using NPS as their one key indicator of where to focus resources. NPS may be useful as one metric among many, giving a more balanced picture that considers a range of factors. Add additional questions relating to areas such as direct satisfaction, intent to repurchase, and include more open questions (“Tell us why…”), to allow customers to express their opinions more fully and in their own words. Consider wave-based research on usage and attitudes, and in-depth satisfaction surveys.
Given the limitations of NPS, companies should also look at objective performance measures (such as waiting time, critical incidents, etc.). For performance management, seek out other metrics, including behavior-based assessments by supervisors.
To put NPS in context, organizations should seek to analyze data to test for drifts in composition of customer demographics over time and to survey changes in customer expectations. Comparing your NPS to a competitor’s can be helpful, as can benchmarking drifts against those of non-competing firms.
Given the various cultural factors we touch on above, a trend-based approach is vital when comparing across regions or countries. Again, differences may be benchmarked with non-competing firms to provide context.
To tackle the issue of gaming, organizations should closely monitor the integrity with which NPS surveys are administered. Unusual favorable jumps in results should be checked thoroughly. Make use of “mystery shoppers” to vet the process from a user’s perspective.
To a certain extent, NPS has been a victim of its own success; what was developed as a useful but limited tool has been applied in ways for which it was never intended. Simply put, NPS should not be the number that businesses consider using to guide growth.
Rather than developing a quasi-religious obsession with NPS, organizations would be better advised to try to develop a more balanced dashboard, of which the NPS could remain one element. Not a magic wand.
Collaboration between humans and new forms of artificial intelligence (AI) are creating opportunities for considerable improvements in written communication. But to do so successfully, business executives must learn to strike a balance...
Teaching computer literacy to children in rural India means building cross-sector partnerships that create value for everyone involved. KidsWhoKode Founder and CEO Shweta Mukesh explains how she brings together diverse organizations to...
8 November 2023 • by Howard H. Yu in Innovation
Are you doing the right things to make your company future-ready? My research suggests you should be asking these four questions to find out. We studied companies that were more successful getting...
Li Shufu, CEO and founder of Chinese carmaker Geely, overcame a host of setbacks to become the billionaire owner of a multi-faceted business ...
Explore first person business intelligence from top minds curated for a global executive audience