The Problems with Net Promoter Score

Posted on:
December 4, 2018
Useful Reading For:

Net Promoter Score (NPS) is a widely used metric for measuring client happiness in SaaS businesses, but it has some serious shortcomings.

What is it

The Net Promoters Score(NPS) was created by Fred Reichheld and a Bain & Co. and has become very popular with Software as a Service companies. NPS is a very simple survey question:


“On a scale of 0 to 10, how likely are you to recommend us to a friend or colleague?”


Many companies love NPS for its sheer simplicity and lack of effort required by the responder.  It's also highly regarded because of the belief that if someone would go as far as to recommend your product to a significant other then they must be a loyal user themselves.


In terms of Customer Success, NPS looks to be a very useful score. After all, one of the main reasons for the role of Customer Success is to develop happy customers and happy customers result in growth.


How is it calculated

The calculation of NPS might not be the simple average you were thinking of.  Many that are new to NPS tend to think the score would be the mean average of all of the results. In reality, the scale of 0-10 is broken down into 3 groups:


From these groups, the calculation is derived as:


% of Promoters − % of Detractors = NPS


The range is from -100 to 100 and anything over 0 is considered ‘good’.




What are the problems?

  1. Very generalised - The advantage of NPS is that it’s very simple, however it’s just one question that covers all aspects of your service.  Would a client give your product a 9, but your customer experience a 6? You don’t know.

  2. No context - There is no ‘why’ to the answer.  Its the first question most people want to know, and in reality Customer Success Managers will go off and try to find out the reasons behind the responses.  So whilst NPS looks simple, to get real information involves more time and effort that you might initially think.

  3. Not actionable - Closely related to the above, what actions can you take from a single broad question?  What does the CEO do next, the VP of Product, the CSM? You have no idea without going back to the client and ask them for more details:


“Thanks for taking the time to respond to our survey recently. I noticed you scored us a 7 out of 10, can I ask for some context behind that”?

  1. Time specific - I imagine those companies that do use NPS, have a process as to when the surveys are sent.  Maybe at key milestones in the customer journey. E.g after onboarding is complete or 3 months before renewal.  However, it would be very easy to ‘game’ the system.  For example, if a client onboarded ahead of schedule with no issues, you might send the survey immediately to gain a good response, but for another client who had a poor onboarding experience, you could hold off sending that survey for one or two weeks in the hope some bad feeling has passed. And in fact, if you do send it after the onboarding, aren't you really just scoring the onboarding process and not the overall company?

  2. Human nature - I don’t know about you, but I don’t think I ever score anything a maximum 100%.  If you do, you are saying this product or service is perfect and nothing could be improved. Who would say that, especially if you are a client that likes to keep their vendors on their toes? I’ve personally followed up with a respondent who gave an 8 and asked them for their reason, and they said they scored us highly because they love the platform which gives them insights no other solutions do and they also love the service levels they get.  So this is a great response, however, they were deemed to be ‘neutral’ in the realm of NPS.

  3. Can be gamed - Along with what was mentioned above about being careful of the timing the survey is sent, you can also attempt to favour your results in a number of other ways. For example, do you send the survey to all of your daily users, even those you know are not a fan? Do you send the survey to the exec sponsor and the budget holder? By being selective with how and when the survey is sent, you unfortunately have some control over the expected result.


What else needs to be done

If you are going to use NPS, it’s vital that your CSMs are following up with respondents to find out the ‘why’, especially for your detractors as well as promoters. NPS also needs to feed into a wider Customer Health Score. As we’ve seen, NPS leaves too many unanswered questions and is too simplistic to derive whether a client is truly loyal or not. It is possible that you can set up an automated email to go out to Promoters and heavy detractors. We're also seeing more implementations of NPS have a second follow up question, which might be something like "Would you care to tell us why?". This is great, and just what you need to start to gather a certain level of context behind the initial score.

"Adobe phrase the question slightly different "What would make you more likely to recommend this product to others?"

It’s my belief that a system like NPS should be a constant measure. The best users of NPS will send out a survey at numerous points in the year, just after a platform update, or at a certain time periods, but I think It should be open to being updated at any given point, whenever the client feels like it.  It’s far more useful to you as a company to know how the client is feeling after each interaction with your product and staff. It could easily be baked into the system perhaps as a menu item or in Profile settings. A slightly more controversial idea would be to have this score as mutual, so both the CSM and client can update it as they see fit.  I’d love to see the comparisons of how happy the CSM thinks the client is versus the client's view. That gap in itself could lead to some useful training and education sessions.


The only real way to measure customer satisfaction is by blending a number of metrics.  Usage data, public sentiment, QBRs, and yes NPS/CSAT etc, and this all rolls up to the Customer Health Score, which vitally MUST contain a CSM ‘Gut Score’.  



Written By

Simon Cooper

Simon has over 10 years helping clients achieve their goals through the use of software.  Having previously lead Customer Success teams in London, Europe and New York City, Simon now owns Kupr Consulting working with B2B SaaS companies to improve their Customer Success teams and processes.

https://www.linkedin.com/in/sfcooper/https://twitter.com/kuprhttps://www.kuprconsulting.comAll Articles

Related Articles

When To Start The Renewal Process

Renewals are massively important to SaaS businesses. Make sure you start the conversation with your client sooner, rather than later.

Read

Customer Success Management with Sarah Chasen

We speak to Customer Success Director, Sarah Chasen, as she comes back to work after pregnancy.

Read

Should Customer Success Be Involved Pre-Sales

Customer Success can bring great value to a final stage sales prospect.

Read

Become a Customer Success Expert

Receive regular emails and help yourself become a Customer Success expert. You can unsubscribe in one click, and your email will never be used for anything else.