Customer Experience – Page 9

Why Sample Sizes are Nonsense (in the B2B World)

Posted on January 26, 2016 at 1:37 pm.

Most of Deep-Insight’s work is based on helping large international B2B organisations run effective Customer Experience (CX) programmes.

The key to running a good CX programme is understanding how to change the culture of an organisation to make it truly customer-centric, and that has to be based on regular high-quality conversations – both formal and informal – with your B2B clients.

Without regular client feedback, sales directors and account teams will not be in a position to address small issues before they escalate to a point where they damage or destroy the client relationship.

When we plan Customer Relationship Quality (CRQ™) assessments for our clients, one of the questions I regularly get asked is “How many of our clients should we sample?” The stock answer that I’ve been using for the past decade is “Think Census, Not Sample”. In other words, get feedback from your entire client base – every single one – and it’s the answer I still use.

It’s not meant to be a glib response but there are a few subtleties underpinning the answer that are worth exploring.

TRADITIONAL APPROACH TO SAMPLING

Many market research and customer insight people – even in B2B organisations – tend to approach the subject of customer feedback from a consumer perspective, where there are tried and trusted approaches for surveying large customer bases, or “populations” to use the technical term. If you’re not that familiar with these approaches or terminology like random sampling, margins of error and confidence levels, have a look at the Box below.

MARGINS OF ERROR, CONFIDENCE LEVELS AND SAMPLE SIZES – TRADITIONAL APPROACH

If you’re not a market researcher or statistician, don’t worry – there are plenty of good primers on the Internet explaining the basics of sampling techniques and associated terms – here’s one from YouGov.

You’ll also find several handy little calculators on the Internet (here’s a link to one) which let you know how many respondents are required for a particular population (customer base) in order to give a confidence level and margin of error. From this, it’s easy to calculate the number of individuals you need to invite to participate in a survey in order to get a robust answer.

Most opinion polls are conducted with a random sample of at least 1,000 people and here’s the reason why: pollsters like to be confident that their results are within a margin of error of 3% or less. Supposing the voting population in a country is 10 million people. Plug that number into our online calculator and we see that a 3% margin of error and a confidence level of 95% requires a sample of 1,067.

All that is fine if you’re working in a consumer environment or if you have tens (or hundreds) of thousands of SME customers. However, the traditional sampling techniques have less value when you are a B2B organisation and the vast proportion of revenues is generated by a handful of large clients. There may be a “long tail” of smaller customers but in most cases the Pareto Principle applies, whereby 80% of revenues are generated by 20% of clients. In some cases, the ratio can be 90/10 rather than 80/20. In such cases, the old traditional sampling approach needs to be chucked out of the nearest window and a different set of principles applied.

THE DEEP-INSIGHT APPROACH – FOLLOW THE MONEY

Our approach is to be pragmatic and follow the money – concentrate on those clients that generate the majority of the revenues, and do a ‘deep dive’ into those relationships.

It’s probably easier to explain using an example.

Case Study – Large UK Services Company
Revenues: Over £1 billion
Key Clients: 100
One of Deep-Insight’s UK clients has over 10,000 employees and generates annual revenues in excess of £1 billion. However, its customer base is actually quite small and the contracts it has with these key clients are extremely large. The company has several hundred clients in total but the vast majority of its revenues come from the ‘Top 100’ and even among the ‘Top 100’ the revenues are skewed heavily towards the 10 largest clients.

So how do you run a CX programme when your client base looks like this? In that particular case, the company has chosen to focus exclusively on its ‘Top 100′ clients. Purists might argue that this is not representative of the full customer base. This may well be true but it definitely is representative of the full revenue base, and that’s the commercial perspective of “following the money.”

From a pragmatic perspective, it makes little sense to take a sample of the Top 100 clients. You should attempt to get feedback from every single one and ideally you want to get a wide representation of views from across each of those 100 clients.

Even from a statistical perspective, it makes little sense to sample – if you need convincing, have a look at the second Box below.

APPLYING THE DEEP-INSIGHT APPROACH

Suppose there are 10 key individuals (at most) in each ‘Top 100’ client whose feedback is really “important” (in other words, the decision-makers who will renew the current contract when it’s up for renewal) that’s still only a population of 1,000 individuals across your Top 100 clients – run the numbers and you’ll see that you need to include all, in order to get a statistically significant sample.

Let’s plug those figures from our Case Study into the online calculator and see what happens.

For a population of 1,000 decision-makers, we need 278 responses to get a robust score (robust being a margin of error of 5% and a confidence level of 95%). Deep-Insight will typically achieve completion rates of 35-40% from its online B2B assessments so that means we need to invite 700-800 of those 1,000 key individuals to participate.

If you think a margin of error of 5% is too high, then plug in 3% into the online calculator. Now the number of responses jumps to 517 out of 1,000. This means you DEFINITELY need to invite all 1,000 to participate to get anywhere near your target margin of error.

Successful CX programmes in B2B companies are not built around statistics. They are built around empowering staff and providing account managers with all the customer feedback they need to manage client relationships more effectively. That means getting feedback from ALL individuals in those key clients and working really hard with the account teams to get participation and completion rates as high as possible.

So when you’re planning your next Customer Relationship Quality (CRQ™) assessment, remember to get the account managers involved and “Think Census, Not Sample”.

A Christmas Message to Our Customers

Posted on September 21, 2015 at 1:57 pm.

Written by John O'Connor

We’re coming to the end of another year and all in all, it’s been a good one for us here at Deep-Insight.

We are an Irish company – and proud of it – but our client base is international. Over the past 12 months, we have carried out customer and employee assessments in the UK, Netherlands, Poland and Australia as well as in our home market. That said, one thing that I have noticed in 2015 is a marked increase in activity from local companies. The Irish recovery is definitely under way.

NEW FACES

We’ve had a particularly busy year at Deep-Insight and as a consequence, there are a few new faces in the Cork office these days.

We’re delighted to welcome Jamie Jaggernauth, who is our latest addition to the Deep-Insight team.

Jamie hails from Trinidad and has worked in a variety of research roles in the Caribbean, UK and Ireland before joining Deep-Insight.

NEW CLIENTS

We also added a few new names to our client list during 2015, ranging from large well-established firms like the health insurer VitalityHealth in the UK to newer digital organisations like DoneDeal, which is Ireland’s biggest classifieds site (and which is part of the Norwegian-headquartered Schibsted Media Group).

In fact, DoneDeal celebrated its 10^th birthday this year so a big happy birthday to John, Cathal, Kristian, Simon and the rest of the DoneDeal crew!

DEEP-INSIGHT’S OWN CUSTOMER ASSESSMENT

Earlier this month, we asked you what you thought of your relationship with us.

Now, I must admit I awaited these result with some trepidation. We think we deliver an excellent service to our clients but it’s always slightly scary waiting to hear what people ACTUALLY say about us and the benefits of working with Deep-Insight. It’s scary because we do take your feedback personally and it’s always a little nerve-wracking waiting for the results ton come through.

Last year, we had a Customer Relationship Quality (CRQ) score of 5.2 and a Net Promoter Score (NPS) of 17%. I was a little disappointed with those scores last year as they were down from the scores we received on our previous assessment and I felt that we could – and should – have done better.

A NPS of 17% is above average (more on average and good Net Promoter Scores here) but frankly it’s not that much above average. We had significantly higher scores in the past and we don’t see ourselves as a “slightly better than average” company. We used to be regarded by our clients as ‘Unique’ but in 2014 we dropped out of that zone. It’s a bit like a restaurant losing its Michelin Star – I was extremely keen to see if we could get back into the top bracket again this year.

So how did we do? How did you rate us?

2015 CLIENT FEEDBACK

This year, you gave us a Customer Relationship Quality (CRQ) score of 5.7 and a Net Promoter Score (NPS) of +37%.

I was a little stunned when the results came through as our NPS and CRQ scores had shot up dramatically. As many of you will be used to hearing me say at this stage, it’s quite a challenge to get your CRQ score to jump by more than 0.2 or your NPS to increase more than 10%. We had worked hard on a number of fronts over the past 12 months but the size of the improvement in scores still came as a surprise. So thank you for that vote of confidence in Deep-Insight – it really does mean a lot to us.

The other thing I’m particularly pleased with is the fact that we are back into the ‘Unique’ zone – that’s the light green box in the top right hand corner of the graphic. To be seen as unique, a company has to be able to provide a solution that truly solves its customers’ problems, as well as providing an excellent experience for that client. That’s something that only 10% of B2B companies achieve so it’s nice to be able to claim that accolade again.

There’s still plenty for us to work on. We’re currently analysing each and every verbatim to figure out exactly how to improve our service even further. We will be sharing these results with you as early as we can in the New Year.

LOOKING FORWARD TO 2016

So there it is. 2015 is nearly over but we have some exciting things planned for next year.

Over the past few months, Rose Murphy has been talking to most of you about what you like and dislike about our current product offering. The feedback you have given to Rose, as well as the various suggestions you have made in this recent client assessment, will help us improve what we do and how we do it.

But for the moment, allow me to say a big thank you to each and every one of you for supporting us throughout 2015.

On a personal note, I’d also like to say a big thanks to the following (in no particular order other than alphabetical): Brian, Frank, Grainne, Jamie, Mark, Mary, Peter, Pim, Rose, Yvonne as well as to the rest of the wider Deep-Insight team who have helped to deliver a fantastic service to you – our clients – over the past 12 months.

Have a very peaceful Christmas and I look forward to seeing you all in the New Year,

John

5 Generic Actions to Drive up your Relationship NPS Scores

Posted on April 15, 2015 at 2:38 pm.

Written by John O'Connor

We run B2B customer assessments for large corporate clients in the Netherlands and elsewhere. Very often I get asked questions like this after we deliver the customer feedback to senior management:

“OK, you’ve told us what our customers think of us, but what do we do about it now?”

“Tell us what we do in the next few weeks so that we don’t lose momentum.”

So here’s what I tell them:

1. BRIEF YOUR PEOPLE.

Typically, you need to brief at two levels and both are equally important:

– Executive Team. The senior executives in your organisation need to ‘own’ the overall results. If the customer feedback is negative or requires fundamental change, only the executive team can decide on the appropriate actions.

– Account Managers. These are the people who need to ‘own’ the results at account level. They are also the people you need to set up and run the account feedback sessions (see below).

Make sure to get the senior executives to read all the verbatim comments and see if their summary of the top issues agrees with the Deep-Insight analysis (they will understand the context better than we can). The most effective way of increasing Customer Relationship Quality (CRQ) and NPS scores is to have the programme driven by the executive team. Without the drive and passion at this level, the programme will fail.

2. DISCUSS THE RESULTS WITH YOUR CLIENTS.

Typically this happens in two stages:

– Stage 1. Within 2 weeks of the assessment results being delivered, you should get a general communication to all customers that you invited to give feedback. This should include a general ‘Thank You’ message for participating in the assessment (OK, not everybody completed it – our completion rates are typically 35-40% – but let’s not be mean!) as well as a message that their account manager will be in touch to arrange a feedback session to discuss the results (the overall results plus the specific results for their account)

– Stage 2. Within 8 weeks (or whatever target you set – but you must set a target), the account managers should have completed face-to-face meetings with all key customers. Ideally, they will have used that discussion to create a ‘Joint Action Plan’ setting out what both parties need to do, in order to address any issues unearthed in the assessment. Remember that the actions are on both sides – the account manager may need the client to change certain things as well.

3. FEED THE RESULTS INTO THE ANNUAL BONUS SCHEME.

If you haven’t done so already, think about incorporating the CRQ or NPS scores into account managers’ annual targets and bonus plans. If you have done so already, pay out the bonuses! This is probably the second most important driver of success in any customer experience programme. What gets measured and rewarded, gets done.

4. PLAN THE ACTIONS.

Again, do this at two levels:

– At corporate level. These are typically the key themes mentioned in the verbatim comments. Choose one or two big initiatives to work on during 2015. Keep it focused – any more than one or two initiatives will result in a dilution of effort.

– At client level. Each client will have its own specific set of issues including some ‘quick wins’ that can be addressed immediately.

5. MOST IMPORTANT, MONITOR PROGRESS.

If the results are poor, consider an interim assessment in 6 months (we call this a ‘Healthcheck’) but definitely repeat the feedback assessment after 12 months. If you don’t do this, you won’t know if you have achieved success.

None of the above five items are rocket science, but it strikes me as odd that some clients fail to take these actions. I know from 10 years of delivering Deep-Insight customer feedback to clients in the UK and the Netherlands, I know that these actions will result in better customer experience and improved feedback scores.

Are you going to NPS me? Yes, I am!

Posted on March 3, 2015 at 2:43 pm.

Written by John O'Connor

This is the topic of a talk I’m giving this week at a customer loyalty conference in Melbourne. It is in response to another talk entitled “Are you going to NPS me? No I’m not” in which Dr Dave Stewart of Marketing Decision Analysis will be presenting the case that Net Promoter is a deeply flawed concept. Dave will say that NPS should be discarded by organisations that espouse customer advocacy. To be honest, Dave’s position is close to what I thought of the Net Promoter Score concept when it was first introduced by a pretty smart academic and business consultant called Fred Reichheld back in 2003.

Net Promoter Score

Reichheld’s basic premise was that you only need to ask one question in order to understand if a customer is going to stay loyal to you or not. The question is: “How likely are you to recommend us to a friend or colleague?” Fred, being the excellent marketeer that he is, proclaimed the benefits of this Net Promoter Score (NPS) concept in respected publications like the Harvard Business Review. He then promoted it in his own book The Ultimate Question which came out in 2006, shortly after I took on the CEO role here at Deep-Insight. Since then, NPS has became very popular as a customer loyalty metric.

However, NPS has also attracted some heavy criticism. Tim Keiningham gave NPS a particularly scathing review saying that he and his research team could find no evidence for the claims made by Reichheld. (It should be said that Keiningham worked for the market research company Ipsos so his views may not be completely unbiased.)

At that time, my own view was that NPS was probably too simplistic a metric for business-to-business (B2B) companies. I also felt that Deep-Insight’s own customer methodology – which also included a ‘would you recommend’ question – was a much better fit for complex business relationships. And if I’m honest, there was an element of ‘Not Invented Here’ going on in our own organisation as well.

So we decided to ignore NPS.

The Rise of NPS

But here’s the thing: our customers didn’t ignore it. When we ran customer feedback programmes for customers like Reed Elsevier and Atos in the UK, ABN AMRO in the Netherlands, Santander in Poland, and the Toll Group in Australia, they would all ask: “Can you add in the NPS question for us – we have to report the numbers back to headquarters?” Of course, being the good marketeers that we were, we duly obliged. However, we always gave the results back in a separate spreadsheet, so that it wouldn’t contaminate our own reports and our own wonderful methodology!

Roll the clock forward to 2013. NPS still hadn’t gone away. In fact it had become even more popular. It was particularly popular with large international companies where a simple understandable metric was needed to compare results across different divisions and geographical areas. And when I finally looked into it, I discovered that Deep-Insight had actually been gathering NPS data from customers across 86 different countries since 2006.

Is NPS a good predictor of loyalty?

Around the same time we also did some research into our own database to find out what really drove loyalty and profitability in our clients. Now this is not an easy thing to do, as many of you who have tried will know. But where we had several years of customer feedback data, it was relatively straightforward to analyse how many of our clients’ B2B customers were still with them. If they have deliberately defected, we investigated if that defection could have been predicted by a poor Net Promoter Score, or by any of the metrics in our own CRQ methodology.

I have to say that the results were quite interesting. A low ‘Likelihood To Recommend’ was not the BEST predictor of customer defection. However it was actually a pretty good predictor. Deep-Insight’s overall Customer Relationship Quality (CRQ) metric was a slightly better predictor.

A poor Commitment score – one of the key components of CRQ – was the best predictor of whether a B2B client was going to defect to the competition or not.

So there we had it: NPS did actually work.

It worked not because it’s the BEST predictor of whether a client was going to defect, but because it’s a GOOD predictor, coupled with the fact that NPS has been embraced by some of the world’s leading organisations as an easy-to-use and internationally-accepted customer benchmark.

At Deep-Insight, we came a little late to the party. We only incorporated Net Promoter Score into our customer methodology in early-2014. Today we find that the combination of NPS and our own CRQ metrics works really well for our clients.

The future for NPS

Now let’s go back to the cartoon at the top of the blog (and thank you to the wonderful Tom Fishburne for allowing us to use it). If there is a statistically purer methodology than NPS, why not use that instead?

The answer is simple: most senior executives aren’t interested in re-inventing the wheel. They are much more interested in taking the feedback from their clients and acting on it, so that they can protect and enhance the revenues they get from those clients.

So for those B2B executives who are wondering if NPS is the right customer metric for them or not, I would suggest that you’re asking the wrong question. What good CEOs and Sales Directors are asking these days is:

“If my Net Promoter Score is low or if I have a lot of Opponents and Stalkers as clients, what do I do?”

In fact, the really successful CEOs and Sales Directors are spending the time thinking about the challenges of putting a really effective customer experience (CX) programme in place, rather than worrying about the purity of the metrics. That’s what you should be doing too.

5 Things To Remember To Get Your Completion Rates Up

Posted on November 14, 2014 at 2:54 pm.

Written by John O'Connor

One of the questions we get asked a lot is: “What sort of completion rates do you guys normally get on an assessment?”

Well, the answer is that it depends on what sort of assessment you’re talking about – we provide feedback on relationships with customers, channel partners and suppliers, and the completion rates differ from one type of assessment to the next:

-For employee assessments, our typical completion rate is in excess of 90%.

-For corporate customer and channel partner assessments, it’s typically 35-40%.

-For supplier assessments, the average completion rate are somewhere in the middle: 60-70%.

The next question we get asked is “Is it really that high?”

Well, we mainly get asked that question in connection with customer assessments, as some of our clients think 35-40% sounds impressive. This is particularly the case when people compare our figures to the ones you might get on a typical consumer surveys, where sometimes as few as 2% of consumers will bother to complete a questionnaire (Petchenik & Watermolen, 2011).

Remember that we are talking about existing, often long-standing, business-to-business (B2B) relationships – that’s what we do at Deep-Insight. We’re not a consumer research company. In fact, we’re not even a market research company, although we often are compared to firms like TNS or Gallup. We’re different. We look at – and assess – the quality of the relationships that large companies have with their biggest B2B clients. And if you think about it, why would good customers NOT want to provide feedback on their relationship with you, particularly if their account manager has convinced them that it’s an important part of their ongoing customer feedback process, and that their input is genuinely used to help improve the service given not just to them but to all clients?

The 5 pieces of advice I give to our clients are:

1. Spend Time Getting A Good Contact List Ready.

Most of our clients tell us they can pull together a list of key client contacts in a week. Two at the most. Our experience tells us that it takes at least 4-6 weeks to come up with a really good clean list of customer contacts who have a strong view of their relationship with our client. If the list isn’t compiled properly, we end up polling the views of people who really don’t have a strong view on the company, and who won’t be interested in responding.

2. Pre-Sell The Assessment To Customers.

One of our clients has been achieving customer completion rates in excess of 70% on a consistent basis for the past number of years. It does this because the CEO – together with the account managers – has managed to convince his key accounts that the 10-15 minutes they invest in providing feedback WILL result in a better service. “Tell me what’s wrong, and I promise we’ll do our best to fix it.”

3. Make Sure to Contact Customers While The Assessment Is Live.

We normally hold our assessments open for two weeks and we know from experience that if account managers have been properly briefed to mention the assessment in every conversation they have with a client during those two weeks, the completion rates will improve dramatically.

4. Manage The Campaign Smartly.

This is not rocket science, but you would be amazed at the number of companies that want to run assessments over school holiday periods, or during particular times of the year that may coincide with the most most busy time of the year for their customers. Plan your launch dates in advance, and think about the timing for issuing reminders. We usually recommend launching a customer assessment on a Tuesday morning, with the final reminder going out on the Tuesday two weeks later. That means that even if somebody is out of the office for two weeks, they’ll still have an opportunity to provide feedback.

5. Don’t Panic At The End of Week 1.

We normally see a flurry of activity during the first six or eight hours of a B2B campaign and typically the completion rate after Day 1 is about 8%. At the end of the first week (before we send out a first reminder) it’s often the case that the response rate hasn’t broken through the 10% barrier. This is not unusual. Completion rates will increase and a message in the final reminder that “This assessment is closing today” usually elicits a final flurry of responses!

As I said, a lot of this isn’t rocket science but it does require a bit of advance planning. If you do put the effort in up-front, you’ll see it rewarded in significantly higher completion rates.