Scientology, Psychology, and… Customer Satisfaction Surveys?

Disclaimer: The examples used in this piece are intended to lend clarity to the topic. Kaleo does not have an opinion on the Church of Scientology.  

Your Assumptions Could Be Wrong

Do you know your customers? Are they happy? It’s important that you know them. You issue customer satisfaction surveys and trust they provide insight into how well you’re doing. But what if they’re wrong? What if customer satisfaction is overstated? Would you prioritize your roadmap differently?

The truth is that this is likely the case. Your customers aren’t as happy as you think. And you aren’t prioritizing their needs as you should.

To Understand Why, Let’s Talk About Scientology

Say you’re big into self-improvement. You see a flier for an opportunity to “test the 10 key personality traits that determine your future success and happiness,” so you’re intrigued. You call to make an appointment.

You show and are greeted by a man named Cornelius. He escorts you to a room resembling a psychiatrist’s office. The face of L. Ron Hubbard and his publications adorn the walls and bookshelves. Cornelius hands you a 5-page packet. Its cover page reads, The Oxford Capacity Analysis Test. Sounds credible. It contains 200 questions. You open to the first page and begin.

• Do you sometimes wonder if anyone really cares about you?
• Do your past failures worry you?
• Is your life a constant struggle for survival?
• Are you rarely happy unless you have a special reason?

“Wow,” you think. “I actually have some pretty dark thoughts.”

You complete the test and await your results.

Cornelius returns with a graph. He explains that the test assessed you in three areas: how you view yourself, how you view others, and how you view life. He points to the chart. “This isn’t good,” he admits. “You’re extremely depressed and paranoid of other people.” You didn’t walk in feeling that way, but suddenly you do. “Luckily we’re in the business of raising graphs.” He continues and sells you on the benefits of Scientology.

What Just Happened?

The Oxford Capacity Analysis Test (no affiliation with Oxford University) is used by the Church of Scientology as a recruitment tool. The questions use a tactic called “positive test strategy” to intentionally elicit negative emotional responses. It primes the subject to be more receptive to the Scientology message, and increases the likelihood of conversion.

Questions can promote bias with what they draw attention to. A Canadian study illustrates this. Researchers asked one group if they were happy, and a second group if they were unhappy. Both questions should have revealed the same thing–the overall happiness of the groups. And both groups were sampled correctly, so the results should have been very similar. But they weren’t. They were shockingly different. The members of the group that were asked if they were unhappy were 375% more likely to be unhappy than in the other group. How could this be?

When you’re asked a question, your mind naturally searches for confirming examples rather than disconfirming. You’re a happy person. But when you’re asked, “Are you unhappy?” your mind immediately starts coming up with instances of unhappiness. Suddenly you’re very focused on your unhappiness. Examples are front-of-mind, so you’re much more likely to falsely describe yourself as an unhappy person.

How Is This Relevant?

This is why you need to be very careful about how you phrase questions in your customer satisfaction surveys. Too many contain questions like “how convenient is our company to use?” or “how satisfied are you with our services?” The results will be overstated.

If you have 10,000 customers and your customer satisfaction is overstated by just 5%, that’s 500 people who aren’t having a good experience, and their voices aren’t being heard. You may be patting yourself on the back when you actually have a problem.

What’s the Solution?

Rephrasing your question can eliminate the effect. Don’t push respondents down one path by asking how satisfied they are. Instead ask “how satisfied or dissatisfied are you?”

There’s a study that backs this up if you’re not convinced.

A group was asked, “if there is a serious fuel shortage this winter, do you think there should be a law requiring people to lower the heat in their homes?” 38.3% of respondents supported the proposition. Another group was asked the same thing, but “or do you oppose such a law” was added to the end of the question. These respondents considered reasons to support and oppose the bill. This time, approval decreased to 29.4% because people’s minds were not primed to think of only the positive effects.

Lesson Learned

Whether you’re in service management, customer success, or HR, you need an honest pulse on your customers to ensure that you’re serving their interests. Make sure not to ask questions that elicit certain responses. Don’t recruit IT fans like a scientologist.

Shameless Kaleo plug: if you need to improve customer satisfaction, click here.