Research Rage

May 17th, 2013 | Posted by

As many of you know, I’m a very mild mannered person – very, very few people will have ever seen me get angry. But sometimes dodgy PR does just that – forget road rage, I get research rage!

The latest was triggered from an article that starts:

“Britain is a country full of niche shops and charming cafes that attract both locals and tourists from around the world. Unfortunately, some of these shops don’t accept card payments and aren’t realizing that they’re shutting the door to the majority of customers who no longer carry cash.”

“The research revealed that one-in-five consumers reported to have left a store in the past six months, and didn’t make a purchase because they couldn’t pay with their card.”

“Businesses in London are most likely to miss out if they don’t offer card payment services — half (50%) of consumers in the capital have left a shop in the past six months because they couldn’t pay by card. On average, Londoners walk out of shops 8.6 times a year because they can’t pay by card.”

As much as I’d like to have you all believe this, much in that first sentence is rose tinted glasses. But the journalist has a job to do, so we’ll let that slip. But the claim that 120m transactions a year are lost because some shops don’t accept cards stands out particularly.

Let’s pull this apart somewhat.

  • Numbers without context are meaningless. Last year, debit cards alone were used in 48% all retail payments, totaling 7.3 billion transactions. So the number we’re talking about here is less than 2% of card transactions – rather less dramatic
  • “On average, Londoners walk out of shops 8.6 times a year because they can’t pay by card. I pay with my card everywhere, and for everything”. I can’t honestly remember the last time that I couldn’t pay by card, let alone every 6 weeks that this implies. Another highly unscientific survey, but of the 10 people I asked this morning, none of those could think of an occasion either.
  • Making statements that many of us will automatically dismiss because we know they can’t be true (ie. 5 million Londoners have walked out 4 times of a shop because they couldn’t pay by card in the last 6 months is patently untrue) means we’re now likely to be as skeptical about all the numbers in the release.

You’ll note I’ve not mentioned the magazine nor the company concerned. My bigger point is that there are many others who are just as guilty in this regard, and I suspect they don’t even know it. Just as Donald and Stephen laid out in the report that there are do’s and don’ts in doing demos , there are simple things not to do in research.

Here are just a few.

#1 leading questions are not necessarily unacceptable (but certainly not preferable), but misleading with results aren’t. We get that you you’ve got a solution to sell – just don’t invent a problem or a story just for it to solve.

#2 don’t confuse or conflate cause and effect – too often companies make leaps from a data point to the solution, without proving the connection. And often there isn’t. That highlights either there isn’t a problem or poorly designed research or both.

#3 what value is being generated? Stating what is intuitively known already doesn’t add value. In this instance, saying by not accepting cards you’re missing out on card transactions adds no value. Understanding why they don’t accept cards is the more valuable insight.

So what should you do?

Well, use Celent is an obvious one! We can help at various stages of the process from questionnaire design to results interpretation to the resulting PR. Equally, use a research company who follows the appropriate professional codes of practice, and consider using a PR firm. But most importantly, have a clear set of objectives – “sell more” isn’t sufficient – which can be measured to see how effective the campaign has been. Whilst doing research is getting cheaper, it isn’t free.

And finally remember the golden rule with research – 38.2% quoted statistics are made up on the spot anyway!*

 

*Including this one

Tags:
No comments yet.