Views from the Hills by R. E. Stevens, GENESIS II (The Second Beginning) E-Mail

The Liar Factor

Two weeks ago I gave a talk on Unique Testing Protocols to a group at a major food products company.  Leading into the discussion of protocols, I introduced some of the reasons for the protocols.  The basis of these protocols was to eliminate or at least reduce problems encountered in research.  One of the four major problems discussed was that of Sampling.  Sampling as it relates to identifying the sample, contacting the sample and finally obtaining cooperation.  Within this structure, sampling is a very big problem in today's approaches to research.  But it appears that my concerns are only the tip of the iceberg.

Since giving the talk, I have been exposed to the "Liar Factor," a problem I did not address.  Exposed may be putting it mildly.  During the past week I have been trying to catch up on my reading.  The following are just a few of the articles relating to the Sampling problem.

In the January Quirks, Mr. Michael Sack, president of Image Engineering, writes about the "Liar Factor" encountered in an Internet Usage study.  A total of 500 respondents were recruited from 10 Nationwide sites.  Multiple agencies were used in the recruiting.  There were about five eligibility questions but basically they wanted women between the ages of 30 and 60 who use the internet at least once a week and searched for health related information.  The respondents were screened twice, over the phone and at a CLT.  Even after dual screening, 66% gave inconsistent responses to at least one of the eligibility questions.  Some could not even use a computer mouse, while others lacked the skills to do a computer aided interview.

In the February issue of Quirks, Gary Mullet of Mullet Associates writes about a food-item Internet study, where a follow-up interview was done to verify that the respondent was actually the Female Head of Household.  Somewhere around 50% of the purported respondents claimed that they didn't even know about the study.  Seems that another member of the household, generally a teenaged child, was using the computer with mom's internet account.

These kinds of problems are not bad enough.  Now two good friends, Mike Courney (UGA/MMR graduate) and Ms. Janet Pizzarello (Sorensen Associates) each sent me an Internet article that says "Turn Your Opinions into Cash."  GetPaidForOpinions is a Marketing Research Service designed to put people in contact with marketing companies who need their opinion!  For just a one-time fee of $29.95, they will put you in contact with over 100 paid survey, focus group and market research companies.  As Mike says, "amazing, isn't it?"

Do you know who is doing your screening?  Do you know the source of the database?

There is no perfect sampling plan.  There are just degrees of imperfect.  It is our job to be aware of the problems and try to reduce the effects test biases.  After all it is research and research is all about estimating.

As my son would say, "Don't tell me the problem without telling me a solution."  While I continue to use resources like internet and malls, I prefer to recruit in supermarkets for obvious reasons.  There are down sides to in-store recruiting but when I am concerned with validity of sample, this is where I end up.

Sponsor:  Sorensen Associates Inc      Portland, OR  800.542.4321        Minneapolis, MN  888.616.0123
The In-Store Research Company 

[Back] [Index] [Forward]