The Sin of Gluttony
No, not the gluttony we are very famliar with, not gluttony of the
stomach but guttony of information, gluttony of the mind. I was reminded
of this sinful act in an email from Dr. Herb Sorensen. He was relating
a concern stated by someone at the latest Casro Meeting. It seems that
this particular person was being asked to conduct a 45 minute telephone interview
among "Attention deficit Disorder" respondents. To start with, a 45
minute telephone interview among almost any group of people is too long, let
alone people with A.D.D.
I the presentation "Researching Research," where I discuss the weaknesses
within market research and the root causes of poor research, I disucss extremes
in data collection. My favorite example is the questionnaire sent via
mail to me to be completed and returned in one week. This questionnaire
consisted of a 9 by 12 inch book containing 97 pages covering almost everything
in daily life. It covered purchases of appliances, clothing, health
care and beauty care supplies, snacks, dinners, beverages, food, travel, tools,
insurance, movies and television viewing. Could you remember how many
units of a food item you purchased int he past 6 months? Do you remember
how many times a month you watched "ABC College Basketball," or CBS College
Basketball" or "ABC NFL football" or "CBS NFL Football?" Don't they
know that we remember watching the TEAMS, not Networks? I thought this
was an aberrant study until I found six similar studies within a week. Can
you imagine selling this information as
valid data?
The results of the above study were peddled as real data from real market
research. This study may appear ridiculous. It is, but it is not
unusual. I see studies every day that are a clear indication of the
originator trying to get every possible piece of data from a study without
regard to the possibility that the additional data may be biasing other data
collected int he study. For instance, it is not unusual to see a person
conducting a sequential monadic study where after the first product has been
used, the interviewer will ask a series of questions about specific attributes
before placing the second product for use. Now who with any training
in research would not know that while using the second product the person
is influenced by the attributes asked of the first product? This leads
to substantial order effects within the study. And as any statistician
will tell you if an order effect is present, you cannot draw any conclusions
about the overall main effects in the study. In other words you now
have two answers for each product in the study. One for when it was
used first and one for when it was used second. Most researchers will
utilize the first product results, since they were not externally influenced
and pitch the second product results.
How often do we ask questions that are just nice to know but are not
relevant to the current problem? How often do we ask questions because
we always ask them regardless of the need? I have seen many times where
in a laundry detergent study involving only odor differences, questions are
asked about cleaning, dissolving, whiteness, stain removal, etc. I wonder
how many incompletes would have been completes if we only asked the relevant
questions?
From my point of view, we need to put our questionnaires and interviews
on a diet.
Sponsor: Sorensen Associates Inc
Portland, OR 800.542.4321
Minneapolis, MN 888.616.0123
The In-Store Research Company