Views from the Hills by R. E. Stevens, GENESIS II (The Second Beginning) E-Mail

The Sin of Gluttony

No, not the gluttony we are very famliar with, not gluttony of the stomach but guttony of information, gluttony of the mind.  I was reminded of this sinful act in an email from Dr. Herb Sorensen.  He was relating a concern stated by someone at the latest Casro Meeting.  It seems that this particular person was being asked to conduct a 45 minute telephone interview among "Attention deficit Disorder" respondents.  To start with, a 45 minute telephone interview among almost any group of people is too long, let alone people with A.D.D.

I the presentation "Researching Research," where I discuss the weaknesses within market research and the root causes of poor research, I disucss extremes in data collection.  My favorite example is the questionnaire sent via mail to me to be completed and returned in one week.  This questionnaire consisted of a 9 by 12 inch book containing 97 pages covering almost everything in daily life.  It covered purchases of appliances, clothing, health care and beauty care supplies, snacks, dinners, beverages, food, travel, tools, insurance, movies and television viewing.  Could you remember how many units of a food item you purchased int he past 6 months?  Do you remember how many times a month you watched "ABC College Basketball," or CBS College Basketball" or "ABC NFL football" or "CBS NFL Football?"  Don't they know that we remember watching the TEAMS, not Networks?  I thought this was an aberrant study until I found six similar studies within a week.  Can you imagine selling this information as valid data?

The results of the above study were peddled as real data from real market research.  This study may appear ridiculous.  It is, but it is not unusual.  I see studies every day that are a clear indication of the originator trying to get every possible piece of data from a study without regard to the possibility that the additional data may be biasing other data collected int he study.  For instance, it is not unusual to see a person conducting a sequential monadic study where after the first product has been used, the interviewer will ask a series of questions about specific attributes before placing the second product for use.  Now who with any training in research would not know that while using the second product the person is influenced by the attributes asked of the first product?  This leads to substantial order effects within the study.  And as any statistician will tell you if an order effect is present, you cannot draw any conclusions about the overall main effects in the study.  In other words you now have two answers for each product in the study.  One for when it was used first and one for when it was used second.  Most researchers will utilize the first product results, since they were not externally influenced and pitch the second product results.

How often do we ask questions that are just nice to know but are not relevant to the current problem?  How often do we ask questions because we always ask them regardless of the need?  I have seen many times where in a laundry detergent study involving only odor differences, questions are asked about cleaning, dissolving, whiteness, stain removal, etc.  I wonder how many incompletes would have been completes if we only asked the relevant questions?

From my point of view, we need to put our questionnaires and interviews on a diet.

Sponsor:  Sorensen Associates Inc      Portland, OR  800.542.4321        Minneapolis, MN  888.616.0123
The In-Store Research Company

[Back] [Index] [Forward]