Views from the Hills by R. E. Stevens, GENESIS II (The Second Beginning) E-Mail views@aol.com

Are We Losing Important Data?

Recently a company asked that I review a pending research project and offer suggestions. As it turned out, they had built into the protocol a frustrating problem encountered during my career at Procter & Gamble. Their protocol while similar, was not as severe as the one encountered at P&G.

In the current project, the eligibility for placement was that the respondent did a particular task two or three times a week. The test was for the duration of two weeks, so that under normal circumstances we would expect four to six uses in the test period. In the callback, the first question was, "How many times did you use the test product?" If the respondent used the product less than three times, they were told that there would be a callback later, after the respondent had more time to use the product. My concern was that I didn't know why the consumer changed habits or what the reaction might be to the request for more usage and the callback. My concerns can best be conveyed through the P&G example.

For years the Market Research Department at P&G had a rule that stated that a respondent in a consumer test had to use the test product for at least half of the test duration to be eligible to be interviewed. The basis for this ruling was fundamentally good but the execution left a lot to be desired. They only wanted people to give input into the study who gave the test product a fair trial.

My first encounter with this ruling was in the approval of a report to be distributed to management. When looking at the report, I noticed that the report base fell very short of the placement (about 67%). I asked why the low response? The answer was: not able to contact, refusal to be interviewed and a classification "invalid interview." I asked what was an invalid interview? These were the respondents who did not use the product at least half of the test duration (23% of the panel). My first response was "Why did they not use the product for the full test period?" I was told we didn't know and didn't ask. The policy was to ask the usage question and if they did not use it long enough, to terminate the interview; without asking "Why?" I had the respondents re-called and asked why they did not use the product for the full test period. Would you believe that 95% of the 23% stopped because the laundry detergent was damaging their clothes! This was the single most important piece of data we could have given the researcher (our internal client) and as a standard procedure, we didn't bother to ask!

Despite this event, it took two years to get the standard procedure changed. Once something becomes a standard procedure, it becomes very difficult to change the culture and procedure.

As I have said and written, "There is no such thing as a Free Lunch" with every action there is a down side, it may or may not be important, but it is there.


[Back][Index][Forward]