Views from the Hills by R. E. Stevens, GENESIS II (The Second Beginning) E-Mail views@aol.com

There are Better Ways to do Research

A friend stumbled across a concept study in progress and thought I would be interested in what he observed. This concept study was being conducted in a very public place. The interviewer had set up a table in a heavy-traffic area and proceeded to recruit consumers for her study. During her breaks and lunch period, she left the table with all supplies open to the public (problem #1, no security). While the interviews were in progress, all seven of the concepts were in open view for the respondent as well as the general public (problem #2, no longer a monadic test). The interview involved a questionnaire (page size 8½ x 14 inches) consisting of 24 pages of questions with very few skip patterns (problem #3, How do we separate the good data from the bad?) I also wonder how many incompletes were encountered in this study?

I called the Brand Manager responsible for the project. He didn't know it was being conducted under the above-described circumstances. Neither the market Researcher in the company or the Owner of the Field Service were aware that the location was changed. According to the Owner, neither did his field managers or the people in charge of the Agency that was subcontracted in the city (problem #4, no auditing). It seems someone in the subcontracted agency took it upon themselves to accelerate the inter- viewing process moving to a heavy-traffic, public area.

The first two problems are easily rectified, but problems #3 and #4 are becoming more prevalent with the reduction of research funds resulting in people trying to answer all the world's problems in one study while minimizing the overall cost of the study. My biggest concern is not that they use this course of action. It is that in all probability they do not have any concept concerning the validity of such action. I had a long conversation with the owner of the agency about this point in particular. He is well aware of the validity problem but he can only attempt to alert the client to the problem. He cannot refuse to conduct the research as described or the client will only go to another agency that will conduct the research. Is it that my concern for validity is in error, or the brand managers are unaware of the validity problem, or are they aware but are willing to collect numbers without regard to validity?

There were other problems with the specific interview other than length. One of the most obvious was terminology. For instance, without telling the respondent, they classified bread sticks, breakfast bars, and rice cakes as "salty and savory snacks." Again without telling the respondents, they classified cereal bars and granola bars as sweet snacks. Both of these examples were in very important questions (two of the first four questions asked).

What appeared to be the second and third most important questions in the study appeared on pages 13 and 14. This seemed awfully late in the interviewing process to obtain good data.

It is my opinion that studies of this type are not only a disservice to the sponsoring company, their market research group, and the field service, it is also a disservice to the respondents and other companies that rely on the cooperation of the consumer for good research. After this experience, I doubt if any of the participants will ever volunteer to participate in another research study.

These are problems, but what are the solutions? Her are three to start with:
Solution #1-- Audit your research. As I have said before, the poorest view of the world is from behind your desk. Know what is taking place in the field. The client and the agency should be auditing. You can delegate, but you are still accountable.
Solution #2-- Before taking a project to the field, pilot test it. The marketing manager should conduct at least one or two interviews. You can use company employees (from other departments) as respondents. They will tell you if the interview is too long and if there are words and phrases they do not under- stand.
Solution #3-- Assuming that a 24-page interview is really required, pay the price for good data. My suggestion would be to do an in-home interview where the respondent knows the time involved and is willing to spend the time. You may have to pay $30 or more for their service, but if it is necessary, spend it.

When I was hired at Procter & Gamble, they had a policy that required anyone coming out of school to the Market Research Department, to spend six months in the field doing door-to-door interviewing. At the time, I thought, "What a waste," but as the time passed I could see the value. I also thought more people in the company should be given the same learning experience. It reminds me of a comment made recently by Allen Rau, a former P&G-er and a good friend, which is, "If you think education is expensive, try ignorance."

Please accept my apology for going over one page, but in this case I could not adequately reflect the situation without the extension to a second page.

A separate thought:
The more similar the products, the more important the differences.  Accentuate the trivial.

[Back][Index][Forward]