Views from the Hills by R. E. Stevens, GENESIS II (The Second Beginning) E-Mail views@aol.com

Field Auditing

There are companies that field audit all of their studies, there are some that never field audit and there are others that are selective in their auditing. Even when auditing is conducted, some companies will hire an independent group to do the task. Where does your company fit into the auditing protocol? My position is that in the current environment, all studies should be audited.

There are even different levels of auditing ranging from some of the most complete assessment of the field operation to an almost casual glance. I am reminded of one of my supervisors in my early years at P&G. This particular supervisor would always schedule himself as auditor when the study was being conducted in an interesting city. That was not a problem, the problem was his level of involvement in the research. He was noted for being present at the start of the project (for only his introduction) and then disappearing until dinner. Seldom did he participate or observe the training of the interviewers or listen to any of the interviewing.

It is my position that auditing serves two goals when properly conducted. First, it insures the study is conducted in the manner intended. The second goal is to provide an on-going training for our people. The field auditing exposes the strengths and weaknesses of the field operation in general such as: recruiting, interviewers, respondents and even the ability of our interview writing/methods.

There are two levels of auditing, standard and in-depth. In the standard mode of auditing, we are concerned that present research is conducted as intended. We will assess quality of interviewer training, interviewer eligibility (especially security), the presence of all the necessary test materials, quality of the interviews, abilities of agency supervisors, and test security. The atypical auditing is conducted by the more experienced auditors. This auditing goes well beyond the specific study. For instance, we look further at security and recruiting.

In the security area, do they keep confidential information out of sight? This type of information includes production schedules, correspondence to and from other companies, results of previously conducted studies, interviews for other studies, test products and supplies for other studies. Do interviewers and other agency personnel discuss other studies and companies? Do the participants show up in groups? Do the participants know one another? Does the agency personnel know the participants by name? How is the trash handled? Are rooms such as focus group rooms cleaned before another company uses them?

A few examples may present a better picture of the types of problems we can encounter:

Test Results -- While attending focus group interviews at an agency, I noticed book cases (with no doors, open shelving) in the hall outside the client observation room. The hall and book cases were out of sight of anyone unless they were in the hall itself. The book case contained the results of all consumer research projects conducted during the past three years.

Test Results -- A dumpster setting on the curb outside of a major consumer research company was found to contain the current complete reports from four of their major clients. Also in the dumpster were papers on budgets, promotions, study proposals, schedules, correspondence with clients, etc.

Dummy Interviews -- In a door-to-door research project, the auditor was concerned about the production of certain interviewers. They were all staying at the same motel. He decided to mark the tires of the interviewers' cars. He found that while they completed more than the expected number of interviews, the cars were not moved that day. They were creating dummy interviews based on previously collected interviews.

Dummy Interviews -- We had to re-contact some of the participants in a study. When we called them we found that an appreciable number said they never participated in the study and never received the test products. Upon further investigation, we found that even though we had completed interviews from all those participants, the agency had never placed the products and even had many of them in their storage area.

Test Product Security -- While attending interviewer training at a field agency, I found samples in the rest room from four previous studies.

Recruiting -- Needed signs on the outside window of a field agency:

Recruiting -- This study was to include Female Heads of Household recruited in a particular location under very specific conditions. To determine if the recruiting procedure was followed, we asked at the interviewing stage how and where the respondent was when recruited. A major number in one of the cities involved respondents that were recruited at a cocktail party.

Recruiting -- In a study that was to involve random dialing recruiting, we observed that respondents were arriving in groups (four and five to a car). Upon investigation, we found that those arriving together all belonged to the same organization. The agency called specific organizations and gave them the eligibility and asked them to supply (for a fee) members that fit the profile.

There are many good research organizations in the business. It is our responsibility to insure these are the companies involved in our research projects. Contracting the research does not remove our accountability for the results.


[Back][Index][Forward]