A Little Common Sense Goes a Long Way - Right Along with Confirmation
A couple of stories I heard and a few tests I have heard about enhance the above points.
I was told of a masked robber that was holding up a convenience store. After he had all the money from the cash drawer, he asked the clerk for a bottle of Jack Daniels form the liquor cabinet behind the counter. The clerk informed the robber that he could not give it to him or he would lose his job. The robber wanted to know why? The clerk told him that he, the robber, was under age. In frustration, the robber supplied the clerk with identification. Needless to say, it wasn't long before the robber's career ended.
I was also told of a sheriff in a semi-rural town that received a phone call from an elderly resident requesting that a deer crossing sign near her home be removed. She was concerned that too many deer were being killed.
I thought these were a little far fetched until a good friend told me that the deer crossing example actually happened in his city, too.
While on the surface, these events don't seem logical, especially to those of us intellectuals, we would never let something like that happen in our research. But then again maybe we would. Consider the following examples.
My wife and I received a questionnaire to fill out from a leading market research company. On the first page, actually the first question, they asked a series of questions about our household composition. For each person they wanted to know our date of birth, educational attainment, various habits like smoking and etc. What really threw me was in the first two sub questions of question #1, they wanted to know, of the Female Head of the Household, her date of birth and her sex (male of female). Right below that they wanted to know the Male Head of the Household's date of birth and sex (male or female). Am I missing something here in this line of questioning? What confidence does this line of questioning instill in the respondent?
Maybe we should consider the company that wanted to do an employee satisfaction study. They wanted to maintain anonymity so they decided that the questionnaire would be sent via email to the respondent and they could complete it and return it in a similar fashion, by email. Now who is playing with the short deck, the researcher or the respondent who completes the questionnaire and sends it back, thinking the responses are anonymous?
Consider a lengthy questionnaire (96 pages) I was asked to complete. This thing was loaded with problems. For starters, they asked how many times a month I watched football on CBS, ABC, Fox, and NBC networks. They also wanted to know how much attention I was paying and where did I watch the games? At my home, someone else's home, a hotel/motel or somewhere else. It is like they assumed my focus was on the network and not the teams playing. Also since the football season is a little over a half year, did they want me to adjust for a twelve-month year by taking the actual number of games per month and dividing by 2?
I can give example after example of this type of research but I believe you get the point. We at times, will put some very strange research in the field. Yes, I know that all too frequently the client has a great deal to do with the structure of the research. But it is our responsibility to eliminate or at least reduce this type of research. In my forty years, I made many of these types of errors. Most of them were early in my career. The reason the frequency was reduced as I gained experience was not just the actual experience but the fact that I learned that it pays to get a second opinion. Before putting something in the field, Beta Test it. Have others do the task and offer suggestions. Confirm that you have done what you intended to do and that you have communicated adequately to the respondent and you understand what the respondent is saying.
Sponsor: Sorensen Associates Inc Portland, OR 800.542.4321 Minneapolis, MN 888.616.0123