August 10, 2006
- by Robert E. Stevens, GENESIS II (The Second Beginning) E-Mail: email@example.com
There is an old quote from the 1950’s that seems to still hold true today. That is, "If the only tool you have is a hammer, you will treat every problem like a nail." Recently, I was reading a large national marketing magazine where statistical testing of research results was being discussed. One paragraph in an article negatively affected my acceptance of the position being proposed. The paragraph was as follows:
"Advancements in internet technology have made consumers more accessible to researchers in survey research. Obtaining large representative samples of almost any population can generally be accomplished quickly and cost-effectively. The positive impacts of these advances in marketing research have been an increase in robustness and reliability of samples."
But what would I expect them to say since their business is internet research?
Over the years, I have written about problems many of us have encountered with different methods of recruiting but most importantly with Mall Intercept and Internet research. I frequently cite Gary Mullet's article in the February 2003 issue of Quirks where he points out that in his internet research he noted that approximately 50% of the respondents were other members of the household and not the Female Head of the Household as expected. Another article, January 2003 Quirks, titled "Sex, Lies and the Internet," by Michael Sack, cites the inconsistent eligibility responses from 65% of the sample. Are the respondents really who they say they are? We also have data that shows 50% of those starting a project will terminate it before it is completed. It is important to note that we have found that respondents that prematurely terminate the interview are less interested in the product or idea than those completing the study and thereby resulting in a positive bias in the research.
Problems with internet communication has come to light in the distribution of the Views from the Hills of Kentucky. The Views are distributed via two methods, internet and fax. I fax the Views and Sorensen Associates sends it out through the internet. Currently we have a system that shows who opens the email along with other distribution statistics. To me, what is important is that last year the Views email was being opened by about half of those to whom we sent the email. This year, that percent has dropped to between 36 and 40%. To date, our
efforts indicate that Spam Filters are the most likely culprit. It is important to keep in mind that the people on our distribution list are people who have requested to be added to the list, therefore, we would expect them to at least open the email.
My experiences with the internet cause me to approach its use as a research tool with great caution. Especially when the local television stations have advised their viewers to give false information in internet and telephone interviews, especially background information such as age and date of birth.
~ ~ Oregon: 800.542.0123 ~ ~ Minnesota: 888.616.0123 ~ ~ Pennsylvania: 866.993.0123 ~ ~
| Register for