Is it a Measure of Precision or a Measure of Accuracy?

After watching poll after poll during the election and listening to the commentators commenting on the meaning of the numbers, an element of concern started to creep in.  Do those people using the poll numbers really know what they mean?  And then I started to wonder about how many of us in the fields of marketing and marketing research really understand the numbers.  I am mainly focusing on the always present +/-3%.  You know, the confidence interval that is always quoted.  It seems that the commentators read these numbers as a prediction of the final election results.  I am aware that is how they are intended in the absolute, but is that really accurate?  I have been away from statistics classes for almost a half a century.  That is a long time but I don't think the fundamentals have changed all that much.  However, I have seen radical changes in sampling and a dramatic change in the willingness of the population to participate in our research.

Is the +/-3% a precision or an accuracy measure?  It seems that most people use it as an accuracy statistic.  That is, the true result will be within plus or minus three percent of the study's result.  I don't believe that is a valid use of the statistic, especially when we consider how research is being conducted today.  I think it is a precision measure only.  That is, it states that if we were to repeat the exact same study, the results would be within the plus three to minus three percent of the original test result.  In other words, the second study must contain all the biases of the original study for the statistic to be true.  It is really a precision statistic.

Some may say, "What is the difference?"  Consider this example:  We are interested in predicting the results of an election race through the use of an exit poll study.  Further, we interview 10,000 voters as they leave the election site.  Further, our interviewers only interview voters between the ages of 21 and 24.  We tabulate the results and report our results with a confidence of +/- 2%.  Does this mean that the final results of the election will be within the +/-2% of the study results?  No, it means only that the results of voters between the ages of 21 and 24 will most likely be +/-2% of the exit poll results, provided that those not willing to be interviewed voted the same way as those who were willing.  In other words, a limited sample should not be generalized to represent the entire population.

Now think about the studies we conduct on a day to day basis.

In a Mall study, the results reflect the choices of Mall shoppers who agree to participate in a study.
In a Telephone study, the results reflect the choices of people who will be interviewed over the phone.
In an Internet study, the results reflect the choices of internet users who will participate in a study.
In a CLT study, the results reflect the choices of those who will participate in a CLT study.
In a Supermarket study, the results reflect the choices of shoppers who agree to participate in an interview.
Etc.

Does our Management understand the risks?  Do our clients understand the risks?  Do we?

Note:  The above is a radical departure from the words commonly used in the textbooks.  However, I do not believe it is very far from the spirit of the texts.  In all textbooks, the authors will use some form of the phrase "in the absence of a bias."  I contend that most of our research is littered with bias.