Views from the Hills by R. E. Stevens, GENESIS II (The Second Beginning) E-Mail views@aol.com

Further Research - Users vs. Testers

Over the past eight years, I have written a number of Views on the differences between testers and users.  Unfortunately, I had not seen much to write about coming from other researchers.  That has ended with an article appearing in the May 2001 issue of the Journal of Marketing Research, by Ofir and Simonson.  The title of the article is, "In Search of Negative Customer Feedback:  The Effect of Expecting to Evaluate on Satisfaction Evaluations."
 

Ofir and Simonson cite that given the increasingly common practice of asking for evaluations, buyers of products and services often know in advance that they subsequently will be asked to provide their evaluations.  The authors demonstrate that expecting to evaluate leads to less favorable quality and satisfaction evaluations.  The negative bias of expected evaluations is observed when actual quality is either low or high.  The findings are that unless buyers begin the evaluation task with low expectations, they tend to focus during consumption primarily on negative aspects of the product/service quality.

The authors cite in their practical implications section that marketers should take into consideration the degree to which survey respondents expect to evaluate while using the product or service.  More generally, marketers need to be aware of the negative bias of expected evaluations when interpreting the results.  For example, companies that survey the same panel of customers may obtain systematically biased findings.  It is not clear that the negative aspects highlighted by those who expect to evaluate are relevant to and representative of the evaluations and satisfaction of other customers.  Therefore, responding to criticisms of customers who expect to evaluate may be useless and even damaging.

While the authors were not directly addressing the difference between people who participate in tests and those who are just using the product, I believe that their observation that those who expect to give an evaluation of a product should be more relevant in the case of test participants.  I do not believe it takes a great leap of faith to believe that when you ask a person to participate in a test, that they expect to evaluate the product or service involved in the test.  The effect observed by Ofir and Simonson in the normal course of events should be even more apparent in the situation where we ask a person to participate in a test.

How often are our test results reflecting negatives that would not be considered or possibly even seen in the normal course of usage?  Some researchers would say that we should always be aware of the potential negatives.  I agree, but negatives must be put in perspective.  All products will contain negatives.  Some negatives may even enhance the image of a brand.

There is a time and a place for both User Results and Tester Results.
 
 


[Back][Index][Forward]