Views from the Hills by R. E. Stevens, GENESIS II (The Second Beginning) E-Mail views@aol.com

The Effects of the Order of Presentation

In a recent meeting, the topic of conversation drifted to Sequential Single Product Research or as others call it Sequential Monadic Testing.  The more I seem to get involved in discussions about testing in the Pair, Round Robin, or the Sequential Single Product Formats, I'm convinced that there is a limited awareness of the effects of order of presentation and what they mean.

In the most recent discussion, the response to order effects was:  "I don't have any problem with them since I balance the placement, that is, I have an equal number of A/B and B/A placements."  I would agree that the balanced approach is a correction if you have an average order effect.  If the order effect is an order by pair interaction or a product by order interaction, however, the balanced presentation of the data could be very inaccurate if averages are reported.

When raising the concept of an interaction, I usually get a "What is that?" response.  For those who are not familiar with the concept, I recommend the reading of  L. L. Thurstone's Model IV Analysis of Variance Papers.  They were written in 1933.  In his basic analysis, he touches on two concepts I believe are very important in market Research.  First, in his model for round robin testing are the components of "Average Order" and "Order by Pair Interaction."  The second concept is an element called, "Deviation from Substractivity."  This component is the basis for the concept of "Ruler and Pie Market Models."  For anyone studying the market potential of a brand, this is an extremely important concept.  While Dr. Thurstone's examples are related to predicting election results, the same theories apply to market purchase predictions.

But back to Order Effects.  Unfortunately, in pair tests and sequential single product testing, you do not have the degrees of freedom to assess which of the two types of order effects you are encountering.  However, they are still operative.  Therefore to say that your balancing placement order compensates for the order effects could very well be an inaccurate statement.

Let me try to give some examples of the two back types of order effects.

Average Order Effects:  This is the type of order effect that can be compensated for through the use of balancing.  Basically an average order effect states that either the first product used or the second product used has a distinct advantage regardless of which product is in that position.  A theory that has been voiced is that if the bias is a first product bias, it indicates that both products are seen as good products relative to the market.  If the bias is for the second product, it is usually viewed that both products are equal or poorer to the market in general.

Product by Order Interaction:  This interaction implies that the order effect of A/B is different than B/A.  This means that the results are order dependent.  It reflects that one product is controlling the bias and it is order dependent.  Generally this effect is the result of one product being superior in performance not only to the other test brand but the market in general (a performance superiority).  Another way this interaction can occur is if one product has a sensory advantage such as visual, tactile, or sound attribute.  something possibly unrelated to the performance directly but a psychological benefit.  this type of benefit is extremely dangerous to Sequential Single Product Testing.

First Example:  Consider the testing of Conventional Tide with Tide Plus Bleach.  If the Conventional Tide is used first, it may look superior to what the current homemaker is using and will be graded as such.  But when the Tide Plus Bleach is used and the redeposition and some staining is corrected, the difference between the two Tides looks very great.  However, when Tide Plus Bleach is used and the redeposition and staining is cleaned up, the performance of the Conventional Tide which follows does not appear to be much different.

Second Example of Order Interaction:  These types of effects are all over the board and are really difficult to describe in a one-page outline.  The problem depends on how easy they are to be seen and remembered by the user.  They can also be the result of something that the user does not even know is taking place (subliminal).  For the sake of discussion, I will use some examples that are easily seen.  These examples would be a particular problem with Sequential Single product tests where the first product is used and then evaluated before the second product is even seen.  As an example of this order by product interaction, consider the storage bag market where we have "yellow plus blue equals green," and "the zippers -- hear or feel the difference."  In these cases when the first product is the conventional storage bag, the respondent does not know of the new physical attribute, so the evaluation of the first product is in the user's vision of the the current world and the first product is rated as such.  However, in the reverse order the user is exposed to a new world (new attribute) and it affects the evaluation of the second product (current prototype/not new) negatively.  The result is that the New prototype ratings do not change with order but the current prototype gets a much lower rating when it is second in the pair order than when it is first.

Bottom Line:  If you have an order effect present, you have two answers to your question, not one (an average).

SOME TRUE CARTON INSTRUCTIONS:


 


[Back][Index][Forward]