The Two Faces of Evaluative Research

From my perspective there are two sides to evaluative research. The first side deals with the absolute assessment of the product while the second side deals with the perceptual assessment of the product in the market environment. In the first side we are interested in the assessment of the basic product, overall evaluation, likes & dislikes, and attribute evaluations. In the second side we are interested in the same evaluation but made under the context in which the product will be sold (i.e. the positioning of the product, the brand name).

The concept of the two sides is difficult to communicate without an example. Suppose you were considering introducing a new design. In the first case you would be interested in evaluating the watch in the basic state, that is, without a name or the identification of the manufacturer. However, since this is not how the watch will be sold, you would also conduct an assessment evaluation under the brand name.

In the first case we would be evaluating the basic watch, its strengths and weaknesses. In the second case we would be evaluating it under the consumer's expectation of the brand. I would not expect a watch to yield the same results when evaluated under the Timex name as it would under the Seiko name. Would I expect to obtain the same results in a blind test and a brand identified test? I wouldn't, would you? I believe both blind tests and brand identified tests play a very important part in our assessment of market viability. They both are valuable and both play very different roles in the development of a product. I not only expect the overall evaluations to be different but I expect performance or attribute ratings to reflect even greater differences.

Basically I used the blind testing approach as a learning, understanding and development tool. It is ideal for uncovering negatives. The brand identified approach lets me know if the product is appropriate for the brand. The product may be very good but not at all appropriate for a particular brand.

A very good friend recently tested the same product in a blind format and identified, under the brand name of the number one brand in the category and also under the name of the number two brand. The results showed substantial differences among all three versions (base = 480).

                                      Blind     #2 Brand     #1 Brand

Top Two box -- Intent to Purchase     65%       53%          64%

            -- Overall Rating         55%       53%          67%

            -- New & Different        32%       15%          31%