Dinner at the Blue Fox

One of my goals over the years has been to raise the level of awareness concerning the effects of the research environment on the test data.  That is, the time, place and physical environment all play an important part in the test results.  It is called the test effect and all tests contain some element of a test effect bias.  The test effect can never result in better data.  Our goal should be to minimize this effect wherever possible.

A couple of weeks ago my eldest son, Bob, sent me another book.  (He is still trying to educate his father.)  The book is The Invisible Touch by Harry Beckwith.  The book, which focuses on marketing, is loaded with stimulating perspectives.  In the chapter on "The more it costs, the better it seems," he cites a study conducted by Procter & Gamble in 1980.  I remember the study and I prefer to call it, "the Promotion."  I think you may find it interesting.

In the late 1970s, P&G was working on a new instant coffee.  For the younger generation, instant coffee was and I guess still is today some powdered junk that you add to hot water in an attempt to create a convenient cup of coffee.  The marketing department was given the task of finding a way of introducing this new version of instant coffee to the public.  how do you get people on camera to say this new instant coffee is really good and at the same time give the audience a reason to believe the spokesperson?

Let's set the stage for the testimonial, First, recruit people from the San Francisco area that drink coffee, have heard of the Blue Fox Restaurant, know of its reputation, and have never eaten there.  Now invite them to be your guest to sample the dinners at the Blue Fox.  Next make arrangements to serve the guests the new instant coffee.  With an environment like this, how do you think the guests will respond when asked about the coffee?

Consider the atmosphere, a fancy restaurant, free dinner, linen tablecloths, waiters finely dressed, real silverware, china dinnerware, etc.  The guests are focusing their attention on the dinner.  did they even consider that the coffee being served was the real focus of attention (the magician's ploy of mis-direction)?  When asked about the coffee, the responses were generally very positive.  What did you expect?  They were guests and would a guest ever tell the host that the coffee is "terrible" even if it was true?

I think there were guests that believed the coffee was poor but were being polite.  I also believe that there were guests that because of the circumstances, actually thought the coffee was "fabulous" as one responded.  The exercise was not meant to evaluate the acceptance of the instant coffee, it was to create an image of a good replacement for a fresh brewed coffee.

Not only did the resulting commercial paint a good picture of consumer acceptance but it also indirectly reflected restaurant acceptance by implying that Folger's Freeze Dried Instant coffee is served at the Blue Fox.

I think this is a very obvious example of how you can design research to give misleading results.  The bigger question is, "How many researchers are designing misleading protocols accidentally?"