Looking in All the Wrong Places

The above is a line from an old song, a very old song, but it is relevant to events I have experienced during the past two weeks.  I got a call from a Law Firm.  They have a case that has involved data collected through market research.  So they called me about being an expert witness.  I did not want to invest the necessary time that it would take to be an actual witness, but agreed to look at the data, read the depositions and also to suggest a few people that might be interested in being the expert witness for them.  Actually, I thought that they would be much better suited as expert witnesses than I would.

After reviewing the market research papers, two reports, one dealing with the data from the research (about 90 pages), the management summary (about 15 pages) and a 400-page deposition, I was reminded of the line from the song quoted above.  It seemed that they spent a lot of time and money doing research in one of the least desirable areas -- Malls.  But of course, they are cheap and fast.  The data reflected just how cheap the research was in terms of quality, too.

I was also reminded of an old high school joke.  Remember, my high school days were over a half century ago.

It seems that a man walking down a residential street in Cincinnati encountered a Hillbilly on his hands and knees at an intersection.  The man asked the Hillbilly what he was doing down there.  To which the Hillbilly replied, "I'm looking for my house key.  When I took out my car keys, I dropped my house key.  The stranger decided to help the Hillbilly look for his house key.  After about 15 minutes with no luck in finding it, the stranger started asking a little more about how the Hillbilly dropped the key.  When asked specifically where he was standing when he dropped the key, the Hillbilly pointed down the street.  To which the stranger asked why then was he looking way up here?  The Hillbilly replied that there was better light at the corner of the street than in the middle of the block.

The above story sounds ridiculous, but don't we do something similar every day?  How often do we select a test protocol in our research because it is cheaper, faster or easier than a protocol that we know yields much more valid data?  In the research that I reviewed for the law firm mentioned above, it certainly was fast, easy and cheap, but was it reliable and valid?  I don't think so.

I am grateful to the law firm.  In appreciation for my time, they sent a $750 check to the Mary Rose Mission, a non-profit hospice for people without insurance and/or family.

For an example closer to home, during my P&G days, we knew that our most loyal users were purchasers from the lower income area, lower on the socio-economic scale, less educated, etc.  I don't know if things have changed but I do know that P&G frequently talks about their consumer research via the Internet.  To use this method 14 years ago, we would have been ignoring our core users.  Maybe things have changed.  I do know that P&G's product category efforts are moving to categories like the health & beauty aids where the loyal user trends are probably different than those of the package soap category, i.e. higher income, better educated, etc.  (Note:  I still believe in P&G and their methods.  My portfolio contains about 45% P&G stock.)

The Hillbilly cited above has changed much like P&G.  He now parks his car at the intersection under the street light.  Where are you looking for your test participants?