Cost Saving Products (Written three years ago but delayed at the request of a friend.)

Recently a very good friend called to discuss a company policy concerning the evaluation of cost saving products.  As background, there was within the company concern that over the years constant cost savings initiatives lead to a gradual reduction of quality within their brands.  To counteract this type of problem they elected to make it harder for cost saving products to be approved.  Their objective was to insure, within reason, that the reduction in quality did not occur when attempting to maintain parity at a reduced cost.  To do this, the new initiative needed to beat the current version by at least a 53/47 advantage in a 300 base paired comparison.

While I agree with the intent of the restrictions, I do not agree with the process from a number of points of view.

First, I believe the tight (53/47 win) is unnecessary if we keep in mind that no new venture should ever be initiated based on a single study.  Decisions should be based on the aggregate of all data to date.  I cannot think of a single time when I did not have supporting research on any new idea by the time we were ready to put the new product on the market.  In most cases I had dozens of research projects assessing the merits of the new product.  Actually the final, before market, research study was really a confirmation study to confirm the previous research.  At this stage of development, if I did not get parity, I'd be looking to see if the current project or the previous research was in error.

Second, I'm concerned that the idea of doing a paired comparison preference test is really inappropriate for the final test of a cost savings product.  I would be inclined to use a difference test.  That is, to say, "Here are two products.  Use them and tell me how they are different, and if different, which do you prefer and why?"  The emphasis here is on difference not preference.  After all the idea of a cost saving project is to deliver parity.  If there is a difference seen, does the "how different" make sense?  If so, go back to the drawing board.

Third, I would recommend that a "Standard of Excellence" product be maintained where possible for reference.  The Standard can change as technology changes, but basically the "Standard of Excellence" would be a benchmark for the ideal that may be prohibitive in cost for the market or it could be the Standard Production Sample from a given time.

From a corporate viewpoint, I would be very concerned if I had a new formulation of a current brand where the new formulation saved 10% of the production cost and had to overcome a hurdle such as this.  Consider that the probability of a 53/47 observed result being a real 55/45 difference is greater than it being a true 50/50.  And the 55/45 result is a significant win while the 50/50 was our true objective.