Comments on: How the US Open was like a retail promotion analysis

By: Is elitism the source of poor usability? | Retail: Shaken Not Stirred

Is elitism the source of poor usability? | Retail: Shaken Not Stirred — Thu, 07 Jan 2010 01:01:06 +0000

[...] the site. The two key phrases here are “statistically significant” and “trendable.” Per my last post, continuous measurement is important to avoid random outliers and uncover the underlying truth. [...]

By: Are web analytics like 24-hour news networks? | Retail: Shaken Not Stirred

Are web analytics like 24-hour news networks? | Retail: Shaken Not Stirred — Thu, 07 Jan 2010 00:57:49 +0000

[...] I realize the temptation is strong to get an “early read” on the progress of a new site design (or any strategic issue, really). I’ve certainly felt it myself on many occasions. However, since just about every manager and executive I know (including myself) has a strong bias for action, we have to be aware of the risks associated with these “early reads” and our own abilities or inabilities to make conclusions and immediately react. Early reads can lead to the bad decisions associated with the full accelerator/full brake syndrome I’ve referenced previously. [...]

By: Kevin Ertell

Kevin Ertell — Thu, 02 Jul 2009 23:25:25 +0000

You make some fine points, Andy. It’s not only important to define a clear strategy, it’s also important to give that strategy a chance to work by measuring its effectiveness over a reasonable period of time. Along the way, tweaks could certainly be necessary, but frequent wholesale changes in direction can have a detrimental long term effect.

By: Andy Orr

Andy Orr — Wed, 01 Jul 2009 18:59:43 +0000

I think the knee jerk reactions are usually a result of either a poor strategy or more often the lack of a strategy all together. All too often the strategy seems to be to “drive more sales this weekend”. That’s not a strategy it is a desired result. The strategy is about how you intelligently expect to do that over the long run.
When you don’t have a thoughtful strategy, it’s far too easy to change the tactics week after week, based on short term results which, as Kevin points out can be mis-leading.
A thoughtful strategy identifies the target audience and the specific behavior you are trying to change. For example a grocery store might have a core group of customers who shop regularly but never visit the deli or pharmacy. A thoughtful strategy to increase share of wallet from these customers might involve tactics designed to introduce them to your great pharmacy service or fresh cut deli meats at great introductory prices.
The success of these tactics is easier to measure (over the long term), is more likely to drive incremental purchases and is less vulnerable to the knee jerk reactions.
With a thoughtful strategy, sometimes tactics need to be tweaked for optimal effectiveness. Outside of a thoughtful strategy tactics tend to get thrown out altogether and you start from scratch each week.

By: Kevin Ertell

Kevin Ertell — Wed, 01 Jul 2009 15:45:35 +0000

Good comment, Mark. Krugman’s point is the same that measuring continuously is a better way to pull out true trends than simply looking at short term results. There are too many random variables in just about any measurement to be sure of the degree of accuracy of any one measurement. Thanks for pointing out Krugman’s article.