“If it ain’t broke, you ain’t looking hard enough”

The poor economy has done nothing to lower customer expectations of online retailers, and recent mixed results data from ComScore and ForeSee Results indicate that retailers who continue to improve their customer experiences are pulling away from their competitors in both sales and customer satisfaction.

ComScore reports online retail up 4% for the holiday season. While an increase is always nice, this is a much lower growth rate than online retail has seen in the past. And last year’s comparison base was far from stellar. ForeSee Results shows a significant drop in customer satisfaction year over year. Since satisfaction is predictive of future financial results, a drop is concerning.

But still, I wondered how sales could be up at all if satisfaction was so far down.

A deeper look at the ComScore data shows the Top 25 retailers growing 13% while “Small and Mid Tail” retailers are declining 10%. Satisfaction scores are also split, but the differences we’re seeing seem to be more based on those retailers who are continually improving their sites versus those whose cost containment measures have slowed or stopped improvements. It appears that the retailers who closely measure the effectiveness of their sites from their customers’ perspectives and continuously improve their customers’ experiences are the retailers with increasing customer satisfaction scores. Those retailers who didn’t improve customer experience this year are suffering declining satisfaction scores. Many of those in the Top 25 are the retailers who have continued to enhance their customer experiences. Those enhancements are not only helping them to increase their sales, but because of the high visibility and usage of those tops sites, they’re also raising consumer expectations of all sites.

Customer satisfaction can be best defined as the degree to which a customer’s actual experience meets his or her expectations. Therefore, rising expectations can depress satisfaction scores if customer experience improvements don’t keep pace.

In the rapidly changing world of online retail, stopping or delaying improvements is like treading water in a swimming race. While you may temporarily save some energy, you will fall hopelessly behind and your only hope of catching up is spending a lot more energy than you likely saved treading water

Growing online retail businesses realize and fully embrace the need for continuous improvements, and they also realize that online retail in general is far from producing the level of customer experience truly necessary to provide excellent self-service shopping experiences. I recently heard Robin Terrell, Managing Director of John Lewis Direct in the UK (and Amazon alum), say “If it ain’t broke, you ain’t looking hard enough” in a talk about the need to improve customer experience. It’s a brilliant statement, and I totally agree with what he was saying.

So, “improving customer experience” is a huge and vague statement. Where do we start?

  1. Recognize that it’s broke and you ain’t looking hard enough
    We’re still in our infancy in online retail, and we’ve got a long way to go. We too often try to increase our sales by generating more traffic and don’t spend enough time converting the traffic we’re already got. Often, the obstacles to conversion are not the big, shiny, whiz bang functionality; they’re lots of little things that add up to big problems. Those problems are hard to see without a concerted effort, as I discussed in more detail in my Tree Stump Theory post and other posts on conversion.
  2. Truly learn how effective your site is from your customers’ perspective
    We can all identify lots of improvements we’d like to see on our sites, but it’s the improvements our customers most need that will drive our best growth. So understanding where we are and aren’t effective from our customers’ perspectives is critically important, but difficult.Focus groups and usability labs can be very helpful, but they can’t be our first or only methodology because it’s not possible to project learnings from a small group of people onto our entire population of customers.

    First, we need to quantitatively understand our effectiveness in the eyes of our total population, and that requires a statistically solid customer polling and analysis capability. Blatant and shameless plug alert: I’ve had great success using ForeSee Results in the past for exactly this purpose. Once we understand problem areas at a macro level, we can add a lot of color by interacting directly with customers in focus groups and usability labs. More details on this process can be found in my post entitled “Is elitism the source of poor usability?”

  3. Consider getting some help from usability professionals
    Usability audits are different from usability labs. Usability auditors are professionally trained to understand how people interact with websites. Many of them have degrees in Human-Computer Interaction, a field that truly seeks to understand how people interact with software. These types of people can really help to identify problems with our user interfaces that untrained eyes have trouble seeing but which regularly obstruct customers from accomplishing their tasks.
  4. Put in place a process to continuously improve
    This is really about budgetary and project management mindset. We must just accept the fact that we can’t tread water in a never-ending swimming race, and our only chance of competing is to keep swimming. We have to build our staffs, our budgets and our processes with the recognition that competing in the marketplace means continuously improving our customer experiences. Which leads to …
  5. Wash, rinse, repeat
    Since the leaders in the marketplace are running this same cycle, we cannot rest. We must continue to recognize our sites are broken, continue to measure our effectiveness from our customers’ perspectives, find problems, fix them and begin again.


We’ve got a lot of data that shows that retailers who best satisfy their customers generate the best financial results. I suppose that statement doesn’t sound like rocket science. But understanding that satisfaction has a direct relation to expectations and that our customers’ expectations can change independent of what we do on our own site is important. The leaders are continuously improving their sites, and they’re improvements are raising our customers’ expectations. We’ve all got to swim harder to keep pace.

What do you think? What’s your view on the marketplace? How have you see customer satisfaction affect your business?


  • By Kevin, December 8, 2009 @ 11:36 am

    Great blog, Kevin!
    Your point that the larger companies are the only ones growing, versus the declines of SMB companies is interesting. They have the processes, structure, and budget to do it, but do you think SMB companies are selling themselves short?
    Software, optimization strategies, and social media applications can be used in a cost and time effective way to enhance the customer experience, conversions, etc…

  • By Kevin Ertell, December 8, 2009 @ 11:46 am

    Thanks for your comment, Kevin. To be clear, ComScore is reporting that the larger companies as a group are growing versus the SMB companies as a group. What we’re seeing at ForeSee is that companies who are continuously improving their customer experiences are the companies that are seeing satisfaction and sales growth. It happens that a lot of the Top 25 are running continuous improvements, with Amazon leading the pack. SMB companies that continue to improve can see benefits, and many of those improvements (like improving error messages, for example) don’t have to cost tons of money. What do you think?

  • By Jason C, December 8, 2009 @ 7:10 pm

    Great post Kevin. I think the concept of continuous improvement can vary a lot from organization to organization. For some, it means making daily or weekly changes to the site. For others, it means making changes once or twice a year. Regardless of how frequently the improvements are made, I think the argument can be made that continuous measurement is the key to ensuring that what resources are used on improvements are well spent. For those that make changes less frequently, it means being able to make a serious and credible business case for those resources when they become available.
    And…as you point out, many times the improvements don’t have to be major ones. Often times those small incremental improvements add up to a lot.

  • By Chris Eagle, December 9, 2009 @ 9:36 am

    Unfortunately, I’d guess there are two groups, those who get it and those who don’t.
    Those who get it understand that web sites need continuous work. Those who don’t get it think their web site is ‘finished’ and doesn’t need to be changed except for normal maintenance, just like their HR system or inventory management system. Simply put: all software rots, but web sites rot at warp speed and a lot of companies don’t even understand normal software rot. (http://en.wikipedia.org/wiki/Software_rot)
    I love the steps you point out but it seems to me, for a lot of the companies that are suffering decreasing CustSat, they need a step 0 – “It’s breaking all the time.”

  • By Kevin Ertell, December 10, 2009 @ 9:20 am

    Thanks for your comment, Jason. I agree that continuous improvement can mean different frequencies for different companies. To me, the key is to build a business model around the idea that continuous improvements are necessary to compete in the online marketplace. That potentially means thinking differently about resource allocations, organizational structures, budgets and processes. I absolutely agree that continuous measurement of effectiveness in the eyes of the customer is key to understanding when are where those continuous improvements should be directed.

  • By Kevin Ertell, December 10, 2009 @ 10:12 am

    Thanks for your comments, Chris. I hadn’t heard the term “software rot” before, but it makes a lot of sense. I read the wikipedia article you posted, and there’s some good information there. I definitely agree that websites rot much faster, especially because they can quickly seem dated and insufficient when compared so easily to other sites with better usability and capabilities.

  • By Kobi Korsah, December 11, 2009 @ 9:32 am

    When you’re right you’re right. I could not agree more. I would add that converting potential customer traffic into an assured revenue stream, retaining that custom and attracting more, depends on your ability to consistently exceed expectations of the buying experience. As customers interact with progressively sophisticated services – fuelled by the innovative use of web-technologies – they unwittingly reach through complex web apps into layers of IT sediment and the risk of service expectation gaps increases. Alertness and speed of reaction to customer satisfaction risks requires extraordinary service performance insight. With tools like the CA Wily Application Performance Management solution, companies can achieve desired levels of online customer satisfaction and maintain SLAs. This means better customer service, more stable revenue streams and higher productivity. You can explore this viewpoint further at http://www.ca.com/gb/content/campaign.aspx?cid=212132

Other Links to this Post

RSS feed for comments on this post. TrackBack URI

Leave a comment

Retail: Shaken Not Stirred by Kevin Ertell

Home | About