True conversion – the on-base percentage of web analytics?

I just finished re-reading one of my all-time favorite business books, Moneyball by Michael Lewis. While on the surface Moneyball is a baseball book about the General Manager of the Oakland A’s, Billy Beane, I found it to be more about how defying conventional wisdom (a topic I’ll no doubt return to over and over in this space) can be an excellent competitive advantage. In retail, we can be just as prone to conventional wisdom and business as usual as the world of baseball Lewis encountered, and site conversion rate is an excellent example of how we’re already traversing that path in the relatively young world of e-commerce.

In Moneyball, Michael Lewis tells the story of Beane defying the conventional wisdom of longtime baseball scouts and  baseball industry veterans. Rather than trust scouts who would literally  determine a baseball player’s prospects by  how he physically looked, Beane went to the data as a disciple of Bill JamesSabermetrics theories. By following the  James’ approach, Beane was able to put together consistently winning teams while working with one of the lowest payrolls in the Major Leagues.

Lewis describes how James took a new look at traditional baseball statistics and created new statistics that were  actually more causally related to winning games. Imagine that! For example, James found on-base percentage, which  includes walks when calculating how often a player gets on base, to be a much more reliable statistic than batting  average, which ignores walks (even though we’re always taught as Little Leaguers that a walk is as good as a hit). I won’t get into all the details, but suffice to say on-base percentage is more causally related to scoring runs than batting  average, and scoring runs is what wins games.

So why is batting average still so prevalent and what does this have to do with retail?

Basically, an English statistician named Henry Chadwick developed batting average as a statistic in the late 1800s and didn’t include walks because he thought they were caused by the pitcher and therefore the batter didn’t deserve credit for not swinging at bad pitches. Nevermind that teams with batters who got on base scored more runs and won more games. But batting average has been used so long that we just keep on using it, even when it’s been proven to not be very valuable.

OK, baseball boy, what about the retail?

As relatively young as the e-commerce space is, I believe we are already falling prey to  conventional wisdom in some of our metrics and causing ourselves unnecessary churn.  My favorite example is site conversion rate. Conversion is a metric that has been used in physical retail for a very long time, and it makes good sense in stores where the overwhelming purpose is to sell products to customers on their  current visit.

I’ll argue, though, that our sites have always been about more than the buy button, and they are becoming more and more all-purpose every day. They are marketing and merchandising vehicles, brand builders, customer research tools (customers researching products and us researching customers), and sales drivers, both in-store and online. Given the multitude of purposes of our sites, holding high a metric that covers only one purpose not only wrongly values our sites, but it also causes us to churn unnecessarily when implementing features or marketing programs that encourage higher traffic for valuable purposes to our overall businesses that don’t necessarily result in an online purchase on a particular day.

We still need to track the sales generating capabilities of our sites, but we want to find a causal metric that actually focuses on our ability or inability to convert the portion of our sites’ traffic that came to buy. We used our site for many purposes at Borders, so we found that changes in overall site conversion rate didn’t have much to do at all with changes in sales.

If we wanted to focus on a metric that tracked our selling success, we needed to focus on the type of traffic that likely came with an intent to buy (or at least eliminate the type of traffic that came for other reasons), and we knew through our ForeSee Results surveys that our customers who came with an intent to buy on that visit was only a percentage of our total visitors, while the rest came for other reasons like researching products, finding stores, checking store inventory, viewing video content, etc.

So, how could we isolate our sales conversion metrics to only the traffic that came with an intent to buy?
Our web analyst Steve Weinberg came up with something we called “true conversion” that measured adds to cart  divided by product page views multiplied by orders divided by checkout process starts. This true conversion metric was far more correlative to orders than anything else, so it was the place to initially focus as we tried to determine if we could turn the correlation into causation. We still needed to do more work matching the survey data to path analysis to further refine our metrics, but it was a heckuva lot better than overall site conversion, which was basically worthless to us.

Every site is different, so I don’t know that all sites could take the exact same formula described above and make it work. It will take some work from your web analyst to dig into the data to determine customer intent and the pages that drive your customers ability to consummate that intent. For more ideas, I highly recommend taking a look at Bryan Eisenberg‘s excellent recent topic called How to Optimize Your Conversion Rates where he explores some of these topics in more detail.


Whether or not you buy into everything written in Moneyball or all of Billy Beane’s methods, I believe the main lesson to be culled from the book is that it’s critically important that we constantly re-evaluate our thinking (particularly when conventional wisdom in assumed to be true) in order to get at deeper truths and clearer paths to success.

How is overall site conversion rate working for you? Do you have any better metrics? Where have you run into trouble with conventional wisdom?

Innovation by popular opinion – evolutionary or revolutionary?

What do Best Buy, Starbucks and Barack Obama all have in common? They’ve all launched sites designed to solicit customer ideas for their business and policies. (At least Obama used to have this capability on the transition site I can’t seem to find the ability on has been around for over a year, and Best Buy’s Idea X just launched. Given my recent post on how defending the status quo kills companies, I wondered if this sort of solution would help companies find that next great revolutionary idea to transform the company.

I really like this idea as a way to generate some fresh new ideas that come from the people being served by a business. And there are some very good ideas on the new Best Buy site; ideas that range from improvements to their Reward Zone program to ways for customers to register for help in busier stores. I love that customers have this ability to submit their ideas, and I love that Best Buy and Starbucks are using open brand techniques and letting all ideas be shown publicly, even when they aren’t always complimentary.

But, man, what a task to review all of those ideas! As a way of helping to sift through the ideas, each of these sites has a voting mechanism ostensibly designed to move the best ideas to the top.Or at least the most popular ideas.

So, will this concept help companies find that next great revolutionary idea for their businesses? 

I’m not so sure. My experience is that popular voting is a pretty good way to find evolutionary ideas, but it doesn’t work as well for revolutionary ideas. This is because most people tend to gravitate towards incremental improvements to concepts already familiar to them and have trouble visualizing radically new concepts.

To be clear, I’m not in any way knocking these systems or the idea of the popular vote. I love that Best Buy and Starbucks are reaching out to their customers and trying to find ways to improve their businesses by better meeting their customers’ needs and expectations, and my guess is they’re not looking for revolutionary ideas via this mechanism. (Although, given both company’s solid histories of revolutionary innovations, if they come across a revolutionary idea I’m sure they’ll act upon it quickly.)

So, how then do companies find revolutionary ideas?

It seems we often rely on executive brainstorming sessions. Those sessions almost always use the same technique (i.e. generate lots of ideas and vote on your favorites) and I find those techniques are exactly the reason they usually fail to produce transformational ideas. Usually, there aren’t a lot of guidelines issued prior to the session — by design — and the problems to be solved are usually not well-defined and agreed upon. As a result, the most out-of-the-box ideas tend to lack votes and end up in the trash heap.

I recently read a Harvard Business Review article that promoted a different approach. Instead of coming up with random ideas on the fly, the approach is to spend time garnering support with individuals from a variety of functional areas prior to submission of the idea. Feedback is incorporated and the idea is able to develop more fully. Only once it’s gone through some development is an idea submitted for review by executive decision makers.

How different would these strategy brainstorming sessions be if the ideas had some development before they were suggested? Of course, radical ideas that cause transformational change will still meet resistance from all who benefit from the status quo, particularly when those revolutionary ideas are, almost by their nature, not likely to produce immediate financial results. This is the primary issue address by Clayton M. Christensen in his excellent book, The Innovator’s Dilemma. Christensen advocates creating entirely separate groups or even funding new start-up companies as a way to incubate these new strategies while they develop. This incubation approach is much the way many e-commerce organizations got their start, and I think it worked in many ways. Of course, now the new dilemma is how to bring it all back together with the parent companies. (Sigh) Nothing’s easy.

I hope more companies will follow Best Buy’s and Starbucks’ leads in opening their brands to customer ideas and public feedback. I look forward to the many great ideas I’m sure will come from it. I hope also that we can all find some techniques to deliver some really great revolutionary ideas to keep our companies vibrant and relevant for a long time to come.

What do you think? Do you have any great brainstorming techniques to share? Have you seen new ideas come to life in your company?

Defending the status quo kills companies

“Defending the status quo is what kills companies.” That line comes from the excellent book More Than a Motorcycle: The Leadership Journey at Harley-Davidson written by former Harley CEO  Rich Teerlink and his organizational consultant partner Lee Ozley. The book chronicles Teerlink’s and Ozley’s process to change the culture at Harley-Davidson to ensure the company was ready for the challenges to come. What I found most remarkable, though, was that they didn’t initiate this massive change when the company was troubled — they initiated massive change when the company had just completed a successful financial turnaround and the press was actively singing their praises.

They changed when conventional wisdom would have said to keep doing what they were doing.

Bankruptcy courts are littered with companies who kept doing what they were doing and failed to adapt to changing marketplaces and changing customer needs and expectations.

I spent 20 years in the music industry with Tower Records, so I’ve see one of the best examples in recent years of an entire industry that desperately clung to the past rather than embrace the future. The music industry didn’t suffer because of Napster and illegal downloads; it suffered because it turned its back on its customers in favor of short term profits.

The music industry failed to recognize the opportunity that came with the advent of the Internet and digital music formats. Rather than see their industry from their customers’ perspective, the industry fell pray to the elitism I’ve discussed previously. So a computer company took their business from them. Apple‘s iPod and iTunes took the music retailers’ business and substantially wrestled control away from the music labels.

The retailers could have created digital music stores if they weren’t so worried about protecting their current businesses. And there were other opportunities available. Seth Godin spoke to the industry last year and gave some excellent examples of opportunities to change the business model.

Now other traditional industries like newspapers, video stores and bookstores, among others, are also losing substantial market share to new, technology based upstarts. Others, like travel agencies, are mostly gone.

But some companies are embracing change even during the height of success.
A recent Forbes interview with Xerox’s retiring CEO Anne Mulcahy highlighted her strategy to focus Xerox on “paperless printing” even though the entire organization was basically built on paper-producing technology. Rather than focus on paper, Mulcahy instead said the company’s value was always about the creation, management and dispensing of information, “Democratization of information, however it happened.”

Threats to existing business models aren’t only coming in the form of digitization.
Look at the shoe business. In ten years, went from a germ of an idea to a $1 billion company. Their model? “In March of 2003, we made a decision to be about customer service,” say their CEO, Tony Hsieh, in a recent Fast  Company profile. “We view any expense that enhances the customer experience as a marketing cost because it generates more repeat customers through word of mouth.”

Customer experience as a marketing cost. It’s a whole new way of looking at the shoe business (or retail in general), and it’s a hit worth a cool billion in a short amount of time.

I can’t believe that billion dollars was incremental business to the overall market. That share came out of somebody’s  hide. And that means an existing shoe business could have done it first if the thought process and the courage to act  was there. If the Zappos model works, it can be applied to anything, and it appears that’s exactly what Zappos intends  to do.

And the radical ideas keep coming. Chris Anderson has a controversial new book, Free, that describes a future he believes will be centered largely around business models that give away 95% of their offerings and make money on the remaining 5%. Are Anderson’s ideas open for debate? Sure, but they and other seemingly nutty ideas should be regularly and honestlydiscussed. One of them may well be the next billion dollar idea.

It doesn’t take wholesale change in the marketplace to significantly disrupt a business model.

A drop in business of 10-15% can have massive impact, as many have clearly seen in the current economic downturn. But the economic downturn has not sunk all boats. reported a sales increase of 18% and a net income increase of 24% for their first quarter this year.

As e-commerce continues to be the growth vehicle in retail, and as Amazon continues to dominate e-commerce, I wonder how brick and mortar retail models will adapt. I believe there are many opportunities today to leverage both the growth and value of e-commerce and existing physical real estate.

Certainly, tying the web experience and the store experience together via cross-channel capabilities is a must. In the industry, we talk a lot today about capabilities like order online and pick up in store, and I think those are good.

But how can we take it further?
For example, I know from my experience with in-store kiosks at Borders that a lot more people than I expected still aren’t comfortable shopping online. They want someone to help them use the kiosks, and then they want to pay with cash at the register. Why not use our store POS systems to take cash payments for online orders? What if we took it a step further and took cash payments for other sites’ orders. What if the physical store essentially became an affiliate for a pure play e-commerce site and took the cash along with a commission? What type of opportunities might that open for both the pure play and the brick and mortar store? What other reasons should customers continue to shop physical stores well into the future as technology and delivery systems continue to improve?

What challenges does your business face in the coming years, or what businesses in general do you see most at risk? How could your business model change — maybe radically — to address those challenges? Or, do you think this is all hogwash? Let’s discuss.

Is elitism the source of poor usability?

Most sites are still achieving single digit conversion rates even though customer intent-to-purchase rates are 20% or higher in most cases. Customers are continuing to run into obstacles to the purchase process that need to be eliminated. The good news is that during this time of limited capital investments, retailers can use low cost means to find and eliminate as many obstacles to purchase as possible.

The first step is to get into the right mindset and remove what I feel is the biggest disconnect with the customers that many retailers have: we’re way more comfortable and experienced with our own sites than our customers are. We use our sites every day, and we know exactly how they’re supposed to work. However, our customers are generally nowhere near as familiar with our sites as we are.

Two weeks ago, I was lucky to be able to attend GSI‘s Connect conference for its clients. I was even luckier to attend a fantastic session by GSI’s Senior Director of Usability, Michael Summers. Michael got the audience’s attention pretty quickly by calling us all elitists…and he had a good point. He asked us how many of us fit the demographic for today’s main Internet users and quickly made the point that we were higher educated, higher paid and more Internet savvy — by a long shot — than the average site user in the marketplace. If that wasn’t enough, he showed some video of average Americans shopping online who had trouble with some of what we in the industry would consider among the most basic aspects of websites.

To solve this disconnect we need to see our sites through our customers’ eyes. There are a number of ways to do this that I’ve found to be effective.

  1. Use statistically significant customer satisfaction surveys to get trendable data that will  point to the biggest problem areas of the site.
    The two key phrases here are “statistically significant” and “trendable.” Per my last post, continuous measurement is important to avoid random outliers and uncover the underlying truth. When done correctly, customer satisfaction surveys can be extremely reliable, accurate, and predictive and can tell you not only which areas of a site customers complain about most, but also which areas of the site will actually have the biggest impact on purchase intent and loyalty. This is critical information to provide some some direction on where to focus your usability efforts.
  2. Ask open-ended questions to add color to the quantitative information.
    Quantitative analysis is extremely useful, but numbers alone aren’t nearly enough. Numbers will certainly tell you the problem areas of the site, but to really get your arms around what the numbers are saying requires adding some color to them with some qualitative information. Asking more open-ended questions like “If you could make one improvement to our site, what would it be?” are good starters to bring some of the numbers to life. If the numbers tell you that customers in general are having problems with navigation and you see that multiple customers say in open-ended comments they just want to see all the blue dresses in stock, you might start to consider adding color choice to your navigation. Or maybe you already have an option to navigate by color, but the customers aren’t seeing it and you’ll need to find a way to make it more apparent.
  3. Watch your customers use your site.
    The absolute best way to add color to the data is to actually watch customers use the site. In the past, I’ve seen great discoveries come from taking a laptop into a store and asking real customers to shop on the site while I or someone on my team watched silently. In these situations, it’s very important not to be too prescriptive in the tasks the customer is asked to do. Ask them to “find and buy a new pair of dress shoes” rather than “go to the men’s tab, then select dress shoes and find a pair of black, size 9 shoes.” It never fails to amaze me in this situation how many different avenues customers will take to accomplish the task, and they’ll frequently run into trouble. These trouble spots are the areas to find and eliminate. Some of the smallest fixes can often significantly improve conversion and customer satisfaction.If the logistics of getting into a store are too difficult or you don’t have physical stores, there are technology alternatives, like Tealeaf’s CX and ForeSee’s new CS Session Replay, that provide the ability to replay customers’ sessions on your screen.
  4. Have an expert conduct a usability audit.
    Even after discovering where customers are having trouble, it’s sometimes still very difficult to determine exactly what you should be doing differently to make the experience easier and more intuitive for your customers. In those cases, expert advice via a third party usability audit is an excellent solution. I’ve used trained usability experts in the past to identify specific improvements that led to tremendous business results. Third party usability auditors bring to the table both fresh and trained eyes that have likely seen problems similar to those on your site before and have come up with solutions for those problems or seen how other sites have solved those problems.

Regardless of the mechanisms you choose to use, the key to better usability, better customer satisfaction and the resulting better conversion and sales, is finding ways to see your site through your customers’ eyes.

Are you a usability elitist? Do you watch customers use your site? What have you learned in the process?

Retail: Shaken Not Stirred by Kevin Ertell

Home | About