True conversion – the on-base percentage of web analytics?

I just finished re-reading one of my all-time favorite business books, Moneyball by Michael Lewis. While on the surface Moneyball is a baseball book about the General Manager of the Oakland A’s, Billy Beane, I found it to be more about how defying conventional wisdom (a topic I’ll no doubt return to over and over in this space) can be an excellent competitive advantage. In retail, we can be just as prone to conventional wisdom and business as usual as the world of baseball Lewis encountered, and site conversion rate is an excellent example of how we’re already traversing that path in the relatively young world of e-commerce.

In Moneyball, Michael Lewis tells the story of Beane defying the conventional wisdom of longtime baseball scouts and  baseball industry veterans. Rather than trust scouts who would literally  determine a baseball player’s prospects by  how he physically looked, Beane went to the data as a disciple of Bill JamesSabermetrics theories. By following the  James’ approach, Beane was able to put together consistently winning teams while working with one of the lowest payrolls in the Major Leagues.

Lewis describes how James took a new look at traditional baseball statistics and created new statistics that were  actually more causally related to winning games. Imagine that! For example, James found on-base percentage, which  includes walks when calculating how often a player gets on base, to be a much more reliable statistic than batting  average, which ignores walks (even though we’re always taught as Little Leaguers that a walk is as good as a hit). I won’t get into all the details, but suffice to say on-base percentage is more causally related to scoring runs than batting  average, and scoring runs is what wins games.

So why is batting average still so prevalent and what does this have to do with retail?

Basically, an English statistician named Henry Chadwick developed batting average as a statistic in the late 1800s and didn’t include walks because he thought they were caused by the pitcher and therefore the batter didn’t deserve credit for not swinging at bad pitches. Nevermind that teams with batters who got on base scored more runs and won more games. But batting average has been used so long that we just keep on using it, even when it’s been proven to not be very valuable.

OK, baseball boy, what about the retail?

As relatively young as the e-commerce space is, I believe we are already falling prey to  conventional wisdom in some of our metrics and causing ourselves unnecessary churn.  My favorite example is site conversion rate. Conversion is a metric that has been used in physical retail for a very long time, and it makes good sense in stores where the overwhelming purpose is to sell products to customers on their  current visit.

I’ll argue, though, that our sites have always been about more than the buy button, and they are becoming more and more all-purpose every day. They are marketing and merchandising vehicles, brand builders, customer research tools (customers researching products and us researching customers), and sales drivers, both in-store and online. Given the multitude of purposes of our sites, holding high a metric that covers only one purpose not only wrongly values our sites, but it also causes us to churn unnecessarily when implementing features or marketing programs that encourage higher traffic for valuable purposes to our overall businesses that don’t necessarily result in an online purchase on a particular day.

We still need to track the sales generating capabilities of our sites, but we want to find a causal metric that actually focuses on our ability or inability to convert the portion of our sites’ traffic that came to buy. We used our site for many purposes at Borders, so we found that changes in overall site conversion rate didn’t have much to do at all with changes in sales.

If we wanted to focus on a metric that tracked our selling success, we needed to focus on the type of traffic that likely came with an intent to buy (or at least eliminate the type of traffic that came for other reasons), and we knew through our ForeSee Results surveys that our customers who came with an intent to buy on that visit was only a percentage of our total visitors, while the rest came for other reasons like researching products, finding stores, checking store inventory, viewing video content, etc.

So, how could we isolate our sales conversion metrics to only the traffic that came with an intent to buy?
Our web analyst Steve Weinberg came up with something we called “true conversion” that measured adds to cart  divided by product page views multiplied by orders divided by checkout process starts. This true conversion metric was far more correlative to orders than anything else, so it was the place to initially focus as we tried to determine if we could turn the correlation into causation. We still needed to do more work matching the survey data to path analysis to further refine our metrics, but it was a heckuva lot better than overall site conversion, which was basically worthless to us.

Every site is different, so I don’t know that all sites could take the exact same formula described above and make it work. It will take some work from your web analyst to dig into the data to determine customer intent and the pages that drive your customers ability to consummate that intent. For more ideas, I highly recommend taking a look at Bryan Eisenberg‘s excellent recent topic called How to Optimize Your Conversion Rates where he explores some of these topics in more detail.

—————————————–

Whether or not you buy into everything written in Moneyball or all of Billy Beane’s methods, I believe the main lesson to be culled from the book is that it’s critically important that we constantly re-evaluate our thinking (particularly when conventional wisdom in assumed to be true) in order to get at deeper truths and clearer paths to success.

How is overall site conversion rate working for you? Do you have any better metrics? Where have you run into trouble with conventional wisdom?


10 Comments

  • By Matt Cushing, July 28, 2009 @ 3:19 pm

    Interesting article, Kevin. I think another area of Billy Beane’s nose-thumbing at conventional wisdom was that he actually took a statistical approach instead of relying on the subjective instincts of scouts, which I know you touched on in your post about elitism.
    I believe the right metrics depend heavily on the company and its relative positioning in the marketplace. Cross-channel companies will likely be interested in different metrics than pure-play retailers, and companies truly focused on moving product will approach metrics differently than companies trying to build a brand.
    For any accurate measurement, you first need to know who you are and what is important to you, and then determine your appropriate metrics. I think True Conversion in a cross-channel environment really makes sense, and I agree there are likely variations of this to fit other situations. But any conversion statistic is only one piece of the puzzle. A comprehensive, overall site report card is really needed to determine if a site is achieving the goals set for it.

  • By Kevin Ertell, July 28, 2009 @ 9:35 pm

    Thanks for your comment, Matt. I completely agree that metrics for cross-channel companies and pure plays would be different. In fact, I don’t think there is necessarily a one-size-fits-all metric here for anybody. Each site will have its own strategy, and it’s going to be important for each site to think through its metrics to find those KPIs which truly are Key Performance Indicators for that site.

  • By Andrew Orr, July 29, 2009 @ 7:42 am

    I really like the message that Moneyball delivers on gut instincts and conventional wisdom.
    In my experience, it is far too common that management will look at a compelling data set and then make a contradictory decision based on gut instincts.
    The lesson I took away from the book (and your article) is that it is possible to find true cause and effect metrics that predict success. The really hard part is getting yourself and others to beleive in the results when they contradict your instincts.
    Let’s face it, conventional wisdom isn’t really all that wise. If it were, there would not be so many failed or failing retailers out there.
    Gut instinct’s are really no different than placing a bet on black or red. If someone walked up to you in a casino and predicted the results on a roulette wheel 5 out of 7 times, would you bet on their 8th predicition or would you go with your gut?

  • By Kevin Ertell, August 2, 2009 @ 3:23 pm

    Thanks for your comments, Andy. You make a great point about that it is sometimes difficult to believe the data yourself when it contradicts your gut and everything you’ve ever been told. However, in my experience good data properly collected and analyzed consistently beats gut instincts when acted upon. And if your competition is following conventional wisdom in their actions and strategies, you can gain a leg up when the data tells a different story than conventional wisdom.

  • By Ted Vasquez, August 4, 2009 @ 1:55 pm

    Kevin, thanks for connecting one of my favorite books to our work. If I may continue the analogy, all batters MUST touch first base before they can achieve the ultimate goal of scoring a run. So, on-base percentage is really a way of measuring how often a batter completes his most basic job, reaching first base. After that, he must rely on his teammates to ultimately score a run.
    In this sense, Billy Bean broke down a complex and interdependent goal (scoring a run) into manageable components for measuring the performance of individual batters (reaching first base).
    I submit that you achieved something similar with your measure of “true conversion”. You broke down a complex and interdependent goal (completing a sale) into manageable components for measuring the performance of specific page types (e.g., product pages are supposed to get people to add products to the cart).
    I agree sites are more than a buy button. They have more than one goal. The challenge is defining those other goals and then breaking them down into measurable components.
    ***NOTE TO ALL THE BASEBALL FANS OUT THERE: A more accurate definition of a batters job is to reach first base and/or drive in runs. This includes driving in runs by sacrificing or even making an out. By that definition, on-base percentage is not a complete performance measure but still better than batting average.

  • By Derek Monteverdi, September 11, 2009 @ 3:41 pm

    I’m not sure how to read the calculation but if you measured carts or cart adds per product view then measured checkout starts vs. completions then you’re right on target. Overall conversion is an important metric but New Cart Conversion, Add to Cart Conversion, Checkout Conversion are the additional tools that you need to understand progress through the funnel. The goal should be to optimize at every stage.

  • By Kevin Ertell, September 14, 2009 @ 4:41 pm

    Thanks for your comment, Derek. I realized after reading your notes that I didn’t explain the calculation as clearly as I could have. I’ve made a slight adjustment to hopefully make it more clear. You did interpret it correctly, though, and I completely agree that it’s important to optimize at every stage.

  • By Betty, October 19, 2011 @ 1:48 am

    wow, I hadn’t gone that deep but it makes a lot of sense to try to have your own metrics that work, well, for our site too. Just not convinced there’s the right keyed data that will help us identify th intention to buy. THinking cap time though. THanks for the inspiration

Other Links to this Post

  1. Wanna be better with metrics? Watch more poker and less baseball. | Retail: Shaken Not Stirred — January 6, 2010 @ 8:02 pm

  2. What Is Your True Conversion Rate? | ClickZ UK — November 24, 2015 @ 3:21 pm

RSS feed for comments on this post. TrackBack URI

Leave a comment

Retail: Shaken Not Stirred by Kevin Ertell


Home | About