The 3 Levels of Metrics: From Driving Cars to Solving Crimes

Business-MetricsYou can’t manage what you don’t measure. That’s a long-time business mantra espoused frequently by my good friend Larry Freed. And it’s certainly true. But in an e-commerce where we can effectively measure our customers’ every footstep, we can easily become overwhelmed with all that data. Because while we can’t manage what we don’t measure, we also can’t manage everything we can measure.

I’ve found it’s best to break our metrics down to three levels in order to make the most of them.

1. KPIs
The first and highest level of metrics contains the Key Performance Indicators or KPIs. I believe strongly there should be relatively few KPIs — maybe five or six at most — and the KPIs should align tightly with the company’s overall business objectives. If an objective is to develop more orders from site visitors, then conversion rate would be the KPI. If another objective is about maximizing the customer experience, then customer satisfaction is the right metric.

In addition to conversion rate and customer satisfaction, a set of KPIs might include metrics like average order value (AOV), market share, number of active customers,  task completion rate or others that appropriately measure the company’s key objectives.

I’ve found the best KPI sets are balanced so that the best way to drive the business forward is to find ways to improve all of the KPIs, which is why businesses often have balanced scorecards. The reality is, we could find ways to drive any one metric at the expense of the others, so finding the right balance is critical. Part of that balance is ensuring that the most important elements of the business are considered, so it’s important to have some measure of employee satisfaction (because employee satisfaction leads to customer satisfaction) and some measure of profitability.  Some people look at a metric like Gross Margin as the profitability measure, but I prefer something deeper down the financial statement like Contribution Margin or EBITDA because they take other cost factors like ad spend, operational efficiencies, etc. into account and can be affected by most people in the organization.

It’s OK for KPIs to be managed at different frequencies. We often talk about metrics dashboards, and a car’s dashboard is the right metaphor. Car manufacturers have limited space to work with, so they include only the gauges the most help the driver operate the car. The speedometer is managed frequently while operating the car. The fuel gauge is critically important, but it’s monitored only occasionally (and more frequently when it’s low). Engine temperature is a hugely important measure for the health of the car, but we don’t need to do much with it until there’s a problem. Business KPIs can be monitored in a similarly varied frequency, so it’s important that we don’t choose them based on their likelihood to change over some specific time period. It’s more important to choose the metrics that most represent the health of the business.

2. Supporting Metrics
I call the next level of metrics Supporting Metrics. Supporting Metrics are tightly aligned with KPIs, but they are more focused on individual functions or even individual people within the organization. A KPI like conversion rate can be broken down by various marketing channels pretty easily, for example. We could have email conversion rate, paid search conversion rate, direct traffic conversion rate, etc. I also like to look at True Conversion Rate, which measures conversion against intent to buy.

Supporting metrics should be an individual person’s or functional area’s scorecard to measure how their work is driving the business forward. Ensuring supporting metrics are tightly aligned with the overall company objectives helps to ensure work efforts throughout the organization are tightly aligned with the overall objectives.

As with KPIs, we want to ensure any person or functional area isn’t burdened with so many supporting metrics that they become unmanageable. And this is an area where we frequently fall down because all those metrics and data points are just so darn alluring.

The key is to recognize the all-important third level of metrics. I call them Forensic Metrics.

3. Forensic Metrics
Forensic Metrics are just what they sound like. They’re those deep-dive metrics we use when we’re trying to solve a problem we’re facing in KPIs or Supporting Metrics. But there are tons of them, and we can’t possibly manage them on a day-to-day basis. In the same way we don’t dust our homes for prints every day when we come home from work, we can’t try to pay attention to forensic metrics all the time. If we come home and find our TV missing, then dusting for prints makes a lot of sense. If we find out conversion rate has dropped suddenly, it’s time to dig into all sorts of forensic metrics like path analysis, entry pages, page views, time on site, exit links, and the list goes on and on.

Site analytics packages, data warehouse and log files are chock full of valuable forensic metrics. But those forensic metrics should not find their way onto daily or weekly managed scorecards. They can only serve to distract us from our primary objectives.

—————————————————–

Breaking down our metrics into these three levels takes some serious discipline. When we decide we’re only going to focus on a relatively small number of metrics, we’re doing ourselves and our businesses a big favor. But it’s really important we’re narrowing that focus on the metrics and objectives that are most driving the business forward. But, heck, we should be doing that anyway.

What do you think? How do you break down your metrics?

 

8 Comments

  • By Kevin Ertell, October 10, 2011 @ 11:09 am

    Dear readers,

    I apologize for the long break between posts. I started a new gig as CMO at OnlineShoes.com back in February, and I’ve been so busy that I just haven’t had time to write a new post. I finally found some time this weekend, and I hope to find more time in the new future. I can’t promise the same regularity of posting as I had previously, but I really enjoy the conversation so I hope to get more posts up soon. I’d love to know your thoughts on this latest topic.

    Thanks,
    Kevin

  • By Jeff, October 10, 2011 @ 11:12 am

    He’s back!

  • By John Hunter, October 11, 2011 @ 8:16 am

    It’s great to see you writing again, Kevin. I really like this post. I think we have too many KPIs in my business. There are 11 on our company weekly dashboard and then each department has its own. Do you think it’s ever reasonable to have so many?

  • By Sarah, October 11, 2011 @ 8:30 am

    Glad you’re back, Kev-dawg. Would be interested in hearing some examples of these from OnlineShoes if there are any you can share. I hear you guys are doing some cool things there.

  • By Eric Feinberg, October 11, 2011 @ 12:25 pm

    Kevin, it is indeed nice to have you back blogging. I missed ya! Great points above. I think organizing metrics is a problem for many, many people in the business of managing metrics. Without a clean house, how will you be able to find anything? I think these three organizational ‘drawers’ are a decent start to begin the house cleaning.

  • By Steve Groenier, October 11, 2011 @ 6:03 pm

    Kevin, glad to see a new post. I like your approach to breaking metrics into these three buckets. The sheer number of metrics we can track in an eCommerce business is overwhelming and without some framework for helping us understand what, when and how often to look at these we can easily get distracted and lose sight of what’s really important to driving our business.

    I think you are spot on when it comes to using deeper financial metrics like contribution margin. This is especially important to understand the true impact storewide discounts and coupons have on profitability.

    Understanding the contribution margin by product line is also important for making marketing and pricing decisions. If you are just looking at top line revenue, it’s easily fall into the trap of spending too much to promote your best selling products while neglecting the product lines that are actually adding more profitability to your bottom line.

    One forensic metric I would suggest retailer marketers make sure they can track is the Out of Stock Rate by product line or brand. When trying to understand why your ROI has declined for certain paid search campaigns or the cause of falling site conversion rates, the answer might be the result of a high out of stock rate.

  • By Kevin Ertell, October 12, 2011 @ 8:53 am

    Thanks for the nice comments everyone.

    John: My guess is some of the metrics on your dashboard are supporting metrics disguised as KPIs. I don’t think it’s a bad thing necessarily to have some supporting metrics on a dashboard, but I do think it’s important to distinguish them and now which are driving the overall business objectives and which are driving the KPIs.

    Sarah: I feel a little unsure now about giving away too munch internal information now that I’m back in a competitive retail environment, but some of the KPIs examples I mentioned are some of the ones we use, like conversion rate, customer satisfaction and EBITDA

    Eric: I’m glad you like my “drawers.” Ooh, that sounds a little funny. 🙂

    Steve: I totally agree with you on your point about overly focusing on topline and the profitability problems that can come with that focus. I love your Out of Stock Rate example. Thanks a lot!

Other Links to this Post

  1. KPI: Indicadores para Medir lo que se Pierde de Ganar | Blog Tablero de Comando — October 13, 2011 @ 11:15 am

RSS feed for comments on this post. TrackBack URI

Leave a comment

Retail: Shaken Not Stirred by Kevin Ertell


Home | About