The Missing Links in the Customer Engagement Cycle

The Customer Engagement Cycle plays a central role in many marketing strategies, but it’s not always defined in the same way. Probably the most commonly described stages are Awareness, Consideration, Inquiry, Purchase and Retention. In retail, we often think of the cycle as Awareness, Acquisition, Conversion, Retention. In either case, I think there are a couple of key stages that do not receive enough consideration given their critical ability to drive the cycle.

The missing links are Satisfaction and Referral.

Before discussing these missing links, let’s take a quick second to define the other stages:

Awareness: This is basic branding and positioning of the business. We certainly can’t progress people through the cycle before they’ve even heard of us.

Acquisition: I’ve always thought of this as getting someone into our doors or onto our site. It’s a major step, but it’s not yet profitable.

Conversion: This one is simply defined as making a sales. Woo hoo! It may or may not be a profitable sales on its own, but it’s still a significant stage in the cycle.

Retention: We get them to shop with us again. Excellent! Repeat sales tend to be more profitable and almost certainly have lower marketing costs than first purchases.

Now, let’s get to those Missing Links

In my experience, the key to a strong and active customer engagement cycle is a very satisfying customer experience. And while the Wikipedia article on Customer Engagement doesn’t mention Satisfaction as often as I would like, it does include this key statement: “Satisfaction is simply the foundation, and the minimum requirement, for a continuing relationship with customers.”

In fact, I think the quality of the customer experience is so important that I would actually inject it multiple times into the cycle: Awareness, Acquisition, Satisfaction, Conversion, Satisfaction, Retention, Satisfaction, Referral.

Of course, it’s possible to get through at least some of the stages of the cycle without an excellent customer experience. People will soldier through a bad experience if they want the product bad enough or if there’s an incredible price. But it’s going to be a lot harder to retain that type of customer and if you get a referral, it might not be the type of referral you want.

I wonder if Satisfaction and Referral are often left out of cycle strategies because they are the stages most out of marketers’ control.

A satisfying customer experience is not completely in the marketer’s control. For sure, marketing plays a role. A customer’s satisfaction can be defined as the degree to which her actual experience measures up to her expectations. Our marketing messages are all about expectations, so it’s important that we are compelling without over-hyping the experience. And certainly marketers can influence policy decisions, website designs, etc. to help drive better customer experiences.

In the end, though, the actual in-store or online experience will determine the strength of the customer engagement.

Everyone plays a part in the satisfaction stages. Merchants must ensure advertised product is in stock and well positioned. Store operators must ensure the stores are clean, the product is available on the sales floor and the staff are friendly, enthusiastic and helpful. The e-commerce team must ensure advertised products can be easily found, the site is performing well, product information in complete and useful,  and the products are shipped on time and in good condition.

We also have to ensure our incentives and metrics are supporting a quality customer experience, because the wrong metrics can incent the wrong behavior. For example, if we measure an online search engine marketing campaign by the number of visitors generated or even the total sales generated, we can absolutely end up going down the wrong path. We can buy tons of search terms that by their sheer volume will generate lots of traffic and some degree of increased sales. But if those search terms link to the home page or some other page that is largely irrelevant to the search term, the experience will be likely disappointing for the customer who clicked through.

In fact, I wrote a white paper a few months ago, Online Customer Acquisition: Quality Trumps Quantity, that delved into customer experience by acquisition source for the Top 100 Internet Retailers. We found that those who came via external search engines were among the least satisfied customers of those sites with the least likelihood to purchase and recommend. Not good. These low ratings could largely be attributed to the irrelevance of the landing pages from those search terms.

Satisfaction breeds Referral

Referrals or Recommendations are truly wonderful. As I wrote previously, the World’s Greatest Marketers are our best and most vocal customers. They are more credible than we’ll ever be, and the cost efficiencies of acquisition through referral are significantly better than our traditional methods of awareness and acquisition marketing. In my previously mentioned post, I discussed some ways to help customers along on the referral path. But, of course, customers can be pretty resourceful on their own.

We’ve all seen blog posts, Facebook posts or tweets about bad customer experiences. But plenty of positive public commentary can also be found.  Target’s and Gap’s Facebook walls have lots of customers expressing their love for those brands. Even more powerful are blog posts some customers write about their experiences.  I came across a post yesterday from entitled Tales of Perfection that related two excellent experiences the blogger had with Guitar Center and a burger joint called Arry’s. Both stories are highly compelling and speak to the excellent quality of the employees at each business. Nice!

————————————————–

Developing a business strategy, not just a marketing strategy, around the customer engagement cycle can be extremely powerful. It requires the entire company to get on board to understand the value of maximizing the customer experience at every touch point with the customer, and it requires a set of incentives and metrics that fully support strengthening the cycle along the way.

What do you think? How do you think about the customer engagement cycle? How important do feel the customer experience is in strengthening the cycle? Or do you think this is all hogwash?

Why most sales forecasts suck…and how Monte Carlo simulations can make them better

Sales forecasts don’t suck because they’re wrong.  They suck because they try to be too right. They create an impossible illusion of precision that ultimately does a disservice to managers who need accurate forecasts to assist with our planning. Even meteorologists — who are scientists with tons of historical data, incredibly high powered computers and highly sophisticated statistical models — can’t forecast with the precision we retailers attempt to forecast. And we don’t have nearly the data, the tools or the models meteorologists have.

Luckily, there’s a better way. Monte Carlo simulations run in Excel can transform our limited data sets into statistically valid probability models that give us a much more accurate view into the future. And I’ve created a model you can download and use for yourself.

There are literally millions of variables involved in our weekly sales, and we clearly can’t manage them all. We focus on the few significant variables we can affect as if they are 100% responsible for sales, but they’re not and they are also not 100% reliable.

Monte Carlo simulations can help us emulate real world combinations of variables, and they can give us reliable probabilities of the results of combinations.

But first, I think it’s helpful to provide some background on our current processes…

We love our numbers, but we often forget some of the intricacies about numbers and statistics that we learned along the way. Most of us grew up not believing a poll of 3,000 people could predict a presidential election. After all, the pollsters didn’t call us. How could the opinions of 3,000 people predict the opinions of 300 million people?

But then we took our first statistics classes. We learned all the intricacies of statistics. We learned about the importance of properly generated and significantly sized random samples. We learned about standard deviations and margins of errors and confidence intervals. And we believed.

As time passed, we moved on from our statistics classes and got into business. Eventually, we started to forget a lot about properly selected samples, standard deviations and such and we just remembered that you can believe the numbers.

But we can’t just believe any old number.

All those intricacies matter. Sample size matters a lot, for example. Basing forecasts, as we often do, on limited sets of data can lead to inaccurate forecasts.

Here’s a simplified explanation of how most retailers that I know develop sales forecasts:

1. Start with base sales from last year for the the same time period you’re forecasting (separating out promotion driven sales)
2. Apply the current sales trend (which is maybe determined by an average of the previous 10 week comps). This method may vary from retailer to retailer, but this is the general principle.
3. Look at previous iterations of the promotions being planned for this time period. Determine the incremental revenue produced by those promotions (potentially through comparisons to control groups). Average of the incremental results of previous iterations of the promotion, and add that average to the amount determined in steps 1 and 2.
4. Voilà! This is the sales forecast.

Of course, this number is impossibly precise and the analysts who generate it usually know that. However, those on the receiving end tend to assume it is absolutely accurate and the probability of hitting the forecast is close to 100% — a phenomenon I discussed previously when comparing sales forecasts to baby due dates.

As most of us know from experience, actually hitting the specific forecast almost never happens.

We need accuracy in our forecasts so that we can make good decisions, but unjustified precision is not accuracy. It would be far more accurate to forecast a range of sales with accompanying probabilities. And that’s where the Monte Carlo simulation comes in.

Monte Carlo simulations

Several excellent books I read in the past year (The Drunkard’s Walk, Fooled by Randomness, Flaw of Averages, and Why Can’t You Just Give Me a Number?) all promoted the wonders of Monte Carlo simulations (and Sam Savage of Flaw of Averages even has a cool Excel add-in). As I read about them, I couldn’t help but think they could solve some of the problems we retailers face with sales forecasts (and ROI calculations, too, but that’s a future post). So I finally decided to try to build one myself. I found an excellent free tutorial online and got started. The results are a file you can download and try for yourself.

A Monte Carlo simulation might be most easily explained as a “what if” model and sensitivity analysis on steroids. Basically, the model allows us to feed in a limited set of variables about which we have some general probability estimates and then, based on those inputs, generate a statistically valid set of data we can use to run probability calculations for a variety of possible scenarios.

It turns out to be a lot easier than it sounds, and this is all illustrated in the example file.

The results are really what matters. Rather than producing a single number, we get probabilities for different potential sales that we can use to more accurately plan our promotions and our operations. For example, we might see that our base business has about a 75% chance of being negative, so we might want to amp up our promotions for the week in order have a better chance of meeting our growth targets.  Similarly, rather than reflexively “anniversaring” promotions, we can easily model the incremental probabilities of different promotions to maximize both sales and profits over time.

The model allows for easily comparing and contrasting the probabilities of multiple possible options. We can use what are called probability weighted “expected values” to find our best options. Basically, rather than straight averages that can be misleading, expected values are averages that are weighted based on the probability of each potential result.

Of course, probabilities and ranges aren’t as comfortable to us as specific numbers, and using them really requires a shift in mindset. But accepting that the future is uncertain and planning based on the probabilities of potential results puts us in the best possible position to maximize those results. Understanding the range of possible results allows for better and smarter planning. Sometimes, the results will go against the probabilities, but consistently making decisions based on probabilities will ultimately earn the best results over time.

One of management’s biggest roles is to guide our businesses through uncertain futures. As managers and executives, we make the decisions that determine the directions of our companies. Let’s ensure we’re making our decisions based on the best and most accurate information — even if it’s not the simplest information.

What do you think? What issues have you seen with sales forecasts? Have you tried my example? How did it work for you?

“Obscure and pregnant with conflicting meanings”

We’ve all heard the cliché “hindsight is 20/20” a thousand times. And it’s pretty much true. It’s a lot easier to figure out the path to a particular event when you know the final outcome. But if “what happened” is something bad, determining the reason after the fact doesn’t change the negative event.

How can we do a better job finding those problems in advance of our next new strategy implementation, site redesign, store remodel or other big effort?

It’s worth digging a little deeper to better understand why our hindsight is so perceptive. One of the most famous cases of 20/20 hindsight comes from the investigation into the attacks on Pearl Harbor (although, we could also argue the investigation into 9/11 and the more recent Fort Hood shootings have many similarities). In her book Pearl Harbor: Warning and Decision, noted military intelligence historian Roberta Wohlstetter wrote “it is much easier after the event to sort the relevant from the irrelevant signals. After the event, of course, a signal is always crystal clear; we can now see what disaster it was signaling since the disaster has occurred. But before the event it is obscure and pregnant with conflicting meanings.”

Of course, Pearl Harbor was an unexpected disaster that seemingly came out of nowhere. While we have those occasionally in business, more often than not our “disasters” come from strategies, redesigns or promotions that did not perform as expected. And those expectations can also lead to our blindness.

Whenever we’re implementing some new and exciting strategy, we tend to be very optimistic about the results. We’re convinced these new strategies are going to provide positive returns or we wouldn’t be implementing them. That optimism can lead to the same sort of crystal clear signal Wohlstetter referenced, but in the opposite direction; i.e. we tend to only see how everything we’re doing will lead to greatness and can easily overlook variables that have potential to lead to negative outcomes.

So, what do we do about it?

It seems some of the most common solutions today involve pulling together a committee to review what went wrong and putting together processes to prevent those specific problems in the future. These new processes don’t prevent all potential problems in the future, but with any luck they’ll prevent us from repeating the same mistakes.

But all of that happens after the fact.

There’s got to be a better way. My problem with the “committee and new process” approach is there’s a tendency to introduce lots of new and –all too often — needless bureaucracy. Inefficiencies ensue without greatly decreasing the probability of problem-free future efforts.

A technique I’ve found effective invokes much of the clarity of hindsight by drawing on the power of imagination.

During the ROI process for the strategy or project, we’ve already imagined the positive outcome. So before we wrap up planning, let’s also imagine a couple horrific scenarios. For example, imagine that four or five months after a site redesign, sales are down 50% and customer satisfaction has tanked. What happened? Now let’s assemble the same type of committee we would in that scenario and pour over the plan to find the causes of our imagined disaster.

Some might say this technique is really just standard contingency planning, but I find some pretty big differences. Contingency planning tends to look at the current plan to identify execution risks. It doesn’t often uncover key strategic or design problems.

The Scenario Imagination technique provides us with a different sort of lens that taps into our hindsight abilities to separate the signal from the noise.

We certainly won’t find every potential problem, but every problem we mitigate increases our probability of success and reduces our risk. And if we can reduce a lot of risk without strangling ourselves in bureaucracy, we’ll likely lower costs, increase efficiencies, and increase profits. I like the sound of that.

What do you think? Have you run into these types of issues? Do you think this technique would work for you? Do you have any techniques you would like to share?

Photo credit: me’nthedogs

“If it ain’t broke, you ain’t looking hard enough”

The poor economy has done nothing to lower customer expectations of online retailers, and recent mixed results data from ComScore and ForeSee Results indicate that retailers who continue to improve their customer experiences are pulling away from their competitors in both sales and customer satisfaction.

ComScore reports online retail up 4% for the holiday season. While an increase is always nice, this is a much lower growth rate than online retail has seen in the past. And last year’s comparison base was far from stellar. ForeSee Results shows a significant drop in customer satisfaction year over year. Since satisfaction is predictive of future financial results, a drop is concerning.

But still, I wondered how sales could be up at all if satisfaction was so far down.

A deeper look at the ComScore data shows the Top 25 retailers growing 13% while “Small and Mid Tail” retailers are declining 10%. Satisfaction scores are also split, but the differences we’re seeing seem to be more based on those retailers who are continually improving their sites versus those whose cost containment measures have slowed or stopped improvements. It appears that the retailers who closely measure the effectiveness of their sites from their customers’ perspectives and continuously improve their customers’ experiences are the retailers with increasing customer satisfaction scores. Those retailers who didn’t improve customer experience this year are suffering declining satisfaction scores. Many of those in the Top 25 are the retailers who have continued to enhance their customer experiences. Those enhancements are not only helping them to increase their sales, but because of the high visibility and usage of those tops sites, they’re also raising consumer expectations of all sites.

Customer satisfaction can be best defined as the degree to which a customer’s actual experience meets his or her expectations. Therefore, rising expectations can depress satisfaction scores if customer experience improvements don’t keep pace.

In the rapidly changing world of online retail, stopping or delaying improvements is like treading water in a swimming race. While you may temporarily save some energy, you will fall hopelessly behind and your only hope of catching up is spending a lot more energy than you likely saved treading water

Growing online retail businesses realize and fully embrace the need for continuous improvements, and they also realize that online retail in general is far from producing the level of customer experience truly necessary to provide excellent self-service shopping experiences. I recently heard Robin Terrell, Managing Director of John Lewis Direct in the UK (and Amazon alum), say “If it ain’t broke, you ain’t looking hard enough” in a talk about the need to improve customer experience. It’s a brilliant statement, and I totally agree with what he was saying.

So, “improving customer experience” is a huge and vague statement. Where do we start?

1. Recognize that it’s broke and you ain’t looking hard enough
   We’re still in our infancy in online retail, and we’ve got a long way to go. We too often try to increase our sales by generating more traffic and don’t spend enough time converting the traffic we’re already got. Often, the obstacles to conversion are not the big, shiny, whiz bang functionality; they’re lots of little things that add up to big problems. Those problems are hard to see without a concerted effort, as I discussed in more detail in my Tree Stump Theory post and other posts on conversion.
2. Truly learn how effective your site is from your customers’ perspective
   We can all identify lots of improvements we’d like to see on our sites, but it’s the improvements our customers most need that will drive our best growth. So understanding where we are and aren’t effective from our customers’ perspectives is critically important, but difficult.Focus groups and usability labs can be very helpful, but they can’t be our first or only methodology because it’s not possible to project learnings from a small group of people onto our entire population of customers.

   First, we need to quantitatively understand our effectiveness in the eyes of our total population, and that requires a statistically solid customer polling and analysis capability. Blatant and shameless plug alert: I’ve had great success using ForeSee Results in the past for exactly this purpose. Once we understand problem areas at a macro level, we can add a lot of color by interacting directly with customers in focus groups and usability labs. More details on this process can be found in my post entitled “Is elitism the source of poor usability?”

3. Consider getting some help from usability professionals
   Usability audits are different from usability labs. Usability auditors are professionally trained to understand how people interact with websites. Many of them have degrees in Human-Computer Interaction, a field that truly seeks to understand how people interact with software. These types of people can really help to identify problems with our user interfaces that untrained eyes have trouble seeing but which regularly obstruct customers from accomplishing their tasks.
4. Put in place a process to continuously improve
   This is really about budgetary and project management mindset. We must just accept the fact that we can’t tread water in a never-ending swimming race, and our only chance of competing is to keep swimming. We have to build our staffs, our budgets and our processes with the recognition that competing in the marketplace means continuously improving our customer experiences. Which leads to …
5. Wash, rinse, repeat
   Since the leaders in the marketplace are running this same cycle, we cannot rest. We must continue to recognize our sites are broken, continue to measure our effectiveness from our customers’ perspectives, find problems, fix them and begin again.

———————————–

We’ve got a lot of data that shows that retailers who best satisfy their customers generate the best financial results. I suppose that statement doesn’t sound like rocket science. But understanding that satisfaction has a direct relation to expectations and that our customers’ expectations can change independent of what we do on our own site is important. The leaders are continuously improving their sites, and they’re improvements are raising our customers’ expectations. We’ve all got to swim harder to keep pace.

What do you think? What’s your view on the marketplace? How have you see customer satisfaction affect your business?

Wanna be better with metrics? Watch more poker and less baseball.

Both baseball and poker have been televising their World Series championships, and announcers for both frequently describe strategies and tactics based on the statistics of the games. Poker announcers base their commentary and discussion on the probabilities associated with a small number of key metrics, while baseball announcers barrage us with numbers that sound meaningful but that are often pure nonsense.

Similarly, today’s web analytics give us the capability to track and report data on just about anything, but just because we can generate a number doesn’t mean that number is meaningful to our business. In fact, reading meaning into meaningless numbers can cause us to make very bad decisions.

Don’t get me wrong, I am a huge believer in making data-based decisions, in baseball, poker, and on our websites. But making good decisions is heavily dependent on using the right data and seeing the data in the right light. I sometimes worry that constant exposure to sports announcers’ misreading and misappropriation of numbers is actually contributing to a misreading and misunderstanding of numbers in our business settings.

Let’s consider a couple of examples of misreading and misappropriating numbers that have occurred in baseball over the last couple of weeks:

1. Selection bias
   This one is incredibly common in the world of sports and nearly as common in business. Recently, headlines here in Detroit focused on the Tigers “choking” and blowing a seven-game lead with only 16 games to go. In a recent email exchange on this topic, my friend Chris Eagle pointed out the problems with the sports announcers’ hyperbole:

   “They’re picking the high-water mark for the Tigers in order to make their statement look good.  If you pick any other random time frame (say end-of-August, which I selected simply because it’s a logical break point), the Tigers were up 3.5 games.  But it doesn’t look like much of a choke if you say the Tigers lost a 3.5 game lead with a month and change to go.”

   Unfortunately, this type of analysis error occurs far too often in business. We might find that our weekend promotions are driving huge sales over the last six months, which sounds really impressive until we notice that non-sale days have dropped significantly as we’ve just shifted our business to days when we are running promotions (which may ultimately mean we’ve reduced our margins overall by selling more discounted product and less full-price merchandise).

   In a different way, Dennis Mortensen addressed the topic in his excellent blog post “The Recency Bias in Web Analytics,” where he points out the tendency to give undue weight to more recent numbers. He included a strong example about the problems of dashboards that lack context. Dashboards with gauges look really cool but are potentially dangerous as they are only showing metrics from a very short period of time. Which leads me to…

2. Inconsistency of averages over short terms
   Baseball announcers and reporters can’t get enough of this one. Consider this article on the Phillies’ Ryan Howard after Game 3 of the World Series that includes, “Ryan Howard’s home run trot has been replaced by a trudge back to the dugout.The Phillies’ big bopper has gone down swinging more than he’s gone deep…He’s still 13 for 44 overall in the postseason (.295) but only 2 for 13 (.154) in the World Series.” Actually, during the length of the season, he had three times as many strike outs as home runs, so his trudges back to the dugout seem pretty normal. And the problem with the World Series batting average stat is the low sample size. A sample of thirteen at bats is simply too small to match against his season long average of .279. Do different pitchers or the pressures of the situation have an effect? Maybe, but there’s nothing in the data to support such a conclusion. Segmenting by pitcher or “postseason” suffers from the same small sample size problems, where the margin of error expands significantly. Furthermore, and this is really key, knowing an average without knowing the variability of the original data set is incomplete and often misleading.

   This problems with variability and sample sizes arise frequently in retail analysis when we either run a test with too small a sample size and assume we can project it to the rest of the business, or we run a properly sized test but assume we’ll automatically see those same results in the first day of a full application of the promotion. Essentially, the latter point is what is happening with Ryan Howard in the postseason. We often hear the former as well when a player is all of the sudden crowned a star when he outperforms his season averages over a few games in the postseason.

   In retail, we frequently see this type of issue when we’re comparing something like average order value of two different promotions or two variations in an A/B test. Say we’ve run an A/B test of two promotions. Over 3,100 iterations of test A, we have an average order size of $31.68. And over 3,000 iterations of Test B, we have an average order size of $32.15. So, test B is the clear winner, right? Wrong. It turns our there is a lot more variability in test B, which has a standard deviation of 11.37 compared with test A’s standard deviation of 7.29. As a result the margin of error on the comparison expands to +/- 48 cents, which means both averages are within the margin of error and we can say with 95% confidence that there really is no difference between the tests. Therefore, it would be a mistake to project an increase in transaction size if we went with test B.

   Check out that example using this simple calculator created by my fine colleagues at ForeSee Results and play around with your own scenarios.  Download Test difference between two averages.

Poker announcers don’t seem to fall into all these statistical traps. Instead, they focus on a few key metrics like the number of outs and the size of the pot to discuss strategies for each player based largely on the probability of success in light of the risks and rewards of a particular tactic. Sure, there are intangibles like “poker tells” that occur, but even those are considered in light of the statistical probabilities of a particular situation.

Retail is certainly more complicated than poker, and the number of potential variables to deal with is immense. However, we can be much more prepared to deal with the complexities of our situations if we take a little more time to view our metrics in the right light. Our data-driven decisions can be far more accurate if we ensure we’re looking at the full data set, not a carefully selected subset, and we take the extra few minutes to understand the effects of variability on averages we report. A little extra critical thinking can go a long way.

What do you think? Are there better ways to analyze key metrics at your company? Do you consider variability in your analyses? Do you find the file to test two averages useful?

Related posts:

How retail sales forecasts are like baby due dates

Are web analytics like 24-hour news networks

True conversion – the on-base percentage of web analytics

How the US Open was like a retail promotion analysis

The Right Metrics: Why keeping it simple may not work for measuring e-retail performance (Internet Retailer article)