Posts tagged: ForeSee Results

You ARE a technology company

In this day and age, pretty much every company is heavily dependent on technology to operate. But if you have an e-commerce operation (or really any sort of transaction website), you are a consumer technology company. The sooner we recognize and accept this fact, the sooner we can get on with leveraging it to our competitive advantage.

We often talk about focusing on our “core businesses” at the expense of everything else. At a conference I attended last week, I heard a number of speakers and attendees reference Amazon as a “technology company” as sort of a dismissal. They were basically saying, “Yes, Amazon has lots of great features and functionality and people rate their experience highly, but they’re a technology company. We’re retailers. We can’t compete on that level with them.” This type of statement draws the obvious retort: “So, then, on what level do you plan to compete?”

While Amazon does generate some revenue from selling technology services, the vast majority of their revenue comes from retailing products. Their financial statements look pretty much the same as most retailers (except they have much bigger numbers and growth rates). But Amazon and other pure play online retailers are not burdened with the type of legacy thinking that exists in a lot of multichannel retailers. They understand full well the value of creating a quality online experience, and they understand that technology is part of their core business.

Competing with Amazon is clearly very difficult for a variety of reasons (price being high on the list), but how many business elements can we abdicate to them before our very survival is at stake? Shifting our mindsets regarding our sites is one key way to claw back into the game.

Our websites are consumer software applications, in many ways like Microsoft Word or Quicken. And this means that online our business is technology.

People use our website applications to accomplish tasks like buying our products, learning more about our products or getting inspiration. Their perceptions about the quality of our applications can absolutely make the difference in whether or not they complete their tasks and whether or not they return to use our applications again.

And their perceptions of our brand can also be influenced by the quality of our site experiences. A study by ForeSee Results on the Internet Retailer Top 100 sites found that people who were satisfied with the online experience of a retailer were 44% more likely to purchase offline. That indicates significant value in making sure the website is a quality software experience.

Our websites are also an opportunity to differentiate from our competitors, particularly if we’re not selling proprietary products. If consumers can buy the same North Face jacket or Nikon camera from a variety of different retailers online, the quality of the online experience will be a contributing factor in the decision.

Let’s do what it takes to include the quality of our site experience in our value proposition.

Here are 3 ways to get started towards becoming a consumer technology company:

  1. Organization
    We will likely need to make organizational structure changes to support a consumer technology focus. I previously made a case for changes in E-commerce IT organization that goes into more detail, but suffice to say the technology strategy and the business strategy need to be not only aligned, but integrated.

    Furthermore, we need think about different types of roles. Software companies have product — not project — managers and product teams who are dedicated to building customer focused product strategies and life cycles. A quick check on the Amazon careers page reveals many product management positions. Do you have product management positions in your organization?

    Check out a typical set of responsibilities from Amazon’s Baby Registry product management gig and note the mix of business and technology functions and responsibilities:

    • Research and identify opportunities for Amazon to further distinguish our Baby Registry offering.
    • Define a long-term product roadmap, including technical, business development and marketing initiatives.
    • Develop new strategic partnerships ad drive day-to-day partner relationships.
    • Conduct business and financial analysis, including forecasting, monitoring, and reporting.
    • Define requirements, and drive customer experience projects and work with all relevant cross-functional areas and our technology teams to guarantee smooth, efficient implementation.
    • Manage bottlenecks, provide escalation management, anticipate and make trade-offs, balance the business needs versus technical constraints, and maximize business benefit while building great customer experiences
    • Work cross-functionally with designers, software development engineers, salespeople, product managers, and other internal partners.
  2. Budget/Investment
    How might our current budgets change if we considered ourselves  technology companies? Maybe not at all, but we should nonetheless re-examine our customer investment strategy in such a light. At the very least, we might consider revamping our budget processes to accommodate a fast moving, highly innovative competitive marketplace where the features and functionality of our website “product” are key parts of our business strategy and our ability to differentiate from our competitors.
  3. In house or outsource?
    Often we decide to outsource technology (and other elements of our businesses) because they are not “core” to our business and other people can do a better and more cost effective job. How does our thinking on outsourcing change if we consider ourselves technology companies? We might still legitimately consider outsourcing or licensing third party software, as many software companies do. However, we might also consider building up true competencies in at least some areas of software design and development because of the need to differentiate and deliver quality branded experiences for our customers.

—————————————————

Recognizing and accepting the fact that developing an e-commerce operation puts us in the consumer technology business is an important first step to successfully competing in the online marketplace. Once we’ve achieved the consumer technology mindset, we’ve got to take steps to create an organizational structure that executes like a consumer technology company. Without such steps, we will fall further and further behind the companies who are leveraging their technology focus to create the positive customer engagement cycles I discussed in my previous post.

What do you think? Do you think being in e-commerce means you’re in the in consumer technology business? How is your company organized?

Photo credit: Sebastian Bergmann


“If it ain’t broke, you ain’t looking hard enough”

The poor economy has done nothing to lower customer expectations of online retailers, and recent mixed results data from ComScore and ForeSee Results indicate that retailers who continue to improve their customer experiences are pulling away from their competitors in both sales and customer satisfaction.

ComScore reports online retail up 4% for the holiday season. While an increase is always nice, this is a much lower growth rate than online retail has seen in the past. And last year’s comparison base was far from stellar. ForeSee Results shows a significant drop in customer satisfaction year over year. Since satisfaction is predictive of future financial results, a drop is concerning.

But still, I wondered how sales could be up at all if satisfaction was so far down.

A deeper look at the ComScore data shows the Top 25 retailers growing 13% while “Small and Mid Tail” retailers are declining 10%. Satisfaction scores are also split, but the differences we’re seeing seem to be more based on those retailers who are continually improving their sites versus those whose cost containment measures have slowed or stopped improvements. It appears that the retailers who closely measure the effectiveness of their sites from their customers’ perspectives and continuously improve their customers’ experiences are the retailers with increasing customer satisfaction scores. Those retailers who didn’t improve customer experience this year are suffering declining satisfaction scores. Many of those in the Top 25 are the retailers who have continued to enhance their customer experiences. Those enhancements are not only helping them to increase their sales, but because of the high visibility and usage of those tops sites, they’re also raising consumer expectations of all sites.

Customer satisfaction can be best defined as the degree to which a customer’s actual experience meets his or her expectations. Therefore, rising expectations can depress satisfaction scores if customer experience improvements don’t keep pace.

In the rapidly changing world of online retail, stopping or delaying improvements is like treading water in a swimming race. While you may temporarily save some energy, you will fall hopelessly behind and your only hope of catching up is spending a lot more energy than you likely saved treading water

Growing online retail businesses realize and fully embrace the need for continuous improvements, and they also realize that online retail in general is far from producing the level of customer experience truly necessary to provide excellent self-service shopping experiences. I recently heard Robin Terrell, Managing Director of John Lewis Direct in the UK (and Amazon alum), say “If it ain’t broke, you ain’t looking hard enough” in a talk about the need to improve customer experience. It’s a brilliant statement, and I totally agree with what he was saying.

So, “improving customer experience” is a huge and vague statement. Where do we start?

  1. Recognize that it’s broke and you ain’t looking hard enough
    We’re still in our infancy in online retail, and we’ve got a long way to go. We too often try to increase our sales by generating more traffic and don’t spend enough time converting the traffic we’re already got. Often, the obstacles to conversion are not the big, shiny, whiz bang functionality; they’re lots of little things that add up to big problems. Those problems are hard to see without a concerted effort, as I discussed in more detail in my Tree Stump Theory post and other posts on conversion.
  2. Truly learn how effective your site is from your customers’ perspective
    We can all identify lots of improvements we’d like to see on our sites, but it’s the improvements our customers most need that will drive our best growth. So understanding where we are and aren’t effective from our customers’ perspectives is critically important, but difficult.Focus groups and usability labs can be very helpful, but they can’t be our first or only methodology because it’s not possible to project learnings from a small group of people onto our entire population of customers.

    First, we need to quantitatively understand our effectiveness in the eyes of our total population, and that requires a statistically solid customer polling and analysis capability. Blatant and shameless plug alert: I’ve had great success using ForeSee Results in the past for exactly this purpose. Once we understand problem areas at a macro level, we can add a lot of color by interacting directly with customers in focus groups and usability labs. More details on this process can be found in my post entitled “Is elitism the source of poor usability?”

  3. Consider getting some help from usability professionals
    Usability audits are different from usability labs. Usability auditors are professionally trained to understand how people interact with websites. Many of them have degrees in Human-Computer Interaction, a field that truly seeks to understand how people interact with software. These types of people can really help to identify problems with our user interfaces that untrained eyes have trouble seeing but which regularly obstruct customers from accomplishing their tasks.
  4. Put in place a process to continuously improve
    This is really about budgetary and project management mindset. We must just accept the fact that we can’t tread water in a never-ending swimming race, and our only chance of competing is to keep swimming. We have to build our staffs, our budgets and our processes with the recognition that competing in the marketplace means continuously improving our customer experiences. Which leads to …
  5. Wash, rinse, repeat
    Since the leaders in the marketplace are running this same cycle, we cannot rest. We must continue to recognize our sites are broken, continue to measure our effectiveness from our customers’ perspectives, find problems, fix them and begin again.

———————————–

We’ve got a lot of data that shows that retailers who best satisfy their customers generate the best financial results. I suppose that statement doesn’t sound like rocket science. But understanding that satisfaction has a direct relation to expectations and that our customers’ expectations can change independent of what we do on our own site is important. The leaders are continuously improving their sites, and they’re improvements are raising our customers’ expectations. We’ve all got to swim harder to keep pace.

What do you think? What’s your view on the marketplace? How have you see customer satisfaction affect your business?


Wanna be better with metrics? Watch more poker and less baseball.

Both baseball and poker have been televising their World Series championships, and announcers for both frequently describe strategies and tactics based on the statistics of the games. Poker announcers base their commentary and discussion on the probabilities associated with a small number of key metrics, while baseball announcers barrage us with numbers that sound meaningful but that are often pure nonsense.

Similarly, today’s web analytics give us the capability to track and report data on just about anything, but just because we can generate a number doesn’t mean that number is meaningful to our business. In fact, reading meaning into meaningless numbers can cause us to make very bad decisions.

Don’t get me wrong, I am a huge believer in making data-based decisions, in baseball, poker, and on our websites. But making good decisions is heavily dependent on using the right data and seeing the data in the right light. I sometimes worry that constant exposure to sports announcers’ misreading and misappropriation of numbers is actually contributing to a misreading and misunderstanding of numbers in our business settings.

Let’s consider a couple of examples of misreading and misappropriating numbers that have occurred in baseball over the last couple of weeks:

  1. Selection bias
    This one is incredibly common in the world of sports and nearly as common in business. Recently, headlines here in Detroit focused on the Tigers “choking” and blowing a seven-game lead with only 16 games to go. In a recent email exchange on this topic, my friend Chris Eagle pointed out the problems with the sports announcers’ hyperbole:

    “They’re picking the high-water mark for the Tigers in order to make their statement look good.  If you pick any other random time frame (say end-of-August, which I selected simply because it’s a logical break point), the Tigers were up 3.5 games.  But it doesn’t look like much of a choke if you say the Tigers lost a 3.5 game lead with a month and change to go.”

    Unfortunately, this type of analysis error occurs far too often in business. We might find that our weekend promotions are driving huge sales over the last six months, which sounds really impressive until we notice that non-sale days have dropped significantly as we’ve just shifted our business to days when we are running promotions (which may ultimately mean we’ve reduced our margins overall by selling more discounted product and less full-price merchandise).

    In a different way, Dennis Mortensen addressed the topic in his excellent blog post “The Recency Bias in Web Analytics,” where he points out the tendency to give undue weight to more recent numbers. He included a strong example about the problems of dashboards that lack context. Dashboards with gauges look really cool but are potentially dangerous as they are only showing metrics from a very short period of time. Which leads me to…

  2. Inconsistency of averages over short terms
    Baseball announcers and reporters can’t get enough of this one. Consider this article on the Phillies’ Ryan Howard after Game 3 of the World Series that includes, “Ryan Howard‘s home run trot has been replaced by a trudge back to the dugout.The Phillies’ big bopper has gone down swinging more than he’s gone deep…He’s still 13 for 44 overall in the postseason (.295) but only 2 for 13 (.154) in the World Series.” Actually, during the length of the season, he had three times as many strike outs as home runs, so his trudges back to the dugout seem pretty normal. And the problem with the World Series batting average stat is the low sample size. A sample of thirteen at bats is simply too small to match against his season long average of .279. Do different pitchers or the pressures of the situation have an effect? Maybe, but there’s nothing in the data to support such a conclusion. Segmenting by pitcher or “postseason” suffers from the same small sample size problems, where the margin of error expands significantly. Furthermore, and this is really key, knowing an average without knowing the variability of the original data set is incomplete and often misleading.

    This problems with variability and sample sizes arise frequently in retail analysis when we either run a test with too small a sample size and assume we can project it to the rest of the business, or we run a properly sized test but assume we’ll automatically see those same results in the first day of a full application of the promotion. Essentially, the latter point is what is happening with Ryan Howard in the postseason. We often hear the former as well when a player is all of the sudden crowned a star when he outperforms his season averages over a few games in the postseason.

    In retail, we frequently see this type of issue when we’re comparing something like average order value of two different promotions or two variations in an A/B test. Say we’ve run an A/B test of two promotions. Over 3,100 iterations of test A, we have an average order size of $31.68. And over 3,000 iterations of Test B, we have an average order size of $32.15. So, test B is the clear winner, right? Wrong. It turns our there is a lot more variability in test B, which has a standard deviation of 11.37 compared with test A’s standard deviation of 7.29. As a result the margin of error on the comparison expands to +/- 48 cents, which means both averages are within the margin of error and we can say with 95% confidence that there really is no difference between the tests. Therefore, it would be a mistake to project an increase in transaction size if we went with test B.

    Check out that example using this simple calculator created by my fine colleagues at ForeSee Results and play around with your own scenarios.  Download Test difference between two averages.

Poker announcers don’t seem to fall into all these statistical traps. Instead, they focus on a few key metrics like the number of outs and the size of the pot to discuss strategies for each player based largely on the probability of success in light of the risks and rewards of a particular tactic. Sure, there are intangibles like “poker tells” that occur, but even those are considered in light of the statistical probabilities of a particular situation.

Retail is certainly more complicated than poker, and the number of potential variables to deal with is immense. However, we can be much more prepared to deal with the complexities of our situations if we take a little more time to view our metrics in the right light. Our data-driven decisions can be far more accurate if we ensure we’re looking at the full data set, not a carefully selected subset, and we take the extra few minutes to understand the effects of variability on averages we report. A little extra critical thinking can go a long way.

What do you think? Are there better ways to analyze key metrics at your company? Do you consider variability in your analyses? Do you find the file to test two averages useful?



Related posts:

How retail sales forecasts are like baby due dates

Are web analytics like 24-hour news networks

True conversion – the on-base percentage of web analytics

How the US Open was like a retail promotion analysis

The Right Metrics: Why keeping it simple may not work for measuring e-retail performance (Internet Retailer article)

Sitting in the “Marketing Hot Seat”

My good buddy Adam Cohen, a Rosetta partner who heads up their Search, Online and Social Media businesses, issued a challenge called “The Marketing Hot

You’re the CMO.  You
have a marketing budget of $1M.  Your company is a consumer product
company, relatively unknown / early stage.  Customers who know the
product like it. CEO wants ROI within 12 months.  What do you do?

I thought this would be a fun exercise to take on, particularly because the scenario placed me in the seat of a manufacturer, publisher or product company. Would my retail oriented perspective provide a different line of thinking than would typically come from a manufacturer, and would that perspective be worthwhile? I’d certainly love to know your thoughts.

My take is actually the first one Adam posted on his blog, A Thousand Cuts. Check things out over there over the next few weeks to see perspectives from the other 12 bloggers.

Here’s my answer to Adam’s challenge:

OK.
Setting aside all the caveats about the fact that I don’t know what the product is, what it costs to make and what our margins are, here’s generically how I would approach the situation:

Strategy

  1. Thoroughly understand the customers who like our product
    The customers who know our product like it. We need to find out why, in their words, and determine what personality traits, hobbies, demographics, etc. in those customers are relevant to their liking our products so that we can speak to others like them.
  2. Get our online destinations right
    With a relatively small marketing budget, we’re going to need to maximize our online strategy. (Actually, we should do that even if have a large marketing budget.) We need to make sure our website and our retailer websites are highly usable and highly effective in merchandising our product and providing the ability for customers to easily spread the word about us.
  3. Drive traffic with whatever budget is left
    Only when we have ensured that we have solid destinations for our traffic will we start to actively search for traffic.


Tactics

  1. Learn as much as we can about the customers who most love the product.
    Why do they like it? What are there personality types; let’s use the Myers-Briggs personality test and really get a  thorough understanding of these folks. How do they describe our product? Let’s pay attention to the words they  use as we’re going to reuse those words in our copy.
  2. Hire ForeSee Results to measure our site’s effectiveness from our customers’ perspectives.
    I realize this may seem self-serving since it’s my company, but I was a client for seven years before joining the  company three months ago, and I’ve see how well it works.  So, I want it in this role. So there! We’ll use  measurements, analysis, Session Replay and usability audits to ensure we’re providing the best experience  we can.
  3. Hire Bryan Eisenberg to develop archetypes and to implement Persuasion Architecture on our site.
    We need to speak to customers in language that resonates, and Bryan understands how to do that. We’ll also use  his language for product descriptions and other content we give to retailers for their sites.
  4. Create a high quality product video.
    We’ll use this video on our own site and we’ll give it to retailers for their sites. We’ll focus on the key aspects  customers love and use copy that includes words that resonate with those customers. We’ll also show real  customer testimonials.
  5. Launch customer reviews and customer forums on our site
    We need to make sure our customers can openly provide their thoughts about our product, even when  they’re negative.
  6. Launch several blogs on our site
    Since we only have one product, we need to provide some fresh and compelling content on our site to give people a reason to come back. The content doesn’t need to be about the product all the time. It can be able anything, as  long as it’s compelling. I’ll focus on general marketing, our CEO can blog about leadership, and we’ll find some  people to blog about topics our customers are interested in. All of this blog content will also be great for SEO.
  7. Launch a marketing campaign to retailers informing them about key customers and teaching them how to sell the product
    Our initial marketing efforts will essentially be internal. Let’s get the sellers pumped up and doing their jobs well  before we send customers their way.
  8. Develop a widget for retailers that gives customers the ability to easily share information about the product
    We need to give our customers ways to share information about our product on their own in a way that is easy and  positive. Let’s create a fun widget that people want to share on Facebook, Twitter, email, etc.
  9. Get our SEO right, buy search terms, send emails, run re-marketing campaigns, etc.
    I don’t want to minimize the value of these techniques, but we really need to make sure our destinations are right  before we add lots of traffic.So there you have it. My main point here is to focus on the customers first, the destination second and the traffic driving last.

What do you think? Does my strategy make sense? How would you have addressed the challenge? Do your manufacturer/publisher/product partners address your needs?

True conversion – the on-base percentage of web analytics?

I just finished re-reading one of my all-time favorite business books, Moneyball by Michael Lewis. While on the surface Moneyball is a baseball book about the General Manager of the Oakland A’s, Billy Beane, I found it to be more about how defying conventional wisdom (a topic I’ll no doubt return to over and over in this space) can be an excellent competitive advantage. In retail, we can be just as prone to conventional wisdom and business as usual as the world of baseball Lewis encountered, and site conversion rate is an excellent example of how we’re already traversing that path in the relatively young world of e-commerce.

In Moneyball, Michael Lewis tells the story of Beane defying the conventional wisdom of longtime baseball scouts and  baseball industry veterans. Rather than trust scouts who would literally  determine a baseball player’s prospects by  how he physically looked, Beane went to the data as a disciple of Bill JamesSabermetrics theories. By following the  James’ approach, Beane was able to put together consistently winning teams while working with one of the lowest payrolls in the Major Leagues.

Lewis describes how James took a new look at traditional baseball statistics and created new statistics that were  actually more causally related to winning games. Imagine that! For example, James found on-base percentage, which  includes walks when calculating how often a player gets on base, to be a much more reliable statistic than batting  average, which ignores walks (even though we’re always taught as Little Leaguers that a walk is as good as a hit). I won’t get into all the details, but suffice to say on-base percentage is more causally related to scoring runs than batting  average, and scoring runs is what wins games.

So why is batting average still so prevalent and what does this have to do with retail?

Basically, an English statistician named Henry Chadwick developed batting average as a statistic in the late 1800s and didn’t include walks because he thought they were caused by the pitcher and therefore the batter didn’t deserve credit for not swinging at bad pitches. Nevermind that teams with batters who got on base scored more runs and won more games. But batting average has been used so long that we just keep on using it, even when it’s been proven to not be very valuable.

OK, baseball boy, what about the retail?

As relatively young as the e-commerce space is, I believe we are already falling prey to  conventional wisdom in some of our metrics and causing ourselves unnecessary churn.  My favorite example is site conversion rate. Conversion is a metric that has been used in physical retail for a very long time, and it makes good sense in stores where the overwhelming purpose is to sell products to customers on their  current visit.

I’ll argue, though, that our sites have always been about more than the buy button, and they are becoming more and more all-purpose every day. They are marketing and merchandising vehicles, brand builders, customer research tools (customers researching products and us researching customers), and sales drivers, both in-store and online. Given the multitude of purposes of our sites, holding high a metric that covers only one purpose not only wrongly values our sites, but it also causes us to churn unnecessarily when implementing features or marketing programs that encourage higher traffic for valuable purposes to our overall businesses that don’t necessarily result in an online purchase on a particular day.

We still need to track the sales generating capabilities of our sites, but we want to find a causal metric that actually focuses on our ability or inability to convert the portion of our sites’ traffic that came to buy. We used our site for many purposes at Borders, so we found that changes in overall site conversion rate didn’t have much to do at all with changes in sales.

If we wanted to focus on a metric that tracked our selling success, we needed to focus on the type of traffic that likely came with an intent to buy (or at least eliminate the type of traffic that came for other reasons), and we knew through our ForeSee Results surveys that our customers who came with an intent to buy on that visit was only a percentage of our total visitors, while the rest came for other reasons like researching products, finding stores, checking store inventory, viewing video content, etc.

So, how could we isolate our sales conversion metrics to only the traffic that came with an intent to buy?
Our web analyst Steve Weinberg came up with something we called “true conversion” that measured adds to cart  divided by product page views multiplied by orders divided by checkout process starts. This true conversion metric was far more correlative to orders than anything else, so it was the place to initially focus as we tried to determine if we could turn the correlation into causation. We still needed to do more work matching the survey data to path analysis to further refine our metrics, but it was a heckuva lot better than overall site conversion, which was basically worthless to us.

Every site is different, so I don’t know that all sites could take the exact same formula described above and make it work. It will take some work from your web analyst to dig into the data to determine customer intent and the pages that drive your customers ability to consummate that intent. For more ideas, I highly recommend taking a look at Bryan Eisenberg‘s excellent recent topic called How to Optimize Your Conversion Rates where he explores some of these topics in more detail.

—————————————–

Whether or not you buy into everything written in Moneyball or all of Billy Beane’s methods, I believe the main lesson to be culled from the book is that it’s critically important that we constantly re-evaluate our thinking (particularly when conventional wisdom in assumed to be true) in order to get at deeper truths and clearer paths to success.

How is overall site conversion rate working for you? Do you have any better metrics? Where have you run into trouble with conventional wisdom?


Retail: Shaken Not Stirred by Kevin Ertell


Home | About