Posts tagged: Fooled by Randomness

Why most sales forecasts suck…and how Monte Carlo simulations can make them better

Sales forecasts don’t suck because they’re wrong.  They suck because they try to be too right. They create an impossible illusion of precision that ultimately does a disservice to managers who need accurate forecasts to assist with our planning. Even meteorologists — who are scientists with tons of historical data, incredibly high powered computers and highly sophisticated statistical models — can’t forecast with the precision we retailers attempt to forecast. And we don’t have nearly the data, the tools or the models meteorologists have.

Luckily, there’s a better way. Monte Carlo simulations run in Excel can transform our limited data sets into statistically valid probability models that give us a much more accurate view into the future. And I’ve created a model you can download and use for yourself.

There are literally millions of variables involved in our weekly sales, and we clearly can’t manage them all. We focus on the few significant variables we can affect as if they are 100% responsible for sales, but they’re not and they are also not 100% reliable.

Monte Carlo simulations can help us emulate real world combinations of variables, and they can give us reliable probabilities of the results of combinations.

But first, I think it’s helpful to provide some background on our current processes…

We love our numbers, but we often forget some of the intricacies about numbers and statistics that we learned along the way. Most of us grew up not believing a poll of 3,000 people could predict a presidential election. After all, the pollsters didn’t call us. How could the opinions of 3,000 people predict the opinions of 300 million people?

But then we took our first statistics classes. We learned all the intricacies of statistics. We learned about the importance of properly generated and significantly sized random samples. We learned about standard deviations and margins of errors and confidence intervals. And we believed.

As time passed, we moved on from our statistics classes and got into business. Eventually, we started to forget a lot about properly selected samples, standard deviations and such and we just remembered that you can believe the numbers.

But we can’t just believe any old number.

All those intricacies matter. Sample size matters a lot, for example. Basing forecasts, as we often do, on limited sets of data can lead to inaccurate forecasts.

Here’s a simplified explanation of how most retailers that I know develop sales forecasts:

  1. Start with base sales from last year for the the same time period you’re forecasting (separating out promotion driven sales)
  2. Apply the current sales trend (which is maybe determined by an average of the previous 10 week comps). This method may vary from retailer to retailer, but this is the general principle.
  3. Look at previous iterations of the promotions being planned for this time period. Determine the incremental revenue produced by those promotions (potentially through comparisons to control groups). Average of the incremental results of previous iterations of the promotion, and add that average to the amount determined in steps 1 and 2.
  4. Voilà! This is the sales forecast.

Of course, this number is impossibly precise and the analysts who generate it usually know that. However, those on the receiving end tend to assume it is absolutely accurate and the probability of hitting the forecast is close to 100% — a phenomenon I discussed previously when comparing sales forecasts to baby due dates.

As most of us know from experience, actually hitting the specific forecast almost never happens.

We need accuracy in our forecasts so that we can make good decisions, but unjustified precision is not accuracy. It would be far more accurate to forecast a range of sales with accompanying probabilities. And that’s where the Monte Carlo simulation comes in.

Monte Carlo simulations

Several excellent books I read in the past year (The Drunkard’s Walk, Fooled by Randomness, Flaw of Averages, and Why Can’t You Just Give Me a Number?) all promoted the wonders of Monte Carlo simulations (and Sam Savage of Flaw of Averages even has a cool Excel add-in). As I read about them, I couldn’t help but think they could solve some of the problems we retailers face with sales forecasts (and ROI calculations, too, but that’s a future post). So I finally decided to try to build one myself. I found an excellent free tutorial online and got started. The results are a file you can download and try for yourself.

A Monte Carlo simulation might be most easily explained as a “what if” model and sensitivity analysis on steroids. Basically, the model allows us to feed in a limited set of variables about which we have some general probability estimates and then, based on those inputs, generate a statistically valid set of data we can use to run probability calculations for a variety of possible scenarios.

It turns out to be a lot easier than it sounds, and this is all illustrated in the example file.

The results are really what matters. Rather than producing a single number, we get probabilities for different potential sales that we can use to more accurately plan our promotions and our operations. For example, we might see that our base business has about a 75% chance of being negative, so we might want to amp up our promotions for the week in order have a better chance of meeting our growth targets.  Similarly, rather than reflexively “anniversaring” promotions, we can easily model the incremental probabilities of different promotions to maximize both sales and profits over time.

The model allows for easily comparing and contrasting the probabilities of multiple possible options. We can use what are called probability weighted “expected values” to find our best options. Basically, rather than straight averages that can be misleading, expected values are averages that are weighted based on the probability of each potential result.

Of course, probabilities and ranges aren’t as comfortable to us as specific numbers, and using them really requires a shift in mindset. But accepting that the future is uncertain and planning based on the probabilities of potential results puts us in the best possible position to maximize those results. Understanding the range of possible results allows for better and smarter planning. Sometimes, the results will go against the probabilities, but consistently making decisions based on probabilities will ultimately earn the best results over time.

One of management’s biggest roles is to guide our businesses through uncertain futures. As managers and executives, we make the decisions that determine the directions of our companies. Let’s ensure we’re making our decisions based on the best and most accurate information — even if it’s not the simplest information.

What do you think? What issues have you seen with sales forecasts? Have you tried my example? How did it work for you?

Best Business Books of the Year

With the holiday season upon us, I thought I would write about my favorite business books of the year to provide some gift giving ideas for you and your teams. Here, in no particular order, are my favorites among the books I read this year. (Note: These books were not all published this year, but since I read them this year I’m including them in my list.)

Six Pixels of Separation: Everyone is Connected. Connect Your Business to Everyone.
by Mitch Joel

Six Pixels of Separation begins as a primer for any business leader with limited knowledge of the Internet’s capabilities and quickly turns into an indispensable set of guidelines and advice for any business person who plans to make use of the web (which should be any business person). Mitch Joel offers excellent insight and plenty of simple, direct, digestible advice. This is a must read.

The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty
by Sam L. Savage

Every business person should read this book. We are so often looking for precise numbers when precise numbers are unrealistic. The reality is, we would actually be much more accurate to use probabilities and ranges when referencing uncertain number such as sales forecasts or project timelines. Savage takes us through the dangers of using averages to describe distributions and offers solid solutions that can be used to better manage our business.
Preview Flaw of Averages

Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets
by Nassim Nicholas Taleb

This book made me think more than any book in recent memory. That may be partly because it’s pretty dense and I had to read it more slowly than I normally read. However, I’ll give a lot more credit to the fact that Taleb’s makes some very interesting points about the amount of randomness in our lives and how that randomness is all too often mistaken for something more substantive.
Preview Fooled by Randomness

How We Decide
by Jonah Lehrer
I loved this book. Jonah Lehrer takes us through some fairly common behavior economics principles and experiments, but the very interesting twist he takes is to explain the brain mechanics that drive our thinking and decisions. He really uncovers why we’re “predictably irrational” and provides great insight into how we make decisions and how we can use that knowledge to improve our decision making.

The Drunkard’s Walk: How Randomness Rules Our Lives
by Leonard Mlodinow

I’m on a randomness kick lately, and this is the book that got me started on it. Mlodinow does a nice job of illustrating some of the finer statistical points in a pretty accessible manner. While this book isn’t as deep at the book I’m currently reading, “Fooled by Randomness,” it’s definitely an easier read and does a nice job of covering the basics.
Preview The Drunkard’s Walk

Sway: The Irresistible Pull of Irrational Behavior
by Ori Brafman, Rom Brafman

Another one of the behavior economics books I so love. This one has some pretty interesting stories and anecdotes, and its insights benefit from one of the writers being a psychologist and the other a businessman.
Preview Sway

More Than a Motorcycle: The Leadership Journey at Harley-Davidson
By Rich Teerlink and Lee Ozley

This is a very interesting book about culture change at Harley-Davidson during the ’90s written by the CEO and lead consultant who initiated the change. It can be a bit dry at times, but the details behind the thinking and the execution are excellent. I learned a lot by reading it.
Preview More than a Motorcycle

And here are some great books that I re-read this year:

The OPEN Brand: When Push Comes to Pull in a Web-Made World
by Kelly Mooney, Nita Rollins
The world is changing rapidly, and those who fail to realize it will be left in the dust. However, those who open their brand and see the value of allowing their best customers to participate in the brand will not only reap the benefits of those customers ideas, but they will also benefit from those customers becoming the largest and more credible Marketing department a company could have. Kelly Mooney and Nita Rollins explore these themes in an extremely insightful book that comes with lots of examples that help the reader visualize how these ideas could apply to his or her own business. The writing style and formatting is fun and extremely easy to read. This is a great handbook for any marketer in the 21st century.

Moneyball: The Art of Winning an Unfair Game
by Michael Lewis

While this is ostensibly a baseball book about the success of Oakland A’s GM Billy Beane, I actually found this to be an excellent business book. Michael Lewis tells the story of Beane defying the conventional wisdom of longtime baseball scouts about what good baseball players look like. Rather than trust scouts who literally would determine a baseball player’s prospects by how he physically looked, Beane went to the data as a disciple of Bill James’ Sabermetrics theories. Lewis describes how James took a new look at traditional baseball statistics and created new statistics that were actually more causally related to winning games. By following the James’ approach, Beane was able to put together consistently winning teams while working with one of the lowest payrolls in the Major Leagues. How can the same principles of trusting data over tradition and “gut” play in the business world? That is a thought I constantly ponder thanks to reading this book.
Preview Moneyball

The Culture Code: An Ingenious Way to Understand Why People Around the World Live and Buy as They Do
by Clotaire Rapaille

I picked this book up on a whim one day because the title was interesting. I was quickly engrossed by reading the story in the introduction of Clotaire Rapaille’s work with Chrysler on Jeep Wrangler. He describes the “code” word for Jeep in America is HORSE and advises executives to design round headlights instead of square headlights because horses have round eyes. They think he’s nuts, of course, but when it turns out round headlights are cheaper they go with them — and they’re a hit. They also then position the Wrangler as a “horse” in their ads and have great success. Rapaille goes on to describe what he means by “culture code” and details some of the hidden cultural patterns that affect most all of us. Some samples of other codes within the book are:
– The American Culture Code for love is FALSE EXPECTATION
– The female code for sex is VIOLENCE (Whoa! You’ve got to read the book to understand)
– The code for hospital in America is PROCESSING PLANT

There are tons more of these interesting observations embedded in short, easy-to-read chapters. Whether or not you buy into everything he says, it’s very interesting to see how he developed each code and certainly will expand your understanding of how and why people behave as they do under the powerful forces of culture
Preview The Culture Code

Predictably Irrational: The Hidden Forces That Shape Our Decisions
by Dan Ariely

This is the book that first turned me on to the fascinating world of behavioral economics. Ariely does an excellent job of explaining many of the core principles of behavioral economics with stories and experiments. Every retailer should read this book to better understand how people (customers) think and behave. It will absolutely open your eyes.

Those are some of my favorites. I’m always looking for a new read. What books fired you up this year?

The immense value of “slop time”

Lately, I’ve been thinking a lot about thinking. We spend such a large portion of our days reacting to issues flying at us from all directions that we can easily lose sight of where we’re headed and why we’re going there. We’re so busy that we don’t have time to think, and failing to allot time to think is ultimately counterproductive. Taking time (and even scheduling time) to reflect on past actions and consider future courses of action is more important than we often realize.

Consider this quote from former Intel exec Dov Frohman in his book Leadership the Hard Way and also discussed on this Practice of Leadership blog posting:

“Every leader should routinely keep a substantial portion of his or her time—I would say as much as 50 percent—unscheduled. Until you do so, you will never be able to develop the detachment required to identify long-term threats to the organization or the flexibility to move quickly to take advantage of random opportunities as they emerge. Only when you
have substantial ’slop’ in your schedule—unscheduled time—will you have the space to reflect on what  you are doing, learn from experience, and recover from your inevitable mistakes. Leaders without such  free time end up tackling issues only when there is an immediate or visible problem.”

Frohman makes some excellent points about the need to learn from experience and pull the value from the mistakes we make. Truly understanding the pros and cons of past decisions, ideally with the benefit that hindsight and new learning gives us, helps us better prepare for future decisions.

But there’s so much going on every day, and with staff cuts we have more work than ever. How can we possibly afford to time to think?
Well, Frohman has a ready answer:

“Managers’ typical response to my argument about free time is, ‘That’s all well and good, but there are  things I have to do.’ Yet we waste so much time in unproductive activity—it takes an enormous effort on  the part of the leader to keep free time for the truly important things.”

Of course, that’s easy to say and considerably harder to do. But it’s so important. Without taking the time to focus on the most important issues, tactics and strategies, we end up constantly fighting fires and ultimately working our way into a death spiral.

I find that if I give my think time enough priority, I can find a way to get it in. For me, actually scheduling time on my calendar makes all the difference. It also forces me to put some of the daily issues into perspective and postpone or even cancel meetings that don’t rate highly enough on the prioritization scale.

So, what do we do with this newly scheduled time to think?

Reflect on past decisions
I’ve recently started spending some time actively thinking through the decisions I made during the previous week or so. It’s amazing how hard it was at first to think of many decisions I made, particularly the numerous small decisions that happen every day. They came and went so fast that I didn’t really immediately retain them and their effects. Where they good decisions or bad decisions? It made me wonder if I could make better decisions in the future just by doing a better job of examining past decisions.

Open up to new ideas and learn something new
I am constantly hungry for new ideas. I love to read interesting new books, and I try to read as many blogs as I can. Of course, all of that reading takes time, so I look for my opportunities wherever I can. I try to read for at least a half hour every night, and I’m always looking for books that will expand my thinking.

I’m currently reading a very interesting book called How We Decide by Jonah Lehrer. It’s essentially about behavioral economics (a fascinating field with all sorts of retail implications) but the twist is that he actually examines the inner mechanics of the brain to explain why we do what we do. He’s a good story teller and it doesn’t get to “scienc-y.” (Is that a word?)

Fooled by randomness Another book that has me thinking more than any book I’ve read in a very long time is Fooled By Randomness by Nassim Nicholas Taleb. How much time have we mis-spent reacting to data that lacks statistical significance? Could some focused learning on the events that fool us time and again prevent us from making bad decisions in the future?

I use Google Reader to follow many thought provoking blogs, including those listed on the right column here. I also use the Newsstand application on my iPhone, which syncs with Google Reader and allows me to take in a blog or two at all sorts of random moments when I have a little bit of time on my hands. In fact, during my blog reading recently I even came upon a list of new an “out-of-the-box” ways to inject thinking in your business from Mitch Joel.

Anticipate the future
After analyzing past decisions and opening up my mind to new ideas, I try taking some time to start anticipating the future. Here, I think it’s definitely important to imagine large strategic shifts in the marketplace, but it’s also important to consider daily issues that come up with staff, marketing tactics, etc. as well. How are different types of decisions made in the organization, and who makes them? Is decision making authority matched with accountability? Are decision makers aware of their boundaries? Are the boundaries appropriate? Is the business strategy correct and clearly communicated? Are we working towards the right objectives? Should I consider a different approach when working with a particular person? Should I go with the ham or the turkey for lunch. 🙂


You’re clearly reading at least one blog today, so it’s good that you’ve already made some time in your day. Good news! I hope you’ll be back, and I hope you’re also taking some time to read more of the really great content that’s available out there in both book and blog form. I hope you’ll come across something so mind-blowingly thought provoking that it changes the way you think about something. I hope you’ll be so open to new ideas that you won’t be afraid to change your mind about past decisions and direction. (Side note pet peeve of mine: Why do we criticize leaders and politicians who change their minds? Would you rather work with someone who can change his or her mind in the face of new information or someone who stubbornly sticks to convictions no matter what?)

And, if you haven’t already, I hope you’ll consider adding some “slop time” to your schedule to allow you to reflect on past decisions, open up to new ideas and new learning, and anticipate the future.

Now it’s your turn. I’d love to hear how you find time to think. What are your sources of expanded thinking? Will you share any great books or blogs that you’ve read? What’s changed your thinking recently?

Retail: Shaken Not Stirred by Kevin Ertell

Home | About