Imagine I told you that the average height of all the people in the entire world is about 6.2 feet. And when you ask me how I learned that, I replied, “It was simple, really. I just measured myself.”
After getting over the initial shock of my apparent stupidity, you’d probably inform me that I can’t just measure myself and draw such a conclusion. I’d need a much larger sample size.
And, of course, you’d be right. In obvious examples like this, we have an intuitive sense for what statisticians call the law of large numbers,1 which states, “As a sample size grows, its mean gets closer to the average of the whole population.”
But there are many less clear-cut situations where we fail to take sample size into account. Consider, for example, this statement:
In a telephone poll of 300 seniors, 60 percent support the president.2
If you had to create a headline of three words to describe those findings, what would it be? If you’re like most people, you’ll write “Elderly Support President.”
And those words do a good job of conveying the essence of the story. But the details of the poll design—that it was done over the phone with a sample of just 300—gets left out.
The takeaway? We pay much more attention to the content of messages than we do to the reliability of the information.
You’ve probably seen plenty of commercials where four out of five dentists recommend some toothpaste. That seems convincing, but only because we focus on the content of the message—authorities in the field think highly of the product.
If we instead turned our attention to the reliability of the information—the fact that we don’t know the sample size—we’d come to a very different conclusion. After all, there’s a good chance only five dentists were actually asked. And if you did a proper poll with a random sampling of 1000 dentists, perhaps only 20 percent would recommend the brand.
Accurate assumptions require sufficient sample sizes. But in everyday life, we often forget about that and, as a result, make hasty generalizations. Here are a few examples:
- Your dad smokes four packs of cigarettes per day and lives to be eighty years old. He’s just one person, but you still conclude that smoking can’t be that bad for you.
- You meet someone for the first time, and he makes a bad impression on you. It’s just one meeting, but you still assume that he’s a rude person.
- You try investing in the stock market and lose half of the money spent in your first week. It’s a short period of time, but you still conclude the stock market isn’t for you.
As you can see, hasty generalizations often lead to inaccurate conclusions, bad judgments, and poor decisions. So always pay attention to the reliability of information. When the sample size is small, suspend your judgment and try to find more reliable data.
This article is an excerpt from my book The Decision-Making Blueprint.