I’ve been thinking a lot lately about how angry improperly used statistics make me. Now, I know that this might seem like a really geeky thing to get upset about, but it’s a type of purposeful (generally, sometimes it’s just ignorant) falseness that upsets most people in other contexts. Contexts like baseball for example.
Imagine you’re watching a baseball game, a really important one. It’s tied in the 9th inning of game seven in a playoff series between your favorite team and their hated rival. Your best player is at the plate and fires a line drive way back into left field. The batter takes off around the bases—he passes first and second base as the outfielder fumbles for the ball—he gets to third base and now it’s a race between him and the ball to home plate. He’s racing forward, he’s almost there, he throws himself down, he slides in, and his fingers touch home plate just an instant before the ball thumps into the catcher’s glove. He’s in! But wait…the UMPIRE IS SIGNALLING OUT!
Whether you care about baseball or not, you know what happens now. Every fan in the room leaps to his or her feet and start screaming at the TV. That dirty bastard! That looser! That damn, cheating, moron! Is he BLIND? Is he purposefully tipping the game for the other team?! I mean, really, how could he make that obviously false call!!!!!
This is how I feel when people manipulate statistics. All the rage and injustice I feel when some official ruins a game for the Sox or the Steelers also wells up in me when someone makes the statement that more people get ill from pasteurized milk than raw milk. Or that more Americans own guns than Canadians. It makes me livid.
Let me give you a hypothetical example of how statistics can be manipulated.
Here are three made-up facts:
- A) Each year 100,000 Americans drink pasteurized milk and 200 Americans drink raw (unpasteurized) milk.
- B) Each year 20 Americans get sick after drinking pasteurized milk and 5 Americans get sick after drinking raw milk.
- C) Each year 2 Americans die from illness related to the consumption of pasteurized milk and no Americans die from illness related to the consumption of raw milk.
Once again, this is ALL MADE-UP. None of the “facts” above are real.
So, with these “facts,” I can say many different things. All of them true, but some rather misleading.
For example, I can honestly say that more Americans become ill each year after drinking pasteurized milk than do after drinking raw milk—in fact, I can say that four times as many people become ill from drinking pasteurized milk! However, this statement fails to show how much more likely someone is to become ill after drinking raw milk.
A more honest way of comparing the stats on illness from milk drinking would be to use a percentage: only 0.02% of Americans become ill after drinking pasteurized milk as compared to 2.5% of Americans who become ill after drinking raw milk. Now you can see that you are 125% more likely to become ill after drinking raw milk, something that does not come through with just a statement of the numbers 20 versus 5.
But even with percentages, we’re still not telling people the whole story. I think another very important thing when reporting facts that revolve around numbers is to explain things that might not come through on a first glance. What about the severity of the illnesses? True, you’re 125% more likely to get sick from drinking raw milk, but how sick? Are we talking sniffles or kidney failure? This is also important.
In my made-up scenario, milk drinkers might still be better off with raw milk despite the higher incidence of illness. 10% of the people who became sick after drinking pasteurized milk died. But none of the people who because sick after drinking raw milk died. This would be another good way to argue for raw milk. But it’s still leaving out the incredibly small sample size. Only 5 people who drank raw milk got sick at all. Can you get any significant (statistically relevant data that can act as a predictor of future events) data from just 5 people?
If I were going to write something about the relative safety of raw versus pasteurized milk based on the “facts” above, I would say something like this:
What does the science say about the raw milk safety debate? The data is not exactly conclusive. Each year many more Americans consume pasteurized milk than raw milk, however, the percentage of raw milk drinkers who become ill is over 100% greater than that of pasteurized milk drinkers. It’s possible that the illnesses caused by drinking pasteurized milk are more serious, as the rate of death from pasteurized milk infections is higher than that for raw milk. More data needs to be gathered and analyzed on illness contracted by raw milk drinkers.