The Numbers Game

The goal of the British authors, Michael Blastland and Andrew Dilnot, is to help readers view the numbers they encounter in the news with a more critical eye. They do this with humorous but true examples, although sometimes the point is belabored. Luckily, they offer a summary list of pointers, which I have grouped and modified below.

On counting:

• Numbers are tidy; life isn’t. When looking at numbers, ask whether the definitions used for counting are rock-hard or like strawberry jam.
• Most counting isn’t. Think of trying to determine the number of immigrants, the number of HIV/AIDS cases, or the number of people killed during the Iraq war. Drinking from a firehouse is easier than statistical sampling.
• Size is for sharing. When hearing about numbers, always ask, “Is that a big number?” A helpful trick is to think of numbers as seconds: a million seconds is about 11.5 days; a billion seconds is nearly thirty-two years. However, numbers are only meaningful when they have been converted to a human scale that recognizes human experience. So, when encountering big numbers, divide the number by all the people it will affect. For example, divide any number regarding U.S. public spending by 15.6 billion to see its weekly worth. This number is about the amount the U.S. government needs to spend in a year to equal $1 per person per week: 15.6 billion = 300 million (approximate U.S. population) x 52 (number of weeks in a year).

On patterns and risks:

• Chance lurks. Whenever we see patterns or clusters in numbers (e.g. cancer rates across the country; random tosses of a coin), sometimes there are no explanations. Similarly, numbers regarding performance (e.g., economic, school test results) can go up or down for no reason.
• Risk = people. Although numbers aren’t fortune-tellers, they still have an astonishing power to put a probability on our fate. Therefore, such numbers must be brought back down in line with human experience. For example, when you hear of the higher risk of getting cancer if you eat bacon, drink alcohol, etc., ask about the baseline risk because percentage changes completely depend upon where you start. We should also know about the risk’s confidence interval, or estimated error.

On what numbers hide:

• “Average = middle” = muddle. Averages are messy for two reasons. First, they take try to encapsulate the whole of human experience in one number, ignoring the variety – the way the color white averages the colors of a rainbow. An example is tax-cuts that may benefit the “average” taxpayer. The problem is that in the U.S., the rich are so very rich that the average taxpayer is quite rich, so the tax-cut actually wouldn’t benefit most people. Second, averages are usually interpreted to mean “typical” when they might be quite atypical. For example, nearly everyone has more than the average number of feet because the number of people with one or no feet is enough to move the average to less than two.
• You can’t see wholes through keyholes. Think of the blind men trying to summarize an elephant: a single measure of a single facet leaves almost everything useful unsaid. This concept explains why targets and performance measurements (education, health care) often struggle: they typically pick only one aspect of performance.
• Easy shocks are easily wrong. Outliers always exist, but are often reported as typical because they make a better story. One example is the calculated temperatures due to global warming. So, when we see such phrases as “could be high as” or “potentially affect,” ask whether this is the most likely situation or the most extreme possibility (therefore, the most unlikely), and then ask how far it is from something more plausible.

On relationships between numbers:

• Thou art not a summer’s day, sorry. When reading about comparisons (e.g., academic performance among rich countries), ask whether the comparison is of like with like.
• This causes that – maybe. Correlation does not equal causation, but whenever we see such figures, we need to wonder what else could cause the effect. For example, girls in single-sex schools may do better than girls in mixed-schools not because of the lack of boys, but because the schools are require a fee.

Despite the problems with numbers, numbers are still are powerful tool of understanding and argument. So, they should be treated seriously. Asked about figures of American life (e.g., number of abortions, legal immigrants per 1,000 U.S. residents), college students are often so wrong that when they find out the correct answer, they change their minds about policy. But even people who should understand the economy have been shown to not have basic economic figures (e.g., income tax paid by the top 1% of earners) at their fingertips. If we don’t know a number, we can make a good estimated guess – but only if we have direct relevant experience to the question being asked. So, we could take a good stab at estimating the number of gas stations in the U.S., but not the number of penguins in Antartica.

I take this statement as the authors’ manifesto:

A culture that respected data, that put proper effort into collecting and interpreting statistical information with care and honesty and an understanding of its limitations, that valued statistic as an aid to understanding, and took pains to find out what was said by the numbers we have already got, that regarded them as something more than a political plaything…[would] be the most valuable improvement to the conduct of government and setting of policy Britain could achieve.

I think these are worthy goals for the U.S. as well, but what could we do in the meantime? Basically learn more about statistics. The authors list a number of books for further reading. One book that particularly interests me is Derek Rowntree’s Statistics without Tears, which is a formal but elementary introduction to statistics. There are also lots of good websites with statistical stories. The authors’ favorites are blogs.wsj.com/numbersguy (“consistently levelheaded”) and statisticsresources.blogspot.com .