For a feature I’ve just written for Research Magazine I’ve just been chatting to David Spiegelhalter, Professor of the Public Understanding of Risk at the University of Cambridge about how we present statistics. He admits he shouts at the TV when they use statistics that scare or confuse you without helping you.
There are plenty of stats that use percentages or relative likelihoods to compare stuff (before versus after, or this versus that), without really giving us a clue. An example: if you drive 10 miles to by a lottery ticket, you are between 3 and 20 times more likely to die in a car wreck than win the lottery.
The answer to this is not, as my mum pointed out, that you can buy lottery tickets online these days. Comparing the risk of driving (from which, every time you don’t die, you usually get a benefit) and the reward for buying a lottery ticket is like comparing a gun with a gnu because they use the same letters.
The professor would rather we stuck to presenting statistics, where possible, as what would happen to a set of people (10, 100, or for rare events, 1000): for example, according to the Office for National Statistics, for every 1000 people who died in 2008 around 330 died from circulatory (heart) disease – and only five in transport accidents. This might imply that overweight gamblers might be better off walking to buy a lottery ticket than driving. Unless you really, really like living dangerously.
I don’t understand why magazines and newspapers – and marketing departments and think tanks – don’t have a house style on how statistics are presented – for example, insiting that spokespeople qualify “up by 20 per cent” statements with what the expected outcome would be in terms of death, or Euros, or gnus (plus a confidence limit). Newspaper style books have pages about the correct title for a judge and whether you can use aggravate as a synonym for irritate, but I’ve never seen one with instructions on comparative statistics. Maybe it’s because the people who compile style guides know a lot about the meaning of words, but less about the meaning of numbers.
It’s not as if the “for every X people” stat isn’t visual enough. For example, I can give you the interesting (and true) statistic that for every 10 people who come to Talk Normal from a search engine, two have searched for either naked or naked people:
Try this article from Joanna Blythman in The Herald called Scientists must not dictate on public health matters (better leave that job, it seems, to Joanna Blythman). While complaining about Professor David Nutt, she tells us that scientists think their knowledge
is superior to other types of knowledge we might bring to bear on our decisions, such as intuition, experience, observation, or even common sense.
Even when they have used all four, plus scientific method too. She’s a skilled polemicist:
The huffing and puffing of Nutt and his indignant allies has obscured the fact that whatever the rest of society thinks or knows about cannabis…
Note: thinks or knows. As in, if Joanna Blythman thinks something, and has used intuition etc, then she knows it, so it must be better than anything a scientist has boiled up in a laboratory. Especially if she agrees with you.
It doesn’t stop her throwing around a few stats at the end to make her point that the only scientists who know about statistics are the ones who produce statistics she likes. For example:
Now we learn, once again from bona fide scientific research, that pregnant women taking folic acid supplements are up to 30% more likely to produce babies with asthma. Yet still the folic acid lobby is arguing that we should press on regardless with blanket fortification of bread and continue to advocate supplements during pregnancy…
And that, ladies and gentlemen, is how to use statistics to confuse people. Quite apart from the fact that she neglects to point out that the research isn’t from a random sample and shows a weak correlation, that a lack of folic acid causes spina bifida and other problems, we don’t have a chart that shows the effect of this up to 30% as an outcome for 1000 babies born today. We can’t draw one, because so far this research doesn’t tell us enough with enough certainty. On the other hand, we know a lot about the damage caused to babies by poor nutrition during pregnancy.
One of the problems with the presentation of statistics in the press is that you can always slice the results to be more dramatic then they really are, and that suits a speak-your-branes columnist like Blythman. Even journalists who don’t know much about numbers know how to do this. And so I can’t help thinking that in-house standards for newspapers on how they present statistics about are far more important than pages of rules on how to refer to the wife of a marquess or an earl*.
* marchioness and countess, respectively. Pointless as it is, the second one’s good for pub quizzes.
