Worse than nothing

reliabilityFor a feature I’ve just written for Research Magazine I’ve just been chatting to David Spiegelhalter, Professor of the Public Understanding of Risk at the University of Cambridge about how we present statistics. He admits he shouts at the TV when they use statistics that scare or confuse you without helping you.

There are plenty of stats that use percentages or relative likelihoods to compare stuff (before versus after, or this versus that), without really giving us a clue. An example: if you drive 10 miles to by a lottery ticket, you are between 3 and 20 times more likely to die in a car wreck than win the lottery.

The answer to this is not, as my mum pointed out, that you can buy lottery tickets online these days. Comparing the risk of driving (from which, every time you don’t die, you usually get a benefit) and the reward for buying a lottery ticket is like comparing a gun with a gnu because they use the same letters.

The professor would rather we stuck to presenting statistics, where possible, as what would happen to a set of people (10, 100, or for rare events, 1000): for example, according to the Office for National Statistics, for every 1000 people who died in 2008 around 330 died from circulatory (heart) disease – and only five in transport accidents. This might imply that overweight gamblers might be better off walking to buy a lottery ticket than driving. Unless you really, really like living dangerously.

I don’t understand why magazines and newspapers – and marketing departments and think tanks – don’t have a house style on how statistics are presented – for example, insiting that spokespeople qualify “up by 20 per cent” statements with what the expected outcome would be in terms of death, or Euros, or gnus (plus a confidence limit). Newspaper style books have pages about the correct title for a judge and whether you can use aggravate as a synonym for irritate, but I’ve never seen one with instructions on comparative statistics. Maybe it’s because the people who compile style guides know a lot about the meaning of words, but less about the meaning of numbers.

It’s not as if the “for every X people” stat isn’t visual enough. For example, I can give you the interesting (and true) statistic that for every 10 people who come to Talk Normal from a search engine, two have searched for either naked or naked people:

Two from ten

Try this article from Joanna Blythman in The Herald called Scientists must not dictate on public health matters (better leave that job, it seems, to Joanna Blythman). While complaining about Professor David Nutt, she tells us that scientists think their knowledge

is superior to other types of knowledge we might bring to bear on our decisions, such as intuition, experience, observation, or even common sense.

Even when they have used all four, plus scientific method too. She’s a skilled polemicist:

The huffing and puffing of Nutt and his indignant allies has obscured the fact that whatever the rest of society thinks or knows about cannabis…

Note: thinks or knows. As in, if Joanna Blythman thinks something, and has used intuition etc, then she knows it, so it must be better than anything a scientist has boiled up in a laboratory. Especially if she agrees with you.

It doesn’t stop her throwing around a few stats at the end to make her point that the only scientists who know about statistics are the ones who produce statistics she likes. For example:

Now we learn, once again from bona fide scientific research, that pregnant women taking folic acid supplements are up to 30% more likely to produce babies with asthma. Yet still the folic acid lobby is arguing that we should press on regardless with blanket fortification of bread and continue to advocate supplements during pregnancy…

And that, ladies and gentlemen, is how to use statistics to confuse people. Quite apart from the fact that she neglects to point out that the research isn’t from a random sample and shows a weak correlation, that a lack of folic acid causes spina bifida and other problems, we don’t have a chart that shows the effect of this up to 30% as an outcome for 1000 babies born today. We can’t draw one, because so far this research doesn’t tell us enough with enough certainty. On the other hand, we know a lot about the damage caused to babies by poor nutrition during pregnancy.

One of the problems with the presentation of statistics in the press is that you can always slice the results to be more dramatic then they really are, and that suits a speak-your-branes columnist like Blythman. Even journalists who don’t know much about numbers know how to do this. And so I can’t help thinking that in-house standards for newspapers on how they present statistics about are far more important than pages of rules on how to refer to the wife of a marquess or an earl*.

* marchioness and countess, respectively. Pointless as it is, the second one’s good for pub quizzes.

5 Responses to “Worse than nothing”

  1. 1 Manek Dubash November 10, 2009 at 9:19 pm

    Words can’t express how much I agree with you, Tim. Or the impotent fury I feel at articles like Blythman’s, who clearly has no fucking idea how science works but is content to segue her intuition into what she ‘knows’ and peddle it as fact.

    No wonder this country is going to the dogs/doing better than ever/has a severe case of dumb-shit fever. Am I going crazy or is it everyone else?

  2. 2 Tim Phillips November 11, 2009 at 4:22 pm

    Don’t worry Manek, I had a look round, and it’s everyone else that’s crazy.

    I’ve remembered that I once saw JB speak in a debate and I was bouncing about like a pea on a drum because there were so many assertions dressed up as facts. So she may be a serial offender in this regard. That’s just my opinion, I’ve got no proof – but as this opinion is based in Blythmanic intuition, experience, observation and common sense, it might be even better than proof.

  3. 3 Chris Long November 18, 2009 at 6:03 pm

    It bothers me that these columnists are backed into this ‘talk bollocks’ corner by virtue of having to fill the page. I have no idea if she is an arse or not, but from the what she says she sounds like someone building a column that needs to tick certain commercial check boxes – conservative-controversial (sort of playing to a stereotypical Daily Mail audience), polarise an imagined zeitgeist (taking sides on a current story) and of course filing 800 to 1000 words to enable the ‘check’s in the post’ moment.

    It’s like the ‘guns don’t kill people, people do’ idea. You have an itch, you grab a gun. You have a thousand words to file; you sculpt some stats to prove your point.

    Despite these columns simply being part our show biz culture they end up driving opinion, we may just as well have Bruce Forsythe read the news.

    On the other hand I wonder if younger people get so apoplectic at this crude opportunism, I can only feel that they don’t, did we get so mad when we were young and columnists et al spouted nonsense?

  4. 4 Tim Phillips November 19, 2009 at 11:24 am

    If you look back even 10 years in the papers, there’s a fraction of the comment that there is now. Just as much was hateful nonsense as far as I can see, but it was a smaller proportion of the whole. And a columnist who upsets no one is no use at all.

    But in this case I’m worried by her creepy idea that reason and research can’t be right if it tells us things we don’t already believe, because there’s literally no way to argue against it, and so everything just becomes “well that’s your opinion you’re wrong”. It’s not attacking an argument, it’s trying to undermine the stuff that we use to make arguments.

    But you’re bang on, it’s a guaranteed 800 words every time (as we both know).

    Don’t know about you, but I used to get absolutely furious at hack columnists like Woodrow Wyatt (“the Voice of Reason”) and John Junor when I was an irritating kid. Partly to piss off my dad, of course.

  5. 5 Chris Long November 20, 2009 at 1:30 pm

    You are right, of course, not only did we get just as wound up by the ‘look at me’ idiots producing this scum, there is more of this scum around – and more and more mixed in today’s news.

    I suppose what I wonder is, does she really mean it?

    In other words are these columnists so devoid of morals or self respect or social decency or scruples (as you can see I’m having trouble defining it) that they just say these things for the crac.

    Can this woman really beleve that something ‘proven’ can be undermined by thinking it isn’t? My suspicion is that she doesn’t, and doesn’t care if we know it.

    I wonder if we aren’t, at last, seeing the measurable effect of the showbiz-isation of our society. We can no longer tell what is a fact or a thought because the people expressing them are too lazy or irresponsible to decide themselves.

    A couple of weeks ago various people I met told me about a personally disparaging script line from an item I was in. They asked ‘did I know they were going to say it?’ Yes, I said, I wrote it. These people work in TV; I wonder if they can tell the difference between the news and eastenders.

    What I don’t get is how we got to this point where the industry of paying people to say stupid and inflamitory things has got so big it actually warps reality itself.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Cut out your waffle: buy my book

Type your email and click the button and you will automatically get every new post.

“This excellent collection” (Director Magazine). Click to order:

I tweet