I have been intrigued for a while by the way particle physicists seem to talk about statistical certainty in terms of how many “sigmas” they have reached. The more sigmas, the more certain you are that what you observed really is true and not just coincidence. I have to say that I don’t find this a useful way of explaining (un)certainty because it relates everything to a Normal distribution, something that many people don’t need to or want to learn about, and in some statistical analyses, it is completely irrelevant. But today in the BBC coverage of the Higgs announcement, I noticed something else: they had started to relate sigmas to p-values and thence to the equivalent number of coins that you might toss and find them all coming up heads. This sounded rather more intuitive to me, so I present below a little conversion table for anyone who might find it useful.
|How many coins?||p-value||One in…||Sigmas|
Between 4 and 5 coin tosses lies 1.96 sigma, which is the traditional cutoff for “statistical significance” (p=0.05, or 1 in 20). At 24 coin tosses, we start to get results which are less likely than buying one UK national lottery ticket and winning the jackpot (5.39 sigma, or 1 in 13,983,816).
There are two crucially important concepts for the newcomer to stats: we can never reach complete certainty (proof), and just because something is statistically significant does not mean it is important – that requires insight into the context of whatever you are analysing!