Harvey Goldstein of Bristol Uni, a legend in the world of promoting good statistical practice for league tables (by my own experience and others’, a small and demotivating world) , has written a blog post through the LSE on whose role it is to inform the public on how to access, use and critique the great mass of data published by governmental and official agencies. There clearly is a problem here because hardly anyone goes to look at the data (as opposed to the often poorly digested versions in the media), unless they already have a degree in stats, social sciences, vel sim.
Prof Goldstein lays the responsibility firmly at the door of “academics” and also “the profession”, by which I think he means statisticians. So that’s me and a few thousand others around the UK, although blogs have an international reach because a lot of the data released is very similar.
I’m not sure we can do it. We’ve been banging on about it for years, both at the civil servants (who listen politely and then do what they were going to in the first place, but with the imprimatur of having consulted experts) and the public (who can read Nate Silver and Tim Harford books every Christmas and still have no idea where to begin in deciding which of their local schools is best). It needs to be a concerted effort at organisational level. The RSS and other such bodies around the world don’t generally have enough influence or resources to make it happen straight away, but they do get listened to by official agencies, and if we were more outspoken in criticising unhelpful or obfuscatory data releases, we would start to get some traction. Nobody in Whitehall wants their boss to print out the critical press release and slip it into the file marked “performance review”.
Data should be published along with guidance and interpretation. It should be available for consideration at a number of levels, probably through some interactive website. It should be tabular and graphical and textual. That’s what we’ve been doing for years, and that’s what the really good official data producers are doing already. If we see powerful agencies not meeting those same standards of communication, we need to speak out and make a fuss. But there is no use in doing it in isolation.