Just spotted this chart of foodstuffs thanks to NJBiblio. Luckily for me, I love broccoli.
Monthly Archives: March 2013
There’s some detail in there about audiblization / sonification too. That’s an area I’d be really interesting in exploring, maybe convene a working party to come up with guidance and priorities for experimentation, with statisticians, web designers, musicians, psychologists and sound artists all represented round the table. That had better be mission #2.
What an astonishing infographic from Pitch Interactive. Like Periscopic’s gun murders, there are trajectories, it accelerates alarmingly, and on roll-over it displays info on each of the strikes. I tell you what would be fun: let’s get the total cost to the US taxpayer mounting up on the top as well, perhaps represented in little schools and hospital wings of opportunity cost.
I enjoyed reading a news story on the BBC website today about mobile phone location data and how a few regular locations can identify a person. It is certainly true that data do not have to contain your name, address, date of birth or any national ID numbers to identify you on occasion. It doesn’t work for everyone of course, but in large data sets there will always be a few individuals who can be identified after the fact because they are or become famous. I once analysed a large data set of prescriptions in nursing homes. I was suspicious when cleaning the data of the lady in a certain town who was apparently 103 years old, but I didn’t change the age. It stuck in my mind though, so a year or so later when I read that Britain’s oldest person had passed away in a nursing home in that town at the age of 104, I put two and two together. The newspaper gave her name and some of her life story. Without intending to, I had identified one of my participants, not because the data were any less scrupulously anonymised than they should have been, but simply because she became famous.
I am not at all surprised that phone companies are waking up to the money-spinning potential of our location data. After all, they know who each of the trails belongs to, and our credit card details, so if they team up with the credit card company they will have a very attractive data set to marketing people. Not that I think you can actually tell anything about anyone from such data that would help you sell one more widget, but that’s not the point. It’s a data arms race: they don’t care whether the new Big Data is going to make them any more money, they just have to buy it anyway to stop their competitors from getting it, just in case it’s true and the other guys cash in on it. This is where we got to with complex financial products a few years ago…
Today I have updated the website at www.animatedgraphs.co.uk and it is now officially launched, including the new mascot Animated Ant.
It will continue to expand in coming weeks and months. My to-do list is:
- Add a page on 3-D graphics, both things like wireframes that look 3-D and red/blue superimposed images that, when seen through the right glasses, really are 3-D
- Add a page on maps (particularly Google maps)
- Add advice on SPSS, which has no shell command to call ffmpeg from inside, but a batch file can run both the SPSS syntax and ffmpeg.
- Get to grips with Excel VBA. There are a lot of people who would use Excel but not any stats software (they are mistaken of course, but let’s cross that bridge later…)
Suggestions and tasks for me to add to the list are very welcome!
Statistical methods for real life, the carpentry analogy again, and Franny & Zooey: a response to Andrew Gelman
Andrew Gelman has been writing on his excellent blog about how it is the constraint and the unexpected inspiration of real-life, tricky, dirty data problems that lead us to make useful new methods in stats (and probably other methodological fields too).
There is a lot to learn from in this post. The motivation for making new methods is important to their success:
We weren’t trying to shave a half a point off the predictive error … we were attacking new problems that we couldn’t solve in any reasonable way using existing methods
and he goes on to describe a situation where his published maps were shown to be faulty and criticised publicly. Far from shrinking embarrassed back into the ivory tower or explaining it away under a lot of esoteric jargon, he improved them and got a better quality result at the end of the process; that quality is the only thing that matters, not the statistician’s ego.
I was also struck by the mention of caring about the results. This is the central issue to me, more important than whether the inspiration or the exemplar data are real or artificial, current or historical, theoretical or practical. It is because we know how important it will be for future carpenters to construct better homes, that we work hard to make a better tool for the job, even if it is a struggle to get anyone to recognise the value of the new tool. (Last year I met Stef van Buuren at a conference and he told me it took 4 years to get any journal editors and reviewers to take fully conditional specification multiple imputation seriously – now it is everywhere.) Of course it is easier to care about real-life applications that make the world a better place, but Prof Gelman knew the implications of his 8 schools problem so well he cared about it too and that shows through.
In response to Gelman’s reference to Watership Down, I will give you the nugget of gold at the heart of JD Salinger’s underrated Franny & Zooey. Franny has come home from college – in what might have been termed, in the upper East side circa 1950, a blue funk – at what simpletons and dullards her professors turned out to be. All they want is to score points off one another and put the students down. Nobody really cares about literature, which makes her retreat to the sofa under a blanket and look for some higher meaning in the family’s eclectic spiritual and philosophical book collection. Nothing works until her brother Zooey gets frustrated by her high expectations and tells her about when he and their late brother Seymour were regular contestants on a radio quiz show for children. Even when Zooey was feeling rebellious, Seymour insisted they dress up formally and shine their shoes, though not even the audience in the studio could see their shoes.
He said to shine them anyway. He said to shine them for the Fat Lady. I didn’t know what the hell he was talking about, but he had a very Seymour look on his face, and so I did it. He never did tell me who the Fat Lady was, but I shined my shoes for the Fat Lady every time I ever went on the air again … This terribly clear picture of the Fat Lady formed in my mind. I had her sitting on this porch all day, swatting flies, with her radio going full-blast from morning till night. I figured the heat was terrible, and she probably had cancer, and – I don’t know. Anyway, it seemed goddam clear why Seymour wanted me to shine my shoes… but I’ll tell you a terrible secret – are you listening to me? There isn’t anybody out there who isn’t Seymour’s Fat Lady. [Penguin Books 1957]
See, what you can do with data is a valuable thing, and you might not have much time in which to do it, so I don’t have any time to spare for people who choose to waste their energies entering a data mining competition to see who can best predict next week’s NASDAQ, or crunching numbers for a bookmakers or any other parasites upon humanity. The process of making the tool that someone else will use to make the world better is its own reward.
If you crunch numbers, then, like me, I expect you have at some point put a lot of work into a project, got very near the end (publication, sending off the report, whatever) and then found a small error, maybe in your numbers or maybe in the wording and interpretation that accompanies them. Maybe you just found something that made you worry that there might be errors lurking somewhere. Yet, you knew that your non-statistician colleagues would never know, and also that you were extremely tired and just wanted to go home and have a beer. This, then, is the statistical litmus test: I can honestly tell you I have always dragged myself back to the task, sometimes unpicking all my work just to find there was nothing wrong after all. I do have the thought of leaving it and not saying anything – you’d not be human if you didn’t – but I have always gone back. I guess that is because I have always had the pleasure of working on material I care about. If you wake up tomorrow and find you don’t care, that you might fail the litmus test, do yourself a favour and get out of that job – immediately – because we need you to care and the clock is ticking.
Oh wonderful Isotype! What a different world it was, with all those inventive designers and artists beavering away in the service of Mankind, or perhaps just the British Empire. Thanks to Robert Kosara’s blog Eager Eyes, which has led me to Michael Stoll’s lovely Flickr collection of graphics such as this:
You have to admit that is informative, eye-catching and attractive all in one. Only now with the benefit of 21st century living can we react with some horror at the thought of breakfast in London, lunch in New York and dinner in Chicago. Imagine the hours spent waiting and queueing and feeling slightly sick!
Some of the Isotype images have not aged as gracefully – it was a different time: