Lung cancer quality of care map and stats: communicating uncertainty and casemix

I was looking at the new webpage from the Roy Castle Foundation with an interactive map of lung cancer care in the UK. This is quite a nice piece of work! I for one would like to see more communication of public service stats made along these lines. But there are some outstanding questions that I think will come to mind for a lot of readers, and it would be great to see these tackled in the near future by all interactive mappers.

Image

The map is made with the Raphael JavaScript library, and when you select the care provider, some selected stats appear alongside in pie charts (made with pie.js). You can also get a comparison with the “England and Wales benchmark” which is probably the average across the whole country.

Data comes straight from the publicly available Lung Cancer Clinical Audit report, which in my personal opinion (see disclaimer below) is the product of one of the most advanced audit projects we have in the UK. In  particular, they have been very interested in improving communication of results to all sorts of audiences: clinicians, managers, media, patients and the public. Clearly this is paying off, as this sort of product generates interest in a way that you can only dream of if you are still issuing PDF reports tucked away on your website.

Here’s the first of two problems. Casemix is the term we use for the fact that one hospital will see different patients to another, maybe because the local population is different, maybe because they have more specialised services. As the map says “Some cancer Trusts will include a greater proportion of hospitals which treat late stage patients and will therefore appear to have poorer outcomes, due to the more advanced nature of their disease”. Not only outcomes but also processes. Surgery for example will not help some people, while others refuse. We would not expect surgery to be performed on 100% of patients. On the other hand, allowing the hospital to mark the patient as “not eligible” has its own problems of gaming, well documented even in good people with no mendacious intent. (See for example the book “Performance Measurement for Health System Improvement”). What this map would benefit from is at least some accompanying text explaining how each indicator could be affected by this and what they need to bear in mind. Statistical adjustment for casemix is a popular option, although it’s only as good as your model of what affects what, and the quality of data you collected.

Uncertainty is the second problem. What you really want to know is whether your local hospital is any good, or where you should suggest your relative with lung cancer goes. IN other words, you have here some stats about 2011 and you want to project them into 2013/4. The future will be a bit different, of course, but we can use stats to tell us a range within which it is likely to lie if nothing else major changes. Now, this does appear (see pic above) for the routes to diagnosis, although the term “confidence interval” is not explained. Why not do something similar for all the stats? I know it’s not perfect, but it’s better than nothing. Take St George’s for example. There were 173 patients, and 27 (15.6%) had surgery. Going into the future, if nothing major changes, the long-term percentage will probably (95% certain) be between 10.5% and 21.9%, which is quite wide, reflecting the fact that 27 patients is not really enough data to judge the long-term by. The surgery % has apparently dropped 3.5% since the previous audit, which might worry you a bit, but knowing that confidence interval helps put it in perspective: it’s likely to be just normal fluctuation from one year to the next.

Some of the audit statistics are also going to be included in the next NHS Atlas of Variation. I wait to see whether this official Government publication innovates further on the methods used to depict uncertainty.

Disclaimer: I am a member of the National Advisory Group on Clinical Audit and Confidential Enquiries. We advise the Department of Health / NHS England on funding clinical audit projects. This blog post is entirely my own opinion and does not reflect views or policy of NAGCAE, NHS England or HM Government.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s