Evidence based treatment

Reference 1 may have been a Christmas spoof published by the BMJ at the end of 2003, but it seems to have been widely cited since. Maybe the work of disgruntled clinicians having a pop at medical statisticians who were a bit too keen on, perhaps a bit too solemn and serious about the importance of double blind trials with placebos.

The statistical idea of a proper trial of a new treatment for some unpleasant disease goes as follows. Get a large, representative sample of people with the disease. Collect lots of data about those people. Split them, on a random basis, into two groups. One group is given the new treatment and the other is given a placebo, with the ‘double blind’ bit meaning that no-one knows who is in which group, in particular the people with the disease and the people running the trial. After some years, collect the outcomes and analyse all the data. The new treatment is generally good if treatment outcomes are generally better than placebo outcomes and if neither good outcomes nor bad outcomes with the treatment are restricted to some particular part of the sample.

Making due allowance for the fact the response of individuals to a treatment is going to vary, sometimes quite a lot. Some people get on well with treatment A, while others get on well with treatment B – and one may not have the time, energy or knowledge to find out exactly why.

Such proper trials are contrasted with observational, not so say anecdotal data. The stuff of old wives’ tales.

The paper explains, observing due form, that no such proper trials have been conducted in the case of parachutes, use of which has been strongly promoted by the industrial-military complex which makes them, and goes on to draw the appropriate conclusions.

Due form includes sentences like ‘we chose the Mantel-Haenszel test to assess heterogeneity, and sensitivity and subgroup analyses and fixed effects weighted regression techniques to explore causes of heterogeneity’, with the Mantel-Haenszel test existing to the extent of rating a Wikipedia entry, which suggests it might well be used in this general area. No idea whether it is actually appropriate in this particular case.

As well as being a bit of fun, there are serious points and questions here about when proper trials are either not appropriate or not possible. So, for example, at what point in the testing of a break through drug for the treatment of some unpleasant disease, does it become wrong to withhold treatment, to keep people on the placebos?

Who would have thought that a solemn and serious journal like the BMJ would carry such a thing? Not that I know it to be solemn and serious, just that from its front cover it used to look as if it might be.

Perhaps this paper is used as a teaching aid in medical schools. It certainly should be!

PS: my attention was drawn to it by Marco Delogu, the director of the Italian Cultural Institute in London, an artist by trade, of whom more in due course.

Reference 1: Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomized controlled trials – Smith and Pell – 2003.

Reference 2: http://journals.sagepub.com/doi/pdf/10.1177/154510970400300401. An open access copy.

Reference 3: https://en.wikipedia.org/wiki/Cochran–Mantel–Haenszel_statistics.

Group search key: bga.