Evidence and the Real World

Pritpal S Tamber

June 18, 2012

How the randomised controlled trial is preventing us from establishing knowledge fit for the real world

There is a lot of angst around why research evidence takes so long to penetrate routine clinical practice. My view is that it’s a miracle that any of it makes it into practice at all because research-derived information is very much the square peg to clinical practice’s round hole.

The gold standard of experimental clinical research is the randomised, controlled trial (RCT), a study design created to make the research environment look nothing like the real world of the non-compliant, socially-complicated and co-morbid. You likely know all this otherwise you wouldn’t be reading a blog (first) posted on bmj.com but in my experience few do. My favourite example of this blissful ignorance was in a meeting with someone from the Department of Health of England who insisted that the company I was representing had to do what they asked “because it was based on a randomised trial”. That was the full extent of her rationale. I couldn’t help thinking about where my taxes were going.

The limitations of trials are brilliantly laid out in a 2009 article by Shaun Treweek and Merrick Zwarenstein entitled, Making trials matter.

They discuss the tension between ensuring the trial is methodologically robust (internally valid) and of value in the real world (applicability). “An internally valid trial,” they believe, “that has poor applicability… is a lost opportunity to influence clinical practice and healthcare delivery”.

Despite all this, being able to cite published research has a significant influence on clinicians. A friend of mine performs market research for pharmaceutical companies. He’s stood on the anonymous side of a double-sided mirror and watched doctors leaf through marketing material. He says that without fail the presence of a citation to an RCT gets their attention and approval, and is the most likely thing to influence their behaviour.

There’s something crazy about a world in which we know research-derived information bears little resemblance to the real world and yet we continue to generate and cite it to influence behaviour.

There are some big hitters out there trying to get people to see the world as it is, including none other than the Chair of NICE, Michael Rawlins. In his Harveian Oration of 2008 he said: “Randomised controlled trials, long regarded as the ‘gold standard’ of evidence, have been put on an undeserved pedestal… Observational studies are also useful and, with care in the interpretation of the results, can provide an important source of evidence”. I understand he reiterated that point in a lecture to the Office of Health Economics.

The pharmaceutical industry has tried to show leadership in this arena by generating information more closely associated with real-life practice. They call it real world data. Although they still have to generate the usual kind of evidence to illustrate efficacy and safety they’ve recognised that real world data “increasingly plays an important role in ensuring that medicines are accepted by national policy makers and are adopted into practice”.

We often need small children (or Hans Christian Anderson) to point out what’s crazy in this world but luckily for the egos in healthcare we have Richard Bohmer. In his succinct Perspective in the New England Journal of Medicine he describes the four habits of high-value healthcare organisations. The third and fourth are measurement and self-study. In essence, they collect and analyse real world data – round pegs for round holes.

I was lucky enough to hear Bohmer give a presentation and when asked what one thing all organisations need to start doing tomorrow to improve care he said they should start collecting local performance data that local clinicians could identify with and believe in. His view was that research evidence, although important, emerges too slowly and is too abstract to keep up with the demands of real-life practice. Rather than evidence-based medicine, he said, we need to embrace evidence-capturing medicine.

I’m with Treweek, Zwarenstein, Rawlins and Bohmer; how about you?

Competing interests: I helped launch the journal in which Shaun Treweek and Merrick Zwarenstein’s article was published, although I had nothing to do with its peer review, acceptance and publication.

This post was first published on my original blog, Optimising Clinical Knowledge, and co-posted on BMJ Blogs

Pritpal S Tamber

I’m a doctor who trained as a medical editor and publisher and now researches and consults on the link between community power and health equity. My interest in community power started when I was the Physician Editor of TEDMED and is explained in My Perspective. I also work as a freelance medical editor and publisher for organisations that want to write high-quality articles and a strategy for their publishing and promotion. Find out more on my About page.

Leave a Comment

Your email address will not be published. Required fields are marked *

See other articles in this/these project(s):
Community & Health Newsletter
Scroll to Top