Fair tests of treatments in health care

Over the past half century, health care has had a substantial impact on people’s chances of living longer, and being free of serious health problems. It has been estimated that health care has been responsible for between a third and a half of the increase in life expectancy and an average of five additional years free of chronic health problems (Bunker et al. 1994). Even so, the public could have obtained - and still could obtain - far better value for the very substantial resources invested in research intended to improve health. Furthermore, some of the treatment disasters of the past could have been prevented, and others could be prevented in future.

The James Lind Library has been created to improve general understanding of fair tests of treatments in health care, and how these have evolved over time.

Misleading claims about the effects of treatments are common, so all of us should understand how valid claims about the effects of treatments are made. Without this knowledge, we risk concluding that useless treatments are helpful, or that helpful treatments are useless.

Fair tests of treatment are tests that take steps to obtain reliable information about treatment effects by reducing the misleading influences of biases and the play of chance. When the need for fair tests of treatments is ignored, people suffer and die unnecessarily.

The explanatory essays in The James Lind Library have been written to promote wider understanding of why fair tests of treatments are needed, and what they have come to consist of. You can access each essay by clicking on the underlined words, below, or you can select them from the Contents screen. If you want to download all of the essays, so that they can be printed out together for reading off screen, click here.

Fair tests are needed because there are many examples of people being inadvertently harmed when treatment decisions do not take account of reliable evidence.

The principles of fair tests have been evolving for at least a millennium - and they continue to evolve today.

Comparisons are essential to address genuine uncertainties about treatment effects. Fair treatment comparisons must avoid biases, whether from differences between the people compared or differences in the way treatment outcomes are assessed. Reliable identification of unanticipated effects of treatments poses particular challenges.

Interpreting unbiased comparisons is not always straightforward. Effects of treatment are sometimes overlooked because there are differences between the treatments intended and the treatments received. The play of chance can be misleading too.

Fair tests of treatments must take account of all the relevant evidence. Preparing systematic reviews of all the relevant evidence entails minimising the impact of biased reporting and biased selection from the available evidence. A statistical process called meta-analysis may help avoid being misled by the play of chance in systematic reviews.

Up-to-date systematic reviews of all relevant, reliable evidence are needed for fair tests of treatments in health care. Even with up-to-date systematic reviews, however, it's important to be on the lookout for biases and ‘spin’. These can result in separate reviews, which are supposedly addressing the same question, reaching conflicting conclusions.

In summary The James Lind Library contains the following essays:

Why fair tests are needed

Why comparisons are essential

Why comparisons must address genuine uncertainties

Avoiding biased comparisons

Differences between the people compared

Differences in the way treatment outcomes are assessed

Interpreting unbiased comparisons

Differences between intended treatments and treatments actually received

Taking account of the play of chance

Identifying unanticipated effects of treatments

Systematic reviews of all the relevant evidence

Dealing with biased reporting of the available evidence

Avoiding biased selection from the available evidence

Reducing the play of chance using meta-analysis

Up-to-date, systematic reviews of all relevant, reliable evidence

These explanatory essays draw on a wealth of illustrative material in the James Lind Library. This can be accessed by clicking on the underlined links or images in the essays.

The text in these essays may be copied and used for non-commercial purposes on condition that explicit acknowledgement is made to ‘The James Lind Library (www.jameslindlibrary.org)’.

A 100-page book – ‘Testing Treatments: better research for better health care’ – was published by the British Library in 2006 (for reviews see here). Translations have been or are being published in Italian, Arabic, German, Chinese and Japanese. The English text is now being made available without charge (under a Creative Commons Attribution 3.0 Unported Licence).

• Download Testing Treatments: better research for better health care