Interpreting unbiased comparisons:

Differences between intended treatments and treatments actually received

Fair tests of medical treatments have to be planned carefully. The documents setting out these plans are referred to as protocols, and, among other things, they specify details about the treatments that will be compared. The best laid plans don’t always work out quite as intended, however. The treatments actually received by patients in tests sometimes differ from those it was intended they should have received. These departures from intention need to be taken into account in interpreting the results of treatment comparisons.

One of the reasons that placebos were introduced in the evolution of fair tests of medical treatments was to reduce departures from intended treatments (Kaptchuk 1998). But things may go astray even in placebo controlled trials. During the 2^nd World War, people suffering from colds were given a solution of drug called patulin and compared with other people given only the fluid in which the drug had been dissolved (MRC 1944). Analysis of the results failed to reveal any beneficial effects of the drug, but then a concern emerged that the liquid used to dissolve the drug might have inactivated it. In other words, over 1000 patients might have participated in a comparison of two inactive treatments! Fortunately, tests confirmed that the patulin used in the trial had indeed been active, although it had no detectable effects on colds (Chalmers and Clarke 2004)!

Treatments received may differ from treatments intended for a variety of reasons. For example, doctors may decide that the treatment to which some of their patients have been allocated in a formal treatment comparison should not be offered to them; patients may reject the treatments allocated to them, or not take them as intended; doses of the treatment different from those intended may be given; or the supply of one of the treatments may run out.

For example, when differences emerged in the results of apparently identical treatments for leukaemia in British and American children, investigation revealed that the worse results in Britain reflected unwillingness among British clinicians to persist with chemotherapy when nasty toxic effects of treatment developed (Medical Research Council Working Party on Leukaemia in Children 1986).

For these reasons, interpretations of fair tests must consider the possibility that treatments received were not those intended. If discrepancies between intention and practice have occurred, it is important to consider the implications for interpreting the evidence.

References

Chalmers I, Clarke M (2004). The 1944 Patulin Trial: the first properly controlled multicentre trial conducted under the aegis of the British Medical Research Council. International Journal of Epidemiology 32:253-260.

Kaptchuk TJ (1998). Intentional ignorance: a history of blind assessment and placebo controls in medicine. Bulletin of the History of Medicine 72:389-433.

Medical Research Council (1944). Clinical trial of patulin in the common cold. Lancet 2:373-375.

Medical Research Council Working Party on Leukaemia in Children (1986). Improvement in treatment for children with acute lymphoblastic leukaemia. Lancet 1:408-11.