It is a matter of concern in many areas of investigation that systematic distortions can arise from the intervention of personal judgements. This issue was raised in the last paper (published posthumously) by WS Gosset (who wrote under the pseudonym ‘Student’), a chemist who rose to a senior position in Guiness’s brewery in Dublin and London (Student 1938). The focus of his paper was his disagreement with RA Fisher about the relative merits of creating experimental groups: Gosset favoured systematic arrangements, such as alternation; Fisher rejected these in favour of random allocation. Gosset did agree one potential advantage of random allocation, however, when he wrote:
…none of the senses can be trusted to behave without bias. … if comparisons are to be made it is necessary to avoid giving the least inkling of the order in which samples are to be presented, and in fact it is better to let it be known that it is a random order.
To plan and execute successfully a convincing study of the efficacy and safety of a medical intervention requires a variety of skills. Particularly it calls for an understanding of the human and scientific aspects of the specific disease or condition involved. There are, however, some general considerations that apply quite broadly.
One concerns the size of study needed to make it likely that any apparent benefit (or disadvantage) reported for the new intervention is genuine and not a consequence of the play of chance. Equally it is bad if an improvement large enough to be of real interest is not decisively detected. Careful use of reasonably well-established statistical methods should achieve these aims.
A more subtle and often more worrying aspect is the possibility of systematic error – bias. If we compare two groups of patients, one group receiving the new intervention and one group receiving a standard or control regimen, it is important that the two groups do not differ systematically in some other respect. Such systematic differences may arise at any phase of the investigation, at patient entry, at intermediate stages, and at the assessment of the final outcome.
To be specific, think of a new patient possibly suitable to be enrolled in a trial. If the physician concerned knows, or thinks he or she knows in advance, the regimen to be received it is possible that borderline choices about suitability will be influenced. Whether this happens consciously or unconsciously is irrelevant. The result may be a seriously biased study. There is ample empirical evidence from many directions that this is not merely an academic possibility but a serious practical concern.
Note particularly that randomization, that is allocation by an independent objective procedure with known probabilistic properties, is on its own not enough. If indeed the choice had been so randomized but was known to the physician to be, say, allocation to the new treatment, the possibility of the bias under discussion would certainly be present. To be effective, concealment of the intended allocation is essential. Moreover, there should be no successful ‘guessing strategy’ open to the physician. This is achieved by randomization plus concealment. For example the relevant randomized choice may be given in a sealed envelope to be opened only after admission of the patient to the study. Because Bradford Hill had recognised the problems resulting from unconcealed allocation schedules when assessing the conduct of an early MRC trial (MRC 1934; Chalmers 2013) he and his colleagues ensured that, in the celebrated Medical Research Council trial of streptomycin for pulmonary tuberculosis (MRC 1948), the allocation schedule was concealed using sealed envelopes.
Random allocation can help to conceal group membership at stages of an investigation following unbiased creation of comparison groups. For example, judgements based on electrocardiographic (ECG) traces of whether a patient should or should not be regarded as having had a myocardial infarction is best done by one or more experts reading the records blinded to the treatment group of the individual patients concerned. Random allocation will help here, as stressed by Gosset (Student 1938). If duplicate assessments are made for some or all of the patients, each expert should also be blind to the assessment of the other expert, again achieved by appropriate concealment. Of course at a later stage it may be sensible to try to settle disagreements over assessment, but again this should be in ignorance of the treatment allocated to the patient.
In a book published in the 1950’s to provide a non-technical discussion about planning experiments (Cox 1958) I wrote about the justification of randomization (Cox 1958, p 70-90). After subsections dealing with the ‘systematic arrangements’ for creating comparison groups which had been favoured by Gosset, and the serious disadvantages of the biases associated with ‘subjective assignment’, Section 5.4 discusses ‘Randomization as a device for concealment’. The introduction to this section has been selected for attention in the James Lind Library (Cox 1958) because it makes clear that concealment of allocation schedules is needed for bias elimination: schedules based on random allocation can be more successfully concealed than those based on ‘systematic arrangements’. Thus, in situations in which “uncontrolled variation arises from subjective effects due to personal biases of the people taking part in the experiments, including the experimenter himself… randomization achieves its aim by concealing from the persons involved which treatment is applied to each unit.” (Cox 1958, p 79)
Although there are statistical reasons for random allocation in experiments (systematic justification of standard errors, and exact tests of significance and associated confidence limits), the most important function of random allocation is to help achieve concealment of an unbiased allocation schedule and thus avoid selection bias in the creation of comparison groups.
This James Lind Library commentary has been republished in the Journal of the Royal Society of Medicine 2010;103:72-73. Print PDF
References
Chalmers I (2013). UK Medical Research Council and multicentre clinical trials: from a damning report to international recognition. JLL Bulletin: Commentaries on the history of treatment evaluation (https://www.jameslindlibrary.org/articles/uk-medical-research-council-and-multicentre-clinical-trials-from-a-damning-report-to-international-recognition/).
Cox DR (1958). Planning of experiments. New York: Wiley.
Medical Research Council (1934). The serum treatment of lobar pneumonia. BMJ 1:241-45.
Medical Research Council (1948). Streptomycin treatment of pulmonary tuberculosis: BMJ 2:769-82.
Student (1938). Comparison between balanced and random arrangements of field plots. Biometrika 29:365-378.
Cox DR (2009). Randomization for concealment.
© David Cox, Nuffield College, New Road, Oxford OX1 1NF UK. Email: david.cox@nuffield.ox.ac.uk
Cite as: Cox DR (2009). Randomization for concealment. JLL Bulletin: Commentaries on the history of treatment evaluation (https://www.jameslindlibrary.org/articles/randomization-for-concealment/)
It is a matter of concern in many areas of investigation that systematic distortions can arise from the intervention of personal judgements. This issue was raised in the last paper (published posthumously) by WS Gosset (who wrote under the pseudonym ‘Student’), a chemist who rose to a senior position in Guiness’s brewery in Dublin and London (Student 1938). The focus of his paper was his disagreement with RA Fisher about the relative merits of creating experimental groups: Gosset favoured systematic arrangements, such as alternation; Fisher rejected these in favour of random allocation. Gosset did agree one potential advantage of random allocation, however, when he wrote:
To plan and execute successfully a convincing study of the efficacy and safety of a medical intervention requires a variety of skills. Particularly it calls for an understanding of the human and scientific aspects of the specific disease or condition involved. There are, however, some general considerations that apply quite broadly.
One concerns the size of study needed to make it likely that any apparent benefit (or disadvantage) reported for the new intervention is genuine and not a consequence of the play of chance. Equally it is bad if an improvement large enough to be of real interest is not decisively detected. Careful use of reasonably well-established statistical methods should achieve these aims.
A more subtle and often more worrying aspect is the possibility of systematic error – bias. If we compare two groups of patients, one group receiving the new intervention and one group receiving a standard or control regimen, it is important that the two groups do not differ systematically in some other respect. Such systematic differences may arise at any phase of the investigation, at patient entry, at intermediate stages, and at the assessment of the final outcome.
To be specific, think of a new patient possibly suitable to be enrolled in a trial. If the physician concerned knows, or thinks he or she knows in advance, the regimen to be received it is possible that borderline choices about suitability will be influenced. Whether this happens consciously or unconsciously is irrelevant. The result may be a seriously biased study. There is ample empirical evidence from many directions that this is not merely an academic possibility but a serious practical concern.
Note particularly that randomization, that is allocation by an independent objective procedure with known probabilistic properties, is on its own not enough. If indeed the choice had been so randomized but was known to the physician to be, say, allocation to the new treatment, the possibility of the bias under discussion would certainly be present. To be effective, concealment of the intended allocation is essential. Moreover, there should be no successful ‘guessing strategy’ open to the physician. This is achieved by randomization plus concealment. For example the relevant randomized choice may be given in a sealed envelope to be opened only after admission of the patient to the study. Because Bradford Hill had recognised the problems resulting from unconcealed allocation schedules when assessing the conduct of an early MRC trial (MRC 1934; Chalmers 2013) he and his colleagues ensured that, in the celebrated Medical Research Council trial of streptomycin for pulmonary tuberculosis (MRC 1948), the allocation schedule was concealed using sealed envelopes.
Random allocation can help to conceal group membership at stages of an investigation following unbiased creation of comparison groups. For example, judgements based on electrocardiographic (ECG) traces of whether a patient should or should not be regarded as having had a myocardial infarction is best done by one or more experts reading the records blinded to the treatment group of the individual patients concerned. Random allocation will help here, as stressed by Gosset (Student 1938). If duplicate assessments are made for some or all of the patients, each expert should also be blind to the assessment of the other expert, again achieved by appropriate concealment. Of course at a later stage it may be sensible to try to settle disagreements over assessment, but again this should be in ignorance of the treatment allocated to the patient.
In a book published in the 1950’s to provide a non-technical discussion about planning experiments (Cox 1958) I wrote about the justification of randomization (Cox 1958, p 70-90). After subsections dealing with the ‘systematic arrangements’ for creating comparison groups which had been favoured by Gosset, and the serious disadvantages of the biases associated with ‘subjective assignment’, Section 5.4 discusses ‘Randomization as a device for concealment’. The introduction to this section has been selected for attention in the James Lind Library (Cox 1958) because it makes clear that concealment of allocation schedules is needed for bias elimination: schedules based on random allocation can be more successfully concealed than those based on ‘systematic arrangements’. Thus, in situations in which “uncontrolled variation arises from subjective effects due to personal biases of the people taking part in the experiments, including the experimenter himself… randomization achieves its aim by concealing from the persons involved which treatment is applied to each unit.” (Cox 1958, p 79)
Although there are statistical reasons for random allocation in experiments (systematic justification of standard errors, and exact tests of significance and associated confidence limits), the most important function of random allocation is to help achieve concealment of an unbiased allocation schedule and thus avoid selection bias in the creation of comparison groups.
This James Lind Library commentary has been republished in the Journal of the Royal Society of Medicine 2010;103:72-73. Print PDF
References
Chalmers I (2013). UK Medical Research Council and multicentre clinical trials: from a damning report to international recognition. JLL Bulletin: Commentaries on the history of treatment evaluation (https://www.jameslindlibrary.org/articles/uk-medical-research-council-and-multicentre-clinical-trials-from-a-damning-report-to-international-recognition/).
Cox DR (1958). Planning of experiments. New York: Wiley.
Medical Research Council (1934). The serum treatment of lobar pneumonia. BMJ 1:241-45.
Medical Research Council (1948). Streptomycin treatment of pulmonary tuberculosis: BMJ 2:769-82.
Student (1938). Comparison between balanced and random arrangements of field plots. Biometrika 29:365-378.