Chalmers I (2010). Joseph Asbury Bell and the birth of randomized trials.
Email to someoneTweet about this on TwitterShare on FacebookPin on PinterestShare on LinkedIn

© Iain Chalmers, James Lind Library, Summertown Pavilion, Middle Way, Oxford OX2 7LG, UK. E-mail: ichalmers@jameslind.net


Cite as: Chalmers I (2010). Joseph Asbury Bell and the birth of randomized trials. JLL Bulletin: Commentaries on the history of treatment evaluation (http://www.jameslindlibrary.org/articles/joseph-asbury-bell-and-the-birth-of-randomized-trials/)


Introduction

A year after Joseph Asbury Bell’s death in 1968, the American Journal of Epidemiology celebrated his many contributions by publishing a memorial issue of the journal (1969; Vol 90, No.6). The editorial introduction concludes as follows: “We honor Joe Bell for his uncompromising scientific integrity, his productivity as an epidemiologist, his ability to stimulate and guide collaborative research, and for those qualities which made him such a good friend and mentor to many epidemiologists”. Nowhere in the memorial issue of the journal, however, is there any discussion of Bell’s exemplary reports of randomized trials of pertussis (whooping cough) vaccines in the 1940s, the decade during which randomized trials can be said to have been born. This is particularly surprising given that Bell’s reports of his randomized trials were not published in obscure places, but in mainstream journals. Yet, as far as I am aware, none of the many people who have written about the history of randomized trials have referred to the remarkable report that Bell published in 1941, seven years earlier than the now iconic report of the randomized trial of streptomycin in pulmonary tuberculosis conducted under the aegis of the Medical Research Council (MRC 1948).

The MRC streptomycin trial was not the first randomized trial to have been initiated under the Council’s aegis. Between November 1946 and 1950, the MRC conducted a number of randomized trials of pertussis vaccines, and a report on these was published in 1951 (MRC 1951). From both methodological and vaccinological perspectives, these studies were remarkable (Jefferson 2006). I wondered whether the MRC trials had been inspired methodologically by any of the many previous reports cited in the introductory section of the MRC report. This referred to reports of controlled trials done by Doull and his colleagues (1936) in Cleveland, Ohio; Pearl Kendrick and Grace Elderling (1939; 1942; 1943) in Grand Rapids, Michigan; McFarlan, Topley and Fisher (1945) in Oxford; and Bell (1941; 1948a; 1948b) in Virginia. It is probably significant that the MRC report refers to Bell as having “made two well-controlled trials”. ‘Making a trial’ (rather than ‘doing’, or ‘conducting’ a trial), is a form of words used by Austin Bradford Hill, who was principally responsible for designing the MRC whooping cough trials. Bradford Hill would certainly have appreciated those features of Bell’s reports that would justify singling them out for designation as ‘well-controlled’.

Bell’s controlled trials of pertussis vaccines in the 1930s

Bell’s involvement in clinical trials on pertussis vaccines began when he was in his early thirties, while he was Assistant Chief of the Foreign Quarantine and Immigration Division of the US Public Health Service, an Instructor in Epidemiology at the Johns Hopkins School of Hygiene and Public Health, and a member of Dr JP Leake’s staff in the Epidemiology Section of the Division of Infectious Diseases at the National Institute of Health in Bethesda.

The first trial of pertussis vaccine in which Bell was involved began in November 1936 in Cumberland, Maryland. The study was led by WT Harrison, Senior Surgeon, US Public Health Service, who prepared the vaccine, and involved JP Franklin, Maryland’s Deputy State Health Officer, as well as Bell. Children were allocated to a single dose of alum-precipitated pertussis vaccine or to an unvaccinated group based on the initial letters of their family names. The reasons for excluding children after allocation were described in the report, and consideration was given to the possible consequences of some imbalances in the characteristics of the 191 children who remained in the comparison groups. Although the pertussis attack rates in the two groups suggested that the vaccine might confer some protection, the authors judged that the observed differences were “in no way convincing from a statistical standpoint”, but that they justified “more research on a larger scale, using a longer period of antigenic stimulation.”  (Harrison et al 1938).

Accordingly, a study involving nearly a thousand children given two doses of the vaccine 4 weeks apart was begun in April 1938 in Norfolk, Virginia. The study was organised by Bell under the aegis of the US Public Health Service, in co-operation with the Norfolk City Union of King’s Daughters Visiting Nurse Association. The report of this study (Bell 1941) is the earliest methodologically detailed account of a randomized trial of which I am aware. Although a facsimile of the whole of Bell’s report is available on The James Lind Library, certain passages seem to me to be of such importance in the history of controlled trials that I have transcribed them for inclusion in this commentary.

Bell begins his report by setting out the principal challenges presented by the study:

The single question to be answered by this report is whether the vaccine confers any real protection against the disease. Since the public health aspect of the disease is of chief concern, it was necessary that the observed children be representative of the general population. To answer the question it was necessary to have a clinical-epidemiological arrangement whereby a large group of children injected with the vaccine could be uniformly observed together with a group of children not injected with the vaccine but otherwise identical in all attributes which might influence the results. Three major problems were immediately evident: (1) The obtaining of an injected group identical in such attributes with a noninjected group; (2) the uniform observation of injected and noninjected groups over a period of time long enough to give an adequate experience of pertussis; and (3) the definition of pertussis as a clinical entity which, within the limits of observation available, could be uniformly and readily recognized in the injected and noninjected groups.

“The obtaining of an injected group identical with a noninjected group”

In discussing the first of the three problems identified, Bell covers (i) the desirability of studying, as far as possible, representative samples in the population; (ii) the need to study all individuals allocated to the two groups, whether or not they received the intended treatment – the ‘intention-to-treat’ principle; (iii) the use of random sampling numbers to assemble vaccine and control groups; and (iv) a comparison of the characteristics of the children in the two randomized groups generated.

The first problem was that of locating for observation a group of children to be vaccinated, identical, in all attributes, which might influence the occurrence and recognition of pertussis, with another group to receive no vaccine. It is impossible to select such identical groups because many of the attributes involved are not known, and many of those that are known cannot be quantitatively assessed; and, furthermore, even if such attributes could be made identical in the two groups at any one moment, they would not remain identical throughout the time necessary for adequate observation. Some attributes without apparent influence on the results may under certain circumstances be of real importance.

The only practical approach appeared to rest in the selection of two groups, each of which is a random sample of the combined groups in the exact sense of the term. Thus only can the prediction be made that should the vaccine have no real influence on the occurrence of pertussis, the occurrence in each group will approximate that of the combined group, deviating therefrom strictly within the range of chance sampling variation. On the other hand, if the vaccine confers real protection against the disease, or otherwise really influences its occurrence, the occurrence in each group will differ from that of the combined group outside the range of chance sampling variation. Obviously it is not practically possible to preselect two large strictly random groups of children who are representative of the general population and to ensure that every child in one group receives the vaccine while every child in the other group receives no vaccine during the observation period. Children in the general population have the prerogative to refuse vaccine offered and the liberty to obtain other vaccine when desired. In these premises there is no known way of changing the two groups so that one would include only children actually vaccinated, and the other include only children not vaccinated, without destroying the randomness of the selection and to that extent possibly invalidating the answer to the question asked. After it has been established that the vaccine confers protection, then questions concerning the amount and duration of such protection might in part demand direct comparison of the experience of the children actually vaccinated with those not vaccinated, providing adequate data are at hand to equalize the two groups with respect to attributes which apparently influence the occurrence of the disease.

For this report, the approach to the primary problem involved the preselection of two large strictly random groups of children and the subsequent injection of a large proportion of only one group with the vaccine. All analyses herein presented are a comparison of the experience of such preselected groups regardless of their actual status with respect to receiving the vaccine. The difficulties encountered in this approach are chronologically described in detail so that the reader may evaluate any possible errors involved.

During March, April, and May, 1938, a public health nurse transcribed the names of children born between May 1, 1935, and March 31, 1938, who were on the various rolls of the King’s Daughters Visiting Nurse Association. All of the names were not transcribed. Children whose records indicated that they had had prior whooping cough, children who were known to have left the city permanently, and a few of the children of well-to-do parents who the nurse thought, were subject to pertussis vaccination apart from that given in the course of this study, were not transcribed.

The city of Norfolk, adjacent suburbs, and South Norfolk were divided into 14 geographic sections. The definition of boundaries of each section was the result of an endeavor to group people somewhat according to their usual association in schools, churches, theaters, and shopping districts.  The children in each section are either white or colored, no one section having both.

A total of 1,954 names was transcribed, together with information as to sex, birth date, and address of residence, as recorded on the rolls from which they were copied. The names were transcribed in 14 groups according to geographic section of residence. In each group the names were listed in alphabetical order for each year of birth and the years of birth were ordered chronologically. A numbering machine was used to stamp a serial number after each name in the above order. Using the “Random Sampling Numbers” as assembled and published by L.H.C. Tippett, the allotted numbers in each section were divided at random into two equal groups hereinafter designated as the “V” and “N” groups.

When reading these excerpts it is at first unclear what exactly Bell did: whether he sampled his groups independently from each other (two random samples from population), or whether he randomized after he had made up his mind who would be in the trial. In his theoretical preliminaries, he describes the first, but in practice, he appears to have done the second.

Since there was an epidemic of pertussis in Norfolk during the spring and early summer of 1938, an effort was made to have the children selected in the “V” group injected with vaccine at an early date. Hence a search to locate the children selected in the “V” group and get consent for their injection with pertussis vaccine was made in April and May 1938, whereas no search for the “N” group was made until July. The vaccine was offered only to the “V” group and not to the “N” group or to other children. There were very few refusals even though no promises were made as to the effectiveness of the vaccine. Parents giving consent for vaccination were mailed an appointment card to bring the “V” child to a health station for his first dose of vaccine on May 23-25, 1938. The second dose was given 4 weeks later, in a similar manner. Certain “V” children who did not keep their appointments, and others who were ill, or who were temporarily out of the city, or who were not located as of those dates, were offered vaccine at later dates. Eighty-eight percent of the “V” children who received vaccine had their first dose prior to July 3, and practically all the vaccinations accomplished were completed before the end of September 1938.

During the early period of observation when visiting records of all children located were being checked, 129 selected children were found whose names had been transcribed from several rolls and hence each had been assigned two or more numbers, often being selected in both the “V” and “N” groups. Thus a supplementary sampling process was necessary to allocate these children into either the “V” or “N” group in a strictly random manner. To this end the lowest of the numbers assigned to any child became his final number and designated his selected status in either the “V” or the “N” group. Of the 129 children, all numbers of 61 were either in the “V” or in the “N” group and hence their selected status remained unchanged. Of the remainder, 38 were assigned to the “N” group, including 21 who previously received vaccine, and 30 were assigned to the “V” group, of whom 21 subsequently received vaccine.

The above description of the first problem sets forth the practical difficulties encountered in this effort to preselect the names of a large number of children in two strictly random groups, locate the children represented by those names, eliminate the duplicates, secure two groups suitable for adequate observation, and ensure that a large proportion of one group receive alum-precipitated pertussis vaccine prophylactically, while a large proportion of the other group receive no vaccine. Table 1 reveals the result of this effort. It is noted that the two groups completely observed are not equal in number, even though they were originally so selected.  The search for the children in each group was pursued with equal diligence and all available evidence is consistent with the belief that the smaller size of the “N” group of children observed does not disturb the randomness of either group and was due to the later date of search for this group, after school was dismissed and summer vacation in progress.

“Uniform observation of injected and noninjected groups over a period of time long enough to give an adequate experience of pertussis”

Bell stressed that only two, well-trained nurses were engaged to follow-up the children selected for the study, and that precautions had been taken to try to ensure that they were unaware of the group to which each child had been allocated. An independent paediatrician was engaged to examine suspected cases of pertussis, but he was deliberately kept unaware of the precise diagnostic criteria adopted for the study. Bell demonstrated good concordance between the pediatrician’s diagnoses and his own.

The second problem, that of obtaining adequate uniform observation of the injected and noninjected children, was handled by the full-time employment of only two highly capable, sympathetic, and interested public health nurses experienced in communicable disease work in this city. Together they received special training to perfect their uniform approach to the families of the selected children, their uniform use of nonleading questions, their uniform vocabulary for eliciting information desired, and their uniform accuracy in obtaining, evaluating, and recording such information. The nurses expended no little effort in gaining the confidence of the families under their surveillance so as to enhance the amount and trustworthiness of the information elicited. To aid cooperation, the nurses contributed their experienced health teaching service and offered free clinic and hospitalization service for illnesses in children of families unable to afford medical service. They were not officially concerned with the quarantine of communicable disease; however, they endeavored to persuade voluntary isolation of cases during the communicable period.

A visiting record was prepared for each household in which a selected child resided. The nurses divided the records about equally, primarily on the basis of routes convenient for a routine visit once each month to each child, but during the earlier months of observation, time did not permit revisiting within the month children found not at home at the time of the routine monthly visit. It is of interest to note that division of the work on this basis resulted in each nurse having a nearly equal proportion of children in the “V” and ”N” groups and a nearly equal proportion of older and younger children, and also, as would be expected, a disproportionate number of white and colored children.

Soon after the start of monthly surveillance it became evident that adequate observation required that the nurses make repeated weekly and more frequent visits during the course of pertussis infection. To accomplish this the families were requested to call the nurse for other than routine visits whenever anyone living in the household with the selected child was exposed to or had suspicious symptoms of common communicable disease. Other public health nurses in the city cooperated by the daily reporting of all cases of communicable disease coming to their attention. This special effort to effect early visiting of cases was not instituted until after the 1938 pertussis epidemic. Partly as the result of this, 20 percent of the cases of pertussis occurring in the “V” group of children under observation throughout the interval from June l, 1938, to March 30, 1941, did not receive weekly visits before the end of the fourth week following onset of cough, and 17 percent of cases in the “N” group did not receive such early weekly visits. Of the remaining cases, 73 percent occurring in the “V” and 76 percent occurring in the “N” group of children were under weekly or more frequent observation before the end of the second week of cough. This indicates the uniformity with which the “V” and “N” groups were observed and suggests the adequacy of the observation. The nurses did not know which of the children were in the “V” or “N” group and made every reasonable effort to avoid knowing which of the children had received pertussis vaccine. Of course, informants not infrequently would invite the nurses’ attention to children supposed to have received the vaccine, but little credence was given to such information because some parents were obviously confused between pertussis and diphtheria injections.

A consulting pediatrician was employed to examine many of the cases and suspected cases of pertussis and to make a written report especially noting other diseases which might influence severity or obscure diagnosis. He did not know what criteria the author used for the diagnosis of clinical pertussis, and his report was not submitted until the end of the observation period here reviewed; he did not have access to the nurses’ records and his opinions were not made known to them during the course of the study. The author also made at least one visit with the nurse to every case of suspected pertussis, evaluated symptoms elicited, and arrived at a conclusion with respect to diagnosis.  This was done entirely independently of the consulting pediatrician, and, like him, without knowledge of whether the child had had pertussis vaccine or had been selected in the “V” or “N” group.

Although the Pediatrician was at a disadvantage in that he was seldom acquainted with the family visited, and usually made only one visit, and often that visit was not at the optimum time for obtaining the best history upon which to base his opinion as to diagnosis, there was remarkable uniformity between his diagnoses and those independently made by the author, after more detailed and complete information was available. The consultant considered at the time of his visit that 2 “V” and 1 “N” children had pertussis, whereas the author finally classified these children as doubtful cases, that is, not definite clinical pertussis. On the other hand, the consultant considered another “V” child to have a doubtful case, whereas the author finally classified the case as definite.

“Definition of pertussis as a clinical entity which…could be uniformly and readily recognized.”

Bell’s third problem was to define pertussis as a clinical entity which could be “uniformly and readily recognized in the vaccinated and nonvaccinated groups”. He described the minimal criteria for diagnosing a definite case, and made clear that the clinical information obtained had to be sufficiently reliable and complete to establish beyond reasonable doubt that these minimal criteria had been met. Because he recognised that the defining criteria he had set out were “more or less arbitrary”, he arranged for detailed clinical records to be kept for each child, one purpose of which was “to permit eventually an objective decision as to the definition of a case of pertussis, subject to modification as data accumulated.”

The two paragraphs with which Bell concludes his 8-page description of the materials and methods used in his trial show that he was aware of the possibility of recall bias, and that he needed to account for all randomized children and dropouts:

The above discussion of the three major problems involved in this effort to determine whether or not the vaccine confers any real protection against pertussis describes the clinical-epidemiological arrangement whereby certain specific information was accumulated. A history of the past and current experiences of each child with measles, chickenpox, mumps, and pertussis was meticulously sought. It was recorded together with the date of such experience as accurately as this could be reasonably approximated. These data were checked and rechecked during the period of observation, particularly at times when the disease occurred in the neighborhood and the informant’s memory was stimulated thereby. Whenever a history of experience with communicable disease was obtained, every reasonable effort was made to confirm it. This history, in practically every instance, consisted of a detailed description of the disease experience and was further confirmed by consultation with the private or clinic physician whenever one had been in attendance. On each monthly visit to the household the nurse made specific inquiry and record concerning each child with respect to all illnesses, particularly coughs and coryzas, and exposures to communicable disease that occurred during the month since the last visit.

The period of observation covered by this report began June 1, 1938, and ended March 30, 1941, and was approximately the 34 months following the injection of the first dose of vaccine in the “V” group. Table 1 shows that 137 located children had had pertussis prior to this period. These 137 children were dropped from further observation prior to June 1, 1938. No child was dropped on account of an attack of pertussis or any other disease subsequent to June 1, 1938.  All other selected children have been kept under continuous monthly observation since, unless they became lost on account of moving away from the city or dying. As indicated in table 1, 79 of the “V” and 71 of the “N” group were thus lost during the observation period; of these, 1 of the “V” and 7 of the “N” group had an attack of pertussis during the observation period prior to being lost. The experience of these 150 lost children and of the 137 with a previous history of pertussis is included in the first part of table 2 but not included in the other tables, 3 and 4, of this report.

Results and conclusions

In view of the fact that there were only 51 cases of pertussis in the vaccinated group compared with 150 in the unvaccinated group, it is not surprising that Bell (who was clearly statistically literate) felt no need to perform tests of statistical significance.  He looked for possible effect modifiers by performing and presenting subgroup analyses by age, sex, colour, severity of the disease, and even by the two nurse observers, but did not detect any evidence that the overall effect was not reflected in these subgroups.

Bell concludes his report as follows:

Since the “V” and “N” groups of children in this study were strict random samples of the combined groups, since the observation of each group was pursued with equal diligence and uniform criteria were used to enumerate cases, and since the only known difference between the groups was the injection of alum-precipitated pertussis vaccine into a large proportion of the “V” group of children, whereas only a small proportion of the “N” group of children were so injected, it is believed that the vaccine used was responsible for the disproportionately smaller number of cases of pertussis observed in the “V” as compared with the “N” group or with the combined groups. No other conceivable influence could be consistent with these results, operating in each geographic section of the city, in the white as well as in the colored race, in males as well as in females, and in the younger as well as in the older children.

Comment

Bell conducted further trials of pertussis vaccines (combined with diphtheria toxoid) in Norfolk, Virginia during the early 1940s. These, too, were meticulously designed and reported (Bell 1948a; 1948b), and Bell’s Doctorate in Public Health was awarded in 1948 on the basis of this work (Bell 1948c). After war service in Europe between 1943 and 1945, he returned to Bethesda to become Chief of the Epidemiology Section of the Division of Infectious Diseases in the National Microbiological Institute, and was commissioned Medical Director, US Public Health Service in 1949. Bell went on to have a very distinguished career in infectious disease epidemiology (see details in the James Lind Library), and remained a stickler for methodological detail right up to his death in 1968 (Philip et al. 1969).

As the memorial issue of American Journal of Epidemiology makes clear (1969; Vol 90, No.6), Joseph Bell’s many contributions in the 1950s and 1960s have been recognised. But what were the origins of his ‘methodological precociousness’ in the late 1930s?  As will be clear to anyone looking through the records published during the late 1930s and early 1940s on the James Lind Library, Bell’s 1941 report of the controlled trial he designed in 1938 is exceptional.

The year before the epidemiologist Richard Doll died at the age of 92, I asked him whether he had known Joseph Bell, and what he thought of his 1941 report. In his response (letter to IC 24 Aug 2004) he suggested that the study reported by Bell in 1941 was “not quite perfect, but nearly so”. “I say not quite, because efforts were made to trace the controls later than those to trace the children to be vaccinated, with the result that the ‘N’ [control group] was slightly smaller. But I can’t imagine that this seriously upset the balance – certainly not enough to account for the results. We live and learn. No, I’d never heard of him”.

How did Joseph Bell come to be so exceptional? There are few clues from the bibliography of his 1941 paper, which has just two references – one to the report of the earlier study which he had coauthored with Harrison and Franklin (Harrison et al. 1938); the other to Tippett’s random number tables, which had been published a decade before Bell used them to generate the comparison groups in his Norfolk, Virginia trial (Tippett 1927). Nor are his later reports of trials assessing combined pertussis and diphtheria immunization much help (Bell 1948a; 1948b), other than to confirm that he had an exceptional grasp of what was needed to design, run and analyse randomized intervention studies. His grasp of the essentials is made particularly clear in his doctoral thesis (Bell 1948c), which might almost be characterised as a textbook on how to design randomized trials.

Bell’s involvement in pertussis trials began not long after he had attended a 3-month course at the National Institute of Health in 1935. The following year he became a postgraduate student at Johns Hopkins School of Hygiene and Public Health, graduating Master in Public Health in 1937. He was one of a number of innovative ‘trialists’ associated with the Johns Hopkins School of Hygiene in the 1930s. At the beginning of the 1930s, James Doull and his colleagues conducted a clinical trial using an innovative urn device to assign faculty and student volunteers to experimental and control groups (Doull et al. 1931), and later in the decade Doull reported a trial of pertussis immunization which used alternate allocation (Doull et al. 1936). In the mid 1930s, Pearl Kendrick used random selection of controls for a large trial of pertussis vaccine done for the Michigan Department of Health (Kendrick et al. 1939; Marks 2006). In the 1940s, Carroll Palmer was influential in persuading the US Public Health Service to conduct a randomized trial of streptomycin for pulmonary tuberculosis (Long and Ferebee 1950).

What all these people had in common was their contact with Wade Hampton Frost, who taught epidemiology at the Hopkins School from 1919 to 1938. Joseph Bell arrived at the School late in Frost’s life, and he completed all but his initial pertussis studies after Frost had died; but some link with Frost, direct or indirect, seems likely. I hope that this commentary may stimulate some new research, not only on Joseph Bell, but also on the role of the Johns Hopkins School of Hygiene in the early history of randomized trials in the United States.

This James Lind Library commentary has been republished in the Journal of the Royal Society of Medicine 2007;100:287-293. Print PDF

Acknowledgements

I am indebted to Harry Marks for sending me a photocopy of Bell’s doctoral thesis and for providing information about Wade Hampton Frost’s influence at the Johns Hopkins School of Hygiene in the 1930s; and to Harry Marks and Jan Vandenbroucke for helpful comments on an earlier draft of this commentary.

References

Bell JA (1941). Pertussis prophylaxis with two doses of alum-precipitated vaccine. Public Health Reports 56:1535-1546.

Bell JA (1948a). Use of an alum-precipitated mixture of pertussis vaccine and diphtheria toxoid. JAMA 137:1009-1016.

Bell JA (1948b). Use of two doses of an alum-precipitated mixture of diphtheria toxoid and pertussis vaccine. JAMA 137:1276-1281.

Bell JA (1948c). The epidemiological principles and procedures involved in a study of the prophylactic value of an alum-precipitated mixture of diphtheria toxoid and pertussis vaccine. Thesis submitted to the School of Hygiene and Public Health of the Johns Hopkins University in conformity with the requirements for the Degree of Doctor of Public Health, April.

Doull JA, Hardy M, Clark JH, Herman NB (1931). The effect of irradiation with ultra-violet light on the frequency of attacks of upper respiratory disease (common colds). American Journal of Hygiene 13:460-77.

Doull JA, Shibley GS, McClelland JE (1936). Active immunization against whooping cough: interim report of the Cleveland experience. American Journal of Public Health 26:1097-1105.

Harrison WT, Franklin JP, Bell JA (1938).Prophylactic value of a single dose of precipitated pertussis vaccine. Public Health Reports 53:793-796.

Jefferson T (2006). Why the MRC randomised trials of whooping cough (pertussis) vaccines remained important 40 years after they were done. The James Lind Library (www.jameslindlibrary.org).

Kendrick P, Eldering G (1939). A study in active immunization against pertussis.  American Journal of Hygiene 29 Sec. B:133-153.

Kendrick PL (1942). Use of alum-treated pertussis vaccine, and alum-precipitated combined pertussis vaccine and diphtheria toxoid, for active immunization. American Journal of Public Health 32:615-626.

Kendrick PL (1943). Field study of alum-precipitated combined pertussis vaccine and diphtheria toxoid for active immunization.  American Journal of Hygiene 38:193-202.

Leake JP(1969). Joseph Asbury Bell. 1904-1968. A biographical appreciation.American Journal of Epidemiology 90:464-470.

Long ER, Ferebee SH (1950). A controlled investigation of streptomycin treatment in tuberculosis. Public Health Reports 65:1421-1451.

Marks HM (2006). The Kendrick-Eldering-(Frost) pertussis vaccine field trial. The James Lind Library (www.jameslindlibrary.org).

McFarlan AM, Topley E, Fisher M (1945). Trial of whooping-cough vaccine in city and residential nursery groups. BMJ 2:205-208.

Medical Research Council (1948). Streptomycin treatment of pulmonary tuberculosis: a Medical Research Council investigation. BMJ 2:769-782.

Medical Research Council (1951). The prevention of whooping-cough by vaccination. BMJ 1:1463-1471.

Philip RN, Bell JA, Davis DJ, Beem MO, Beigelman PM, Engler JI, Mellin GW, Johnson JH, Lerner AM (1969). Epidemiologic studies on influenza in familial and general population groups, 1951-1956. V. Effectiveness of adjuvant vaccines. Am J Epidemiology 90:471-83.

Tippett LHC (1927). Random sampling numbers.  Tracts for computers, No. 15, edited by Karl Pearson. London: Cambridge University Press.