Quantifying uncertainty

Chance may affect the results of a study if too few outcomes have been observed to yield reliable estimates of treatment effects. Small studies in which few outcome events occur are usually not informative and the results are sometimes seriously misleading.


JLL Essay
3.2 Quantifying uncertainty in treatment comparisons

Assessing the role that chance may have played in fair tests

 

FILTER CLEAR FILTERS

SORT date author


FILTER RECORDS BY


Lavoisier A-L de (1784)
Undated documents contained in: Mémoires de Lavoisier, Oeuvres, Tome III. Paris: Imprimerie Impériale, 1865. p 509.

View

de Laplace P-S (1820)
Théorie analytique des probabilités [Analytic theory of probabilities]. Oeuvres complètes 7 (3ème édition). Paris:Courcier, page lxxvii.

View

Louis PCA (1835)
Recherches sur les effets de la saignée dans quelques maladies inflammatoires et sur l'action de l'émétique et des vésicatoires dans la pneumonie [Research on the effects of bloodletting in some inflammatory illnesses and on the action of emetics and blistering in pneumonia]. Paris: Librairie de l'Académie royale de médecine.

View

Gavarret LDJ (1840)
Principes généraux de statistique médicale: ou développement des régles qui doivent présider à son emploi [General principles of medical statistics: or the development of rules that must govern their use]. Paris: Bechet jeune & Labé.

View

Bartlett E (1844)
An essay on the philosophy of medical science. Philadelphia: Lea and Blanchard.

View

Balfour TG (1854)
Quoted in West C. Lectures on the Diseases of Infancy and Childhood. London, Longman, Brown, Green and Longmans, p 600.

View

Schweig G (1854)
Auseinandersetzung der statistischen Methode [Deliberation about the statistical method]. Archiv fὔr physiologische Helkunde 13:305-355.

View

Treatment Committee of the Medical Council (1855)
Report on the results of the different methods of treatment pursued in epidemic cholera. [Metropolitan Report and Report on the Provinces throughout England and Scotland]. Report to the General Board of Health. London: Her Majesty’s Stationery Office.

View

Airy GB (1861)
On the algebraical and numerical theory of errors of observations and the combination of observations. Cambridge: MacMillan and Co.

View

Jürgensen T (1866)
Klinische Studien über die Behandlung des Abdominalytyphus mittelst des kalten wassers [Clinical studes on the treatment of abdominal typhus with cold water]. Leipzig, Vogel, pviii.

View

Fick A (1866)
Die Medicinische Physik [Medical physics]. Braunschweig,Vieweg, pp 430-447.

View

Jessen W (1867)
Zur analytischen statistik [On analytic statistics]. Zeitschrift for biologie 3:128-136.

View

Bain A (1870)
Logic. London: Longmans, Green, Reader & Dyer p 362.

View

Liebermeister C (1877)
Ueber Wahrscheinlichkeitsrechnung in Anwendung auf therapeutische Statistik [On the calculus of probabilities applied to therapeutic statistics]. In: Volkmann R (ed), Sammlung Klinischer Vortrȁge, Innere Medicin, No 39, pp 935-962.

View

Ephraim A (1893)
Uber die Bedeutung de statistischen Methode für die Medicin [On the relevance of the statistical method for medicine]. Volkmann’s Sammlung Klinische Vortraege N.F. Innere Medicin 24:706-716. Leipzig: Breitkopf and Härtel.

View

Heiberg P (1897)
Studier over den statistiske undersøgelsesmetode som hjælpemiddel ved terapeutiske undersøgelser [Studies on the statistical study design as an aid in therapeutic trials]. Bibliotek for Læger 89:1-40.

View

Davenport CB (1899)
Statistical methods, with special reference to biological variation. New York: John Wiley & Sons.

View

Pearson K (1904)
Report on certain enteric fever inoculation statistics. BMJ 3:1243-1246.

View

Alt K (1909)
Behandlungsversuche mit Arsenophenylglyzin bei Paralytikern [Treatment experiments with arsenophenylglycine in paralytics]. Muenchener Medizinische Wochenschrift 56:1457-1459.

View

Advisory Committee on Plague Investigations in India (1912)
The serum treatment of human plague. Journal of Hygiene. Plague Supplement II, LVI:326-39.

View

Brunt D (1917)
The Combination of Observations. Cambridge University Press.

View

Pearl R (1919)
A statistical discussion of the relative efficacy of different methods of treating pneumonia. Archives of Internal Medicine 24:398-403.

View

Acton HW (1920)
Researches on the treatment of benign tertian fever. Lancet 1:1257-1261.

View

Locke EA (1924)
The serologic treatment of lobar pneumonia. Boston Medical and Surgical Journal 190:196-203.

View

Ferry NS, Gordon EJ, Munro FW, Steele AH, Fisher LW (1928)
Clinical results with measles streptococcus toxin and antitoxin. JAMA 91:1277-1280.

View

Park WH, Bullowa JGM, Rosenbluth NM (1928)
The treatment of lobar pneumonia with refined specific antibacterial serum. JAMA 91:1503-1508.

View

Bullowa JGM (1928)
The control. Contribution to a symposium on the use of antipneumococcic refined serum in lobar pneumonia, 15 December 1927. Bulletin of the New York Academy of Sciences 4:339-343.

View

Bullowa JGM (1928)
Use of antipneumococcic refined serum in lobar pneumonia: data necessary for a comparison between cases treated with serum and cases not so treated, and the importance of a significant control series of cases. JAMA 90: 1354-1358.

View

Bullowa JGM (1929)
The serum treatment and its evaluation in lobar pneumonia. Bulletin of the New York Academy of Medicine 5:328-362.

View

Woods HM, Russell WT (1931)
An introduction to medical statistics. London: Staples Press.

View

Ellison JB (1932)
Intensive vitamin therapy in measles. BMJ 2:708-711.

View

Greenwood M (1934)
Epidemics and crowd-diseases. Oxford: Oxford University Press.

View

Hicks EP, Diwan Chand S (1935)
The relative clinical efficacy of totaquina and quinine. In: Records of the Malaria Survey of India 5:39-50.

View

Gilliam AG, Onstott RH (1936)
Results of field studies with poliomyelitis vaccine. American Journal of Public Health 26:113-118.

View

Theobald GW (1937)
Effect of calcium and vitamin A and D on incidence of pregnancy toxaemia. Lancet 2:1397-1399.

View

Bullowa JGM (1937)
Serum therapy. In: Bullowa JGM. The management of the pneumonias. New York: Oxford University Press, pp 283-298.

View

Aykroyd WR, Krishnan RSBG (1938)
Effect of calcium lactate on children in a nursery school. Lancet 2:153-155.

View

Kendrick P, Eldering G (1939)
A study in active immunization against pertussis. American Journal of Hygiene 29 Sec. B:133-153.

View

Price MR (1940)
Effects of a supplement of vitamin B (adsorbate) on the growth of infants. BMJ 2:80-82.

View

Lindquist EF (1940)
Statistical analysis in educational research. Boston: Houghton Mifflin.

View

Dahlberg G (1940)
Statistical methods for medical and biological students. London: George Allen & Unwin.

View

Bell JA (1941)
Pertussis prophylaxis with two doses of alum-precipitated vaccine. Public Health Reports 56:1535-1546.

View

Greenwood M (1943)
Statistical note. Lancet 2:634-5.

View

Anderson T (1944)
Clinical studies in sulphonamide chemotherapy. MD thesis, University of Glasgow.

View

Mackenna RMB, Cooper-Willis ES (1945)
Impetigo contagiosa in the army, treated with microcrystalline sulphathiazole. Lancet 2:357-358.

View

Mainland D (1948)
Statistical methods in medical research. Canadian Journal of Research E, 26:1-166.

View

Bell JA (1948)
Diphtheria immunization use of an alum-precipitated mixture of pertussis vaccine and diphtheria toxoid. JAMA 137:1009-1016.

View

Bell JA (1948)
Pertussis immunization use of two doses of an alum-precipitated mixture of diphtheria toxoid and pertussis vaccine. JAMA 137:1276-1281.

View

Bell JA (1948)
The epidemiological principles and procedures involved in a study of the prophylactic value of an alum-precipated mixture of diphtheria toxoid and pertussis vaccine. Thesis submitted to the School of Hygiene and Public Health of the Johns Hopkins University in conformity with the requirements for the Degree of Doctor of Public Health.

View

Reid DD (1950)
Statistics in clinical research. Annals of the New York Academy of Sciences 52:931-934.

View

Lancaster HO (1951)
Quantitative methods in biological and medical sciences. Springer: New York.

View

Bernstein L, Weatherall M (1952)
Statistics for medical and other biological students. London: E & S Livingstone.

View

Bross I (1952)
Sequential medical plans. Biometrics 8:188-205.

View

Hamilton M, Wilson GM, Armitage P, Boyd JT (1953)
The treatment of intermittent claudication with Vitamin E. Lancet 1: 367-370.

View

Kilpatrick GS, Oldham PD (1954)
Calcium chloride and adrenaline as bronchial dilators compared by sequential analysis. BMJ 2:1388-1391.

View

Miall WE, Oldham PD, Cochrane AL (1954)
The treatment of complicated pneumoconiosis with isoniazid. British Journal of Industrial Medicine 11:186-191.

View

Mainland D (1955)
An experimental statistician looks at anthropometry. Annals of the New York Academy of Sciences 63:474-483.

View

Tebrock HE, Arminio JJ, Johnston JH (1956)
Usefulness of bioflavinoids and ascorbic acid in treatment of common cold. JAMA 162:1227-1233.

View

Newton DRL, Tanner JM (1956)
N-acetyl-para-aminophenol as an analgesic. A controlled clinical trial using the method of sequential analysis. BMJ 2:1096-9.

View

Snell ES, Armitage P (1957)
Clinical comparison of diamorphine and pholcodine as cough suppressants by a new method of sequential analysis. Lancet 1:860-862

View

Watkinson G (1958)
Treatment of ulcerative colitis with topical hydrocortisone hemisuccinate sodium. A controlled trial employing restricted sequential analysis. BMJ 2:1077-82.

View

Robertson JD, Armitage P (1959)
Report of a clinical trial to compare two hypotensive agents. Anaesthesia 14:53-64.

View

Zubrod CG, Schneiderman M, Frei E, Brindley C, Gold GL, Shnider B, Oviedo R, Gorman J, Jones R, Jonsson U, Colsky J, Chalmers T, Ferguson B, Dederick M, Holland J, Selawry O, Regelson W, Lasagna L, Owens AH (1960)
Appraisal of methods for the study of chemotherapy in man: Comparative therapeutic trial of nitrogen mustard and thiophosphoramide. Journal of Chronic Diseases 11:7-33.

View

Armitage P (1960)
Sequential medical trials. Oxford: Blackwell.

View

Mainland D (1960)
The use and misuse of statistics in medical publications. Clinical Pharmacology and Therapeutics 1:411-22.

View

Cohen J (1962)
The statistical power of abnormal-social psychological research: a review. Journal of Abnormal Social Psychology 65:145-153.

View

Truelove SC, Watkinson G, Draper G (1962)
Comparison of corticosteroid and sulphasalazine therapy in ulcerative colitis. BMJ 2:1708-1711.

View

Mainland D (1963)
Elementary medical statistics: 2nd edn. Philadelphia: WB Saunders Co.

View

Percy JS, Stephenson P, Thompson M (1964)
Indomethacin in the treatment of rheumatic diseases. Annals of Rheumatic Diseases 28:157-162.

View

Ley HL (1969)
Antibiotic drugs: procedural and interpretive regulations. Federal Register Vol. 34, No. 180, pp14596-597.

View

Armitage P (1971)
Statistical methods in medical research. Blackwell Scientific Publications: Edinburgh.

View

Shaikh W, Vayda E, Freeman W (1976)
A systematic review of the literature on the studies of tonsillectomy and adenoidectomy. Pediatrics 57:401-407.

View

Farquhar JW (1978)
The community-based model of life style intervention trials. American Journal of Epidemiology 108:103-111.

View

Freiman JA, Chalmers TC, Smith H, Kuebler RR (1978)
The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 "negative" trials. New England Journal of Medicine 299:690-94.

View

Gruppo Italiano per lo Studio della Streptochinasi nell'Infarto Miocardico (GISSI) (1986)
Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet 1: 397-402.

View

ISIS-2 (second International Study of Infarct Survival) Collaborative Group (1988)
Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17 187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 332:349–360.

View

Chalmers TC (1988)
Data analysis for clinical medicine: the quantitative approach to patient care in gastroenterology. Rome: International University Press.

View

Newcombe RG (1988)
Explanatory and pragmatic estimates of the treatment effect when deviations from allocated treatment occur. . Statistics in Medicine Volume 7, Issue 11 p. 1179-1186.

View

Donner A, Brown KS, Brasher P (1990)
A methodological review of non-therapeutic intervention trials employing cluster randomization, 1979-1989. International Journal of Epidemiology 19:795-800.

View

Simpson JM, Klar N, Donner A (1995)
Accounting for cluster randomization: a review of primary prevention trials, 1990 through 1993. American Journal of Public Health 85:1378-1383.

View

Montori VM, Devereaux PJ, Adhikari NK, Burns KE, Eggert CH, Briel M, Lacchetti C, Leung TW, Darling E, Bryant DM, Bucher HC, Schunemann HJ, Meade MO, Cook DJ, Erwin PJ, Sood A, Sood R, Lo B, Thompson CA, Zhou Q, Mills E, Guyatt GH (2005)
Randomized trials stopped early for benefit: a systematic review. JAMA 294:2203-2209.

View

Trotta F, Apolone G, Garattini S, Tafuri G (2008)
Stopping a trial early in oncology: for patients or for industry? Annals of Oncology Feb 29; doi:10.1093/annonc/mdn042.

View

Kramer MS, Martin RM, Sterne JA, Shapiro S, Mourad D, Platt RW (2009)
The double jeopardy of clustered measurement and cluster randomisation. BMJ 339:503-505.

View

Bassler D, Briel M, Montori VM, Lane M, Glasziou P, Zhou Q, Heels-Ansdell D, Walter SD, Guyatt GH; STOPIT-2 Study Group (2010)
Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA. 303:1180-7.

View

Farewell V, Johnson T (2010)
Woods and Russell, Hill, and the emergence of medical statistics. Statistics in Medicine 29:1459-1476.

View

Farewell V, Johnson T (2014)
Major Greenwood’s Early Career and the First Departments of Medical Statistics. Statistics in Medicine 33: 2161 – 2173.

View

Tudur Smith C, Marcucci M, Nolan SJ, Iorio A, Sudell M, Riley R, Rovers MM, Williamson PR (2016)
Individual participant data meta-analyses compared with meta-analyses based on aggregate data. Cochrane Database of Systematic Reviews 2016, Issue 9. Art. No.: MR000007.DOI: 10.1002/14651858.MR000007.pub3.

View

Imberger G, Thorlund K, Gluud C, Wetterslev J (2016)
False-positive findings in Cochrane meta-analyses with and without application of trial sequential analysis: an empirical review. BMJ Open 6(8):e011890.

View

Farewell V, Johnson T (2016)
Major Greenwood (1880 – 1949): a biographical and bibliographical study. Statistics in Medicine 35(5):645-670.

View

Farewell V, Johnson T (2016)
Major Greenwood (1880 – 1949): the biography. Statistics in Medicine 35: 5533-5535.

View

Horby P, Lim WS, Emberson J, Mafham M, Bell J, Linsell L, Staplin N, Brightling C, Ustianowski A, Elmahi E, Prudon B, Green C, Felton T, Chadwick D, Rege K, Fegan C, Chappell LC, Faust SN, Jaki T, Jeffrey K, Montgomery A, Rowan K, Juszcak E, Baillie JK, Haynes R, Landray MJ, on behalf of the RECOVERY Collaborative Group (2020)
Effect of Dexamethasone in Hospitalized Patients with COVID-19: Preliminary Report. medRχiv. Doi: https://doi.org/10.1101/2020.06.22.20137273

View

Horby P, Mafham M, Linsell L, Bell JL, Staplin N, Emberson JR, Wiselka M, Ustianowski A, Elmahi E, Prudon B, Whitehouse A, Felton T, Williams J, Faccenda J, Underwood J, Baillie JK, Chappell L, Faust SN, Jaki T, Jeffery K, Lim WS, Montgomery A, Rowan K, Tarning J, Watson JA, White NJ, Juszczak E, Haynes R, Landray MJ (2020)
Effect of Hydroxychloroquine in Hospitalized Patients with COVID-19: Preliminary results from a multi-centre, randomized, controlled trial. doi: https://doi.org/10.1101/2020.07.15.20151852

View

RECOVERY Trial Team (2020)
Randomized Evaluation of COVID019 Therapy (RECOVERY). RECOVERY Central Coordinating Office, Oxford. www.recoverytrial.net

View

Pan H, Peto R, Karim QA, Alejandria M, Henao-Restrepo AM, García CH, Kieny MP, Malekzadeh R, Murthy S, Preziosi MP, Reddy S, Periago MR, Sathiyamoorthy V, Røttingen JA, Swaminathan S, WHO Solidarity trial consortium (2020)
Repurposed antiviral drugs for COVID-19 –interim WHO SOLIDARITY trial results. MedRxiv. doi: https://doi.org/10.1101/2020.10.15.20209817

View

Bradley SH, DeVito NJ, Lloyd KE, Richards GC, Rombey T, Wayant C, Gill PJ (2020)
Reducing bias and improving transparency in medical research: a critical overview of the problems, progress and suggested next steps. Journal of the Royal Society of Medicine 113:433-443.

View

Bradley VC, Kuriwaki S, Isakov M, Sejdinovic D, Meng X, Flaxman S (2021)
Unrepresentative big surveys significantly overestimated US vaccine uptake. Nature 600, 695–700. https://doi.org/10.1038/s41586-021-04198-4.

View

Schoeler T, Speed D, Porcu E, Pirastu N, Pingault JB, Kutalik Z (2023)
Participation bias in the UK Biobank distorts genetic associations and downstream analyses. Nat Hum Behav. 2023 Jul;7(7):1216-1227. doi: 10.1038/s41562-023-01579-9.

View

Show


Armitage P† (2003).
Some recollections of the early years of the Medical Research Council (MRC) Statistical Research Unit. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Huth EJ (2006).
Transatlantic ideas on the philosophy of therapeutics in the middle of the 19th century. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Huth EJ (2006).
Elisha Bartlett (1804–1855), an American disciple of Jules Gavarret. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Huth EJ (2006).
Jules Gavarret’s Principes Généraux de Statistique Médicale: a pioneering text on the statistical analysis of the results of treatments. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Gluud C (2008).
Povl Heiberg (1868-1963). JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Gluud C, Hilden J (2008).
Povl Heiberg’s 1897 methodological study on the statistical method as an aid in therapeutic trials. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Joosse NP, Pormann PE (2008).
Archery, mathematics, and conceptualising inaccuracies in medicine in 13th century Iraq and Syria. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Shannon H (2008).
A statistical note on Karl Pearson’s 1904 meta-analysis. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Armitage P† (2009).
A statistical note on the analysis of the 1948 MRC streptomycin trial. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Chalmers I, Toth B (2009).
19th century controlled trials to test whether belladonna prevents scarlet fever. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Cox DR (2009).
Randomization for concealment. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Farewell V, Johnson A† (2010).
Hilda Woods (1892-1971). JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Farewell V, Johnson A† (2010).
William Thomas Russell (1888-1953). JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Farewell V, Johnson A† (2010).
The first British textbook of medical statistics. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Farewell V, Johnson A† (2011).
The origins of Austin Bradford Hill’s classic textbook of medical statistics. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Campbell MJ (2012).
Doing clinical trials large enough to achieve adequate reductions in uncertainties about treatment effects. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Armitage P† (2013).
The evolution of ways of deciding when clinical trials should stop recruiting. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

La Rochelle P, Julien A-S (2013).
How dramatic were the effects of handwashing on maternal mortality observed by Ignaz Semmelweis? JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Schlesselman JJ (2015).
Jerome Cornfield’s Bayesian approach to assessing interim results in clinical trials. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Altman DG† (2017).
Donald Mainland: anatomist, educator, thinker, medical statistician, trialist, rheumatologist. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Senn SJ (2017).
Cushny and Peebles, optical isomers, and the birth of modern statistics. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Toth B (2018).
Pioneering controlled trials of treatments for erysipelas and pneumonia in Glasgow, 1936-47. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Bird A (2018).
James Jurin and the avoidance of bias in collecting and assessing evidence on the effects of variolation. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Marson Smith P, Colquhoun D, Chalmers I (2019).
John Henry Gaddum’s 1940 guidance on controlled clinical trials JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Matthews RAJ (2020).
The origins of the treatment of uncertainty in clinical medicine. Part 2: the emergence of probability theory and its limitations. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Tröhler U (2020)
Probabilistic thinking and the evaluation of therapies, 1700-1900. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Matthews RAJ (2020).
The origins of the treatment of uncertainty in clinical medicine. Part 1: Ancient roots, familiar disputes. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Held L, Matthews RAJ (2022).
Paradigm lost: Carl Liebermeister and the development of modern medical statistics. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Glasziou P, Matthews R, Boutron I, Chalmers I, Armitage P† (2023)
The differences and overlaps between ‘explanatory’ and ‘pragmatic’ controlled trials: a historical perspective. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Senn S (2024).
An Early 20th Century Handbook on ‘Meta-analysis’: David Brunt’s The Combination of Observations. JLL Bulletin: Commentaries on the history of treatment evaluation.

View

Big Data, Big Bias? Evidence on the effects of selection bias in large observational studies

View