Quantifying uncertainty

Chance may affect the results of a study if too few outcomes have been observed to yield reliable estimates of treatment effects. Small studies in which few outcome events occur are usually not informative and the results are sometimes seriously misleading.

97 RECORDS
30 ARTICLES
1 BLOG

The need to address treatment uncertainties

Principles of Testing

Treatment comparisons are essential

Treatment comparisons must be fair

Biases

Double dummy

The play of chance

Bringing it all together for the benefit of patients and the public

James Lind LibraryIllustrating the development of fair tests of treatments in health care

Quantifying uncertainty

Chance may affect the results of a study if too few outcomes have been observed to yield reliable estimates of treatment effects. Small studies in which few outcome events occur are usually not informative and the results are sometimes seriously misleading.

SORT date ▲ author

97 RECORDS

30 ARTICLES

1 BLOG

Lavoisier A-L de (1784) Undated documents contained in: Mémoires de Lavoisier, Oeuvres, Tome III. Paris: Imprimerie Impériale, 1865. p 509.

de Laplace P-S (1820) Théorie analytique des probabilités [Analytic theory of probabilities]. Oeuvres complètes 7 (3ème édition). Paris:Courcier, page lxxvii.

Gavarret LDJ (1840) Principes généraux de statistique médicale: ou développement des régles qui doivent présider à son emploi [General principles of medical statistics: or the development of rules that must govern their use]. Paris: Bechet jeune & Labé.

Bartlett E (1844) An essay on the philosophy of medical science. Philadelphia: Lea and Blanchard.

Balfour TG (1854) Quoted in West C. Lectures on the Diseases of Infancy and Childhood. London, Longman, Brown, Green and Longmans, p 600.

Schweig G (1854) Auseinandersetzung der statistischen Methode [Deliberation about the statistical method]. Archiv fὔr physiologische Helkunde 13:305-355.

Airy GB (1861) On the algebraical and numerical theory of errors of observations and the combination of observations. Cambridge: MacMillan and Co.

Jürgensen T (1866) Klinische Studien über die Behandlung des Abdominalytyphus mittelst des kalten wassers [Clinical studes on the treatment of abdominal typhus with cold water]. Leipzig, Vogel, pviii.

Fick A (1866) Die Medicinische Physik [Medical physics]. Braunschweig,Vieweg, pp 430-447.

Jessen W (1867) Zur analytischen statistik [On analytic statistics]. Zeitschrift for biologie 3:128-136.

Bain A (1870) Logic. London: Longmans, Green, Reader & Dyer p 362.

Liebermeister C (1877) Ueber Wahrscheinlichkeitsrechnung in Anwendung auf therapeutische Statistik [On the calculus of probabilities applied to therapeutic statistics]. In: Volkmann R (ed), Sammlung Klinischer Vortrȁge, Innere Medicin, No 39, pp 935-962.

Ephraim A (1893) Uber die Bedeutung de statistischen Methode für die Medicin [On the relevance of the statistical method for medicine]. Volkmann’s Sammlung Klinische Vortraege N.F. Innere Medicin 24:706-716. Leipzig: Breitkopf and Härtel.

Heiberg P (1897) Studier over den statistiske undersøgelsesmetode som hjælpemiddel ved terapeutiske undersøgelser [Studies on the statistical study design as an aid in therapeutic trials]. Bibliotek for Læger 89:1-40.

Davenport CB (1899) Statistical methods, with special reference to biological variation. New York: John Wiley & Sons.

Pearson K (1904) Report on certain enteric fever inoculation statistics. BMJ 3:1243-1246.

Alt K (1909) Behandlungsversuche mit Arsenophenylglyzin bei Paralytikern [Treatment experiments with arsenophenylglycine in paralytics]. Muenchener Medizinische Wochenschrift 56:1457-1459.

Advisory Committee on Plague Investigations in India (1912) The serum treatment of human plague. Journal of Hygiene. Plague Supplement II, LVI:326-39.

Brunt D (1917) The Combination of Observations. Cambridge University Press.

Pearl R (1919) A statistical discussion of the relative efficacy of different methods of treating pneumonia. Archives of Internal Medicine 24:398-403.

Acton HW (1920) Researches on the treatment of benign tertian fever. Lancet 1:1257-1261.

Locke EA (1924) The serologic treatment of lobar pneumonia. Boston Medical and Surgical Journal 190:196-203.

Ferry NS, Gordon EJ, Munro FW, Steele AH, Fisher LW (1928) Clinical results with measles streptococcus toxin and antitoxin. JAMA 91:1277-1280.

Park WH, Bullowa JGM, Rosenbluth NM (1928) The treatment of lobar pneumonia with refined specific antibacterial serum. JAMA 91:1503-1508.

Bullowa JGM (1928) The control. Contribution to a symposium on the use of antipneumococcic refined serum in lobar pneumonia, 15 December 1927. Bulletin of the New York Academy of Sciences 4:339-343.

Bullowa JGM (1928) Use of antipneumococcic refined serum in lobar pneumonia: data necessary for a comparison between cases treated with serum and cases not so treated, and the importance of a significant control series of cases. JAMA 90: 1354-1358.

Bullowa JGM (1929) The serum treatment and its evaluation in lobar pneumonia. Bulletin of the New York Academy of Medicine 5:328-362.

Woods HM, Russell WT (1931) An introduction to medical statistics. London: Staples Press.

Ellison JB (1932) Intensive vitamin therapy in measles. BMJ 2:708-711.

Greenwood M (1934) Epidemics and crowd-diseases. Oxford: Oxford University Press.

Hicks EP, Diwan Chand S (1935) The relative clinical efficacy of totaquina and quinine. In: Records of the Malaria Survey of India 5:39-50.

Gilliam AG, Onstott RH (1936) Results of field studies with poliomyelitis vaccine. American Journal of Public Health 26:113-118.

Theobald GW (1937) Effect of calcium and vitamin A and D on incidence of pregnancy toxaemia. Lancet 2:1397-1399.

Bullowa JGM (1937) Serum therapy. In: Bullowa JGM. The management of the pneumonias. New York: Oxford University Press, pp 283-298.

Aykroyd WR, Krishnan RSBG (1938) Effect of calcium lactate on children in a nursery school. Lancet 2:153-155.

Kendrick P, Eldering G (1939) A study in active immunization against pertussis. American Journal of Hygiene 29 Sec. B:133-153.

Price MR (1940) Effects of a supplement of vitamin B (adsorbate) on the growth of infants. BMJ 2:80-82.

Lindquist EF (1940) Statistical analysis in educational research. Boston: Houghton Mifflin.

Dahlberg G (1940) Statistical methods for medical and biological students. London: George Allen & Unwin.

Bell JA (1941) Pertussis prophylaxis with two doses of alum-precipitated vaccine. Public Health Reports 56:1535-1546.

Greenwood M (1943) Statistical note. Lancet 2:634-5.

Anderson T (1944) Clinical studies in sulphonamide chemotherapy. MD thesis, University of Glasgow.

Mackenna RMB, Cooper-Willis ES (1945) Impetigo contagiosa in the army, treated with microcrystalline sulphathiazole. Lancet 2:357-358.

Mainland D (1948) Statistical methods in medical research. Canadian Journal of Research E, 26:1-166.

Bell JA (1948) Diphtheria immunization use of an alum-precipitated mixture of pertussis vaccine and diphtheria toxoid. JAMA 137:1009-1016.

Bell JA (1948) Pertussis immunization use of two doses of an alum-precipitated mixture of diphtheria toxoid and pertussis vaccine. JAMA 137:1276-1281.

Reid DD (1950) Statistics in clinical research. Annals of the New York Academy of Sciences 52:931-934.

Lancaster HO (1951) Quantitative methods in biological and medical sciences. Springer: New York.

Bernstein L, Weatherall M (1952) Statistics for medical and other biological students. London: E & S Livingstone.

Bross I (1952) Sequential medical plans. Biometrics 8:188-205.

Hamilton M, Wilson GM, Armitage P, Boyd JT (1953) The treatment of intermittent claudication with Vitamin E. Lancet 1: 367-370.

Kilpatrick GS, Oldham PD (1954) Calcium chloride and adrenaline as bronchial dilators compared by sequential analysis. BMJ 2:1388-1391.

Miall WE, Oldham PD, Cochrane AL (1954) The treatment of complicated pneumoconiosis with isoniazid. British Journal of Industrial Medicine 11:186-191.

Mainland D (1955) An experimental statistician looks at anthropometry. Annals of the New York Academy of Sciences 63:474-483.

Tebrock HE, Arminio JJ, Johnston JH (1956) Usefulness of bioflavinoids and ascorbic acid in treatment of common cold. JAMA 162:1227-1233.

Newton DRL, Tanner JM (1956) N-acetyl-para-aminophenol as an analgesic. A controlled clinical trial using the method of sequential analysis. BMJ 2:1096-9.

Snell ES, Armitage P (1957) Clinical comparison of diamorphine and pholcodine as cough suppressants by a new method of sequential analysis. Lancet 1:860-862

Watkinson G (1958) Treatment of ulcerative colitis with topical hydrocortisone hemisuccinate sodium. A controlled trial employing restricted sequential analysis. BMJ 2:1077-82.

Robertson JD, Armitage P (1959) Report of a clinical trial to compare two hypotensive agents. Anaesthesia 14:53-64.

Armitage P (1960) Sequential medical trials. Oxford: Blackwell.

Mainland D (1960) The use and misuse of statistics in medical publications. Clinical Pharmacology and Therapeutics 1:411-22.

Cohen J (1962) The statistical power of abnormal-social psychological research: a review. Journal of Abnormal Social Psychology 65:145-153.

Truelove SC, Watkinson G, Draper G (1962) Comparison of corticosteroid and sulphasalazine therapy in ulcerative colitis. BMJ 2:1708-1711.

Mainland D (1963) Elementary medical statistics: 2nd edn. Philadelphia: WB Saunders Co.

Percy JS, Stephenson P, Thompson M (1964) Indomethacin in the treatment of rheumatic diseases. Annals of Rheumatic Diseases 28:157-162.

Ley HL (1969) Antibiotic drugs: procedural and interpretive regulations. Federal Register Vol. 34, No. 180, pp14596-597.

Armitage P (1971) Statistical methods in medical research. Blackwell Scientific Publications: Edinburgh.

Shaikh W, Vayda E, Freeman W (1976) A systematic review of the literature on the studies of tonsillectomy and adenoidectomy. Pediatrics 57:401-407.

Farquhar JW (1978) The community-based model of life style intervention trials. American Journal of Epidemiology 108:103-111.

Freiman JA, Chalmers TC, Smith H, Kuebler RR (1978) The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 "negative" trials. New England Journal of Medicine 299:690-94.

Gruppo Italiano per lo Studio della Streptochinasi nell'Infarto Miocardico (GISSI) (1986) Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet 1: 397-402.

ISIS-2 (second International Study of Infarct Survival) Collaborative Group (1988) Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17 187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 332:349–360.

Chalmers TC (1988) Data analysis for clinical medicine: the quantitative approach to patient care in gastroenterology. Rome: International University Press.

James Lind Library
Illustrating the development of fair tests of treatments in health care

Lavoisier A-L de (1784)
Undated documents contained in: Mémoires de Lavoisier, Oeuvres, Tome III. Paris: Imprimerie Impériale, 1865. p 509.

de Laplace P-S (1820)
Théorie analytique des probabilités [Analytic theory of probabilities]. Oeuvres complètes 7 (3ème édition). Paris:Courcier, page lxxvii.

Gavarret LDJ (1840)
Principes généraux de statistique médicale: ou développement des régles qui doivent présider à son emploi [General principles of medical statistics: or the development of rules that must govern their use]. Paris: Bechet jeune & Labé.

Bartlett E (1844)
An essay on the philosophy of medical science. Philadelphia: Lea and Blanchard.

Balfour TG (1854)
Quoted in West C. Lectures on the Diseases of Infancy and Childhood. London, Longman, Brown, Green and Longmans, p 600.

Schweig G (1854)
Auseinandersetzung der statistischen Methode [Deliberation about the statistical method]. Archiv fὔr physiologische Helkunde 13:305-355.

Airy GB (1861)
On the algebraical and numerical theory of errors of observations and the combination of observations. Cambridge: MacMillan and Co.

Jürgensen T (1866)
Klinische Studien über die Behandlung des Abdominalytyphus mittelst des kalten wassers [Clinical studes on the treatment of abdominal typhus with cold water]. Leipzig, Vogel, pviii.

Fick A (1866)
Die Medicinische Physik [Medical physics]. Braunschweig,Vieweg, pp 430-447.

Jessen W (1867)
Zur analytischen statistik [On analytic statistics]. Zeitschrift for biologie 3:128-136.

Bain A (1870)
Logic. London: Longmans, Green, Reader & Dyer p 362.

Liebermeister C (1877)
Ueber Wahrscheinlichkeitsrechnung in Anwendung auf therapeutische Statistik [On the calculus of probabilities applied to therapeutic statistics]. In: Volkmann R (ed), Sammlung Klinischer Vortrȁge, Innere Medicin, No 39, pp 935-962.

Ephraim A (1893)
Uber die Bedeutung de statistischen Methode für die Medicin [On the relevance of the statistical method for medicine]. Volkmann’s Sammlung Klinische Vortraege N.F. Innere Medicin 24:706-716. Leipzig: Breitkopf and Härtel.

Heiberg P (1897)
Studier over den statistiske undersøgelsesmetode som hjælpemiddel ved terapeutiske undersøgelser [Studies on the statistical study design as an aid in therapeutic trials]. Bibliotek for Læger 89:1-40.

Davenport CB (1899)
Statistical methods, with special reference to biological variation. New York: John Wiley & Sons.

Pearson K (1904)
Report on certain enteric fever inoculation statistics. BMJ 3:1243-1246.

Alt K (1909)
Behandlungsversuche mit Arsenophenylglyzin bei Paralytikern [Treatment experiments with arsenophenylglycine in paralytics]. Muenchener Medizinische Wochenschrift 56:1457-1459.

Advisory Committee on Plague Investigations in India (1912)
The serum treatment of human plague. Journal of Hygiene. Plague Supplement II, LVI:326-39.

Brunt D (1917)
The Combination of Observations. Cambridge University Press.

Pearl R (1919)
A statistical discussion of the relative efficacy of different methods of treating pneumonia. Archives of Internal Medicine 24:398-403.

Acton HW (1920)
Researches on the treatment of benign tertian fever. Lancet 1:1257-1261.

Locke EA (1924)
The serologic treatment of lobar pneumonia. Boston Medical and Surgical Journal 190:196-203.

Ferry NS, Gordon EJ, Munro FW, Steele AH, Fisher LW (1928)
Clinical results with measles streptococcus toxin and antitoxin. JAMA 91:1277-1280.

Park WH, Bullowa JGM, Rosenbluth NM (1928)
The treatment of lobar pneumonia with refined specific antibacterial serum. JAMA 91:1503-1508.

Bullowa JGM (1928)
The control. Contribution to a symposium on the use of antipneumococcic refined serum in lobar pneumonia, 15 December 1927. Bulletin of the New York Academy of Sciences 4:339-343.

Bullowa JGM (1928)
Use of antipneumococcic refined serum in lobar pneumonia: data necessary for a comparison between cases treated with serum and cases not so treated, and the importance of a significant control series of cases. JAMA 90: 1354-1358.

Bullowa JGM (1929)
The serum treatment and its evaluation in lobar pneumonia. Bulletin of the New York Academy of Medicine 5:328-362.

Woods HM, Russell WT (1931)
An introduction to medical statistics. London: Staples Press.

Ellison JB (1932)
Intensive vitamin therapy in measles. BMJ 2:708-711.

Greenwood M (1934)
Epidemics and crowd-diseases. Oxford: Oxford University Press.

Hicks EP, Diwan Chand S (1935)
The relative clinical efficacy of totaquina and quinine. In: Records of the Malaria Survey of India 5:39-50.

Gilliam AG, Onstott RH (1936)
Results of field studies with poliomyelitis vaccine. American Journal of Public Health 26:113-118.

Theobald GW (1937)
Effect of calcium and vitamin A and D on incidence of pregnancy toxaemia. Lancet 2:1397-1399.

Bullowa JGM (1937)
Serum therapy. In: Bullowa JGM. The management of the pneumonias. New York: Oxford University Press, pp 283-298.

Aykroyd WR, Krishnan RSBG (1938)
Effect of calcium lactate on children in a nursery school. Lancet 2:153-155.

Kendrick P, Eldering G (1939)
A study in active immunization against pertussis. American Journal of Hygiene 29 Sec. B:133-153.

Price MR (1940)
Effects of a supplement of vitamin B (adsorbate) on the growth of infants. BMJ 2:80-82.

Lindquist EF (1940)
Statistical analysis in educational research. Boston: Houghton Mifflin.

Dahlberg G (1940)
Statistical methods for medical and biological students. London: George Allen & Unwin.

Bell JA (1941)
Pertussis prophylaxis with two doses of alum-precipitated vaccine. Public Health Reports 56:1535-1546.

Greenwood M (1943)
Statistical note. Lancet 2:634-5.

Anderson T (1944)
Clinical studies in sulphonamide chemotherapy. MD thesis, University of Glasgow.

Mackenna RMB, Cooper-Willis ES (1945)
Impetigo contagiosa in the army, treated with microcrystalline sulphathiazole. Lancet 2:357-358.

Mainland D (1948)
Statistical methods in medical research. Canadian Journal of Research E, 26:1-166.

Bell JA (1948)
Diphtheria immunization use of an alum-precipitated mixture of pertussis vaccine and diphtheria toxoid. JAMA 137:1009-1016.

Bell JA (1948)
Pertussis immunization use of two doses of an alum-precipitated mixture of diphtheria toxoid and pertussis vaccine. JAMA 137:1276-1281.

Reid DD (1950)
Statistics in clinical research. Annals of the New York Academy of Sciences 52:931-934.

Lancaster HO (1951)
Quantitative methods in biological and medical sciences. Springer: New York.

Bernstein L, Weatherall M (1952)
Statistics for medical and other biological students. London: E & S Livingstone.

Bross I (1952)
Sequential medical plans. Biometrics 8:188-205.

Hamilton M, Wilson GM, Armitage P, Boyd JT (1953)
The treatment of intermittent claudication with Vitamin E. Lancet 1: 367-370.

Kilpatrick GS, Oldham PD (1954)
Calcium chloride and adrenaline as bronchial dilators compared by sequential analysis. BMJ 2:1388-1391.

Miall WE, Oldham PD, Cochrane AL (1954)
The treatment of complicated pneumoconiosis with isoniazid. British Journal of Industrial Medicine 11:186-191.

Mainland D (1955)
An experimental statistician looks at anthropometry. Annals of the New York Academy of Sciences 63:474-483.

Tebrock HE, Arminio JJ, Johnston JH (1956)
Usefulness of bioflavinoids and ascorbic acid in treatment of common cold. JAMA 162:1227-1233.

Newton DRL, Tanner JM (1956)
N-acetyl-para-aminophenol as an analgesic. A controlled clinical trial using the method of sequential analysis. BMJ 2:1096-9.

Snell ES, Armitage P (1957)
Clinical comparison of diamorphine and pholcodine as cough suppressants by a new method of sequential analysis. Lancet 1:860-862

Watkinson G (1958)
Treatment of ulcerative colitis with topical hydrocortisone hemisuccinate sodium. A controlled trial employing restricted sequential analysis. BMJ 2:1077-82.

Robertson JD, Armitage P (1959)
Report of a clinical trial to compare two hypotensive agents. Anaesthesia 14:53-64.

Armitage P (1960)
Sequential medical trials. Oxford: Blackwell.

Mainland D (1960)
The use and misuse of statistics in medical publications. Clinical Pharmacology and Therapeutics 1:411-22.

Cohen J (1962)
The statistical power of abnormal-social psychological research: a review. Journal of Abnormal Social Psychology 65:145-153.

Truelove SC, Watkinson G, Draper G (1962)
Comparison of corticosteroid and sulphasalazine therapy in ulcerative colitis. BMJ 2:1708-1711.

Mainland D (1963)
Elementary medical statistics: 2nd edn. Philadelphia: WB Saunders Co.

Percy JS, Stephenson P, Thompson M (1964)
Indomethacin in the treatment of rheumatic diseases. Annals of Rheumatic Diseases 28:157-162.

Ley HL (1969)
Antibiotic drugs: procedural and interpretive regulations. Federal Register Vol. 34, No. 180, pp14596-597.

Armitage P (1971)
Statistical methods in medical research. Blackwell Scientific Publications: Edinburgh.

Shaikh W, Vayda E, Freeman W (1976)
A systematic review of the literature on the studies of tonsillectomy and adenoidectomy. Pediatrics 57:401-407.

Farquhar JW (1978)
The community-based model of life style intervention trials. American Journal of Epidemiology 108:103-111.

Freiman JA, Chalmers TC, Smith H, Kuebler RR (1978)
The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 "negative" trials. New England Journal of Medicine 299:690-94.

Gruppo Italiano per lo Studio della Streptochinasi nell'Infarto Miocardico (GISSI) (1986)
Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet 1: 397-402.

ISIS-2 (second International Study of Infarct Survival) Collaborative Group (1988)
Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17 187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 332:349–360.

Chalmers TC (1988)
Data analysis for clinical medicine: the quantitative approach to patient care in gastroenterology. Rome: International University Press.

Newcombe RG (1988)
Explanatory and pragmatic estimates of the treatment effect when deviations from allocated treatment occur. . Statistics in Medicine Volume 7, Issue 11 p. 1179-1186.

Donner A, Brown KS, Brasher P (1990)
A methodological review of non-therapeutic intervention trials employing cluster randomization, 1979-1989. International Journal of Epidemiology 19:795-800.

Simpson JM, Klar N, Donner A (1995)
Accounting for cluster randomization: a review of primary prevention trials, 1990 through 1993. American Journal of Public Health 85:1378-1383.

Trotta F, Apolone G, Garattini S, Tafuri G (2008)
Stopping a trial early in oncology: for patients or for industry? Annals of Oncology Feb 29; doi:10.1093/annonc/mdn042.

Kramer MS, Martin RM, Sterne JA, Shapiro S, Mourad D, Platt RW (2009)
The double jeopardy of clustered measurement and cluster randomisation. BMJ 339:503-505.

Bassler D, Briel M, Montori VM, Lane M, Glasziou P, Zhou Q, Heels-Ansdell D, Walter SD, Guyatt GH; STOPIT-2 Study Group (2010)
Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA. 303:1180-7.