Introduction

Inferences about the effects of treatments, including surgical treatments, rely on making comparisons. These comparisons may be with patient’s symptoms before a treatment has been applied. For example, the return of hearing after puncturing the ear drum (tympanotomy) in certain kinds of longstanding deafness can be so dramatic that the change can be confidently ascribed to the treatment (Cooper 1801; Burton 2015). More usually, treatments have more modest effects, and alternative treatments may differ from each other only slightly, if at all. In these circumstances disagreements are common about the mechanisms, the magnitude of any effects and the value of a particular treatment. Examples include disputes about different ways of treating wounds (Hidalgo 1604; Solis 2011), the timing and methods of limb amputations (Monro 1737; Bilguer 1764; Alanson 1782; Hunter 1794; Guthrie 1815; Hutchinson 1817), and about lithotripsy as an alternative to lithotomy (Cheselden 1740; Poisson et al 1835; Troehler 2013).

Reducing biases in assessing the effects of surgery

Obtaining reliable estimates of treatment effects, especially when differences between the active and control treatment are moderate, requires the effects of treatments to be distinguished from placebo effects, the effects of biases and the play of chance. Of the many biases that can lead to unreliable estimates of the effects of treatments, there are two in particular that researchers have attempted to reduce over the past century.

The first of these, allocation bias, happens when the patients in treatment comparison groups differ in prognosis, so that it is impossible to assess whether any differences in outcomes following treatment are due to differential effects of the treatments, or differences in their prognoses before treatments. Towards the end of the 19th century, prospective, alternate allocation to treatment comparison groups was introduced to generate groups of patients with similar prognoses (Chalmers et al. 2011). Surgeons were among those who contributed to this methodological development (Keith 1878; Hill 1879; Webster Fox 1894; Langton 1899), which, half a century later, led on to the adoption of concealed random allocation to treatment comparison groups (Medical Research Council 1948; Chalmers 2010; Chalmers et al. 2011).

The second of the two key biases, observer bias, happens when knowledge of treatment allocation may influence the way that treatments are administered and their possible effects measured. This bias was recognised to be a problem and steps taken to reduce it towards the end of the 18th century. A Royal Commission in Paris used blindfolds to assess whether, as claimed by Anton Mesmer, a force he had dubbed ‘animal magnetism’ was real (Lavoisier 1784; Commission Royale 1784; Donaldson 2016). A few years later, William Haygarth compared the effects of metallic ‘tractors’ with sham, wooden ‘tractors’ in patients with arthritis (Haygarth 1800).

These studies helped to demonstrate that patients’ expectations and perceptions could result in ineffective treatments being deemed effective. The need for blinding and placebo controls may be debatable if outcome measures are objective, like, for example mortality. However, when outcome measures are subjective and reflect satisfaction with the treatment in domains important to patients, such as pain, function, and quality of life (Ware 1993), comparisons of active treatment with placebo are important to reduce biased estimates of treatment effects (Gøtzsche et al. 2004; Wood et al. 2008; Savovic et al. 2012; Wartolowska et al. 2014).

This bias may be caused by patients’ expectations of treatment effects and what they were told to expect from the treatment. They may report symptoms based on their “hunches” about treatment being effective or give answers they believe are “correct” or expected from them, for example, because it would be regarded as impolite not to report improvement. (Sackett 2007; Hrobjartsson et al. 2014).

Blinding means concealing the treatment allocation from patients and any other people involved in the trial, such as assessors or researchers collecting data, caregivers, data analysts, who may bias the results of the trial by knowing the group to which patients have been allocated. Blinding also improves patient retention in trials, for example, across the trials we reviewed the withdrawal rate was low (4%) and comparable between the treatment and the placebo arm (Wartolowska et al. 2016). Blinding of patients may also reduce the likelihood of  patients in the control group not following the assigned treatment, so called adherence bias, and help to prevent the control group from seeking and receiving additional treatment outside the trial, which may lead to contamination of the control group (Hrobjartsson et al. 2014).

Although designing and carrying out unbiased controlled trials of surgery is undoubtedly challenging (Fairbank 1999), failure to take steps to reduce observer biases in surgical trials results in untrustworthy estimates of treatment effects (Wartolowska et al. 2016).

A few examples of sham/placebo treatments in the 1950s

Although adoption of blinding and use of placebos in drug trials began in the 19th century (Kaptchuk 2011), these measures were not used in surgical trials until the middle of the 20th century; even today, surgical comparison groups receiving sham treatments or placebos remain rare (McLeod 1999; Wente et al. 2003; Ergina et al. 2009; Cook 2009).

The earliest example of the use of placebo treatments in surgical research of which we are aware is a little known psychosurgical study reported in 1953 (Livingston 1953). Some patients with psychoses underwent anterior cingulate cortex isolation. i.e., excision of a part of cingulum, but four patients had only a skin incision and removal of a fragment of skull bone without any alteration to brain anatomy. None of the patients who had had placebo treatment improved and all of them subsequently underwent excision of a fragment of cingulum.

Better known early placebo controlled trials of surgical treatment compared the effects of internal mammary artery ligation with an inactive procedure consisting of bilateral skin incisions without actual artery ligation in patients with angina (Cobb et al. 1959; Dimond et al. 1960). The symptoms, exercise tolerance and use of nitroglycerin of patients in the ligation and the sham groups improved to similar extents. The improvement was attributed to patient interaction with the surgeon, the surgeon’s personality, participation in the treatment, alleviation of anxiety, and spontaneous improvement (Cobb et al. 1959; Dimond et al. 1960).

Use of placebo treatments during the 1980s and 1990s

These early trials in the 1950s studied few patients who were not informed that some of them would receive a placebo intervention. As was noted by Thomsen and colleagues in 1981, “knowledge of the possibility of having a purely placebo operation probably reduces the efficacy also in the actively treated group, and the outcome of the trial will be equivocal” (Thomsen et al. 1981). In their trial of treatment for Menière’s disease, half of the patients underwent an endolymphatic sac shunt operation, while the other half underwent regular mastoidectomy. The patients were told that the trial was comparing two different operations that were believed to be equally effective (Thomsen et al. 1981). Thomsen and his colleagues recognised the need to blind patients and investigators when subjective outcomes were being used to assess treatment effects: “In dealing with a disease or a syndrome like Menière’s disease, with its fluctuations in intensity of symptoms, spontaneous remission of symptoms, etc., open, uncontrolled studies are without any value in testing a treatment modality, be it medical or surgical. (…) The double-blind technique therefore is an absolute necessity in such investigations, and any leak in the blind must be considered disastrous” (Thomsen et al. 1981).

The advent of minimally-invasive surgical techniques and scopes opened up new possibilities for blinding in surgical trials. A skin incision imitating open surgery was no longer necessary because using a scope meant that there was either no visible surgical wound in the case of endoscopy or bronchoscopy, or only a small incision after laparoscopy or arthroscopy. For example, in trials of interventions to treat bleeding oesophageal varices (Fleisher et al. 1985; Hartigan 1994), peptic ulcers (MacLeod et al. 1983; Freitas et al. 1985), and endometriosis (Sutton et al. 1994) all patients underwent endoscopy or laparoscopy but only some patients had an additional active procedure to help secure haemostasis or remove the lesions. Other investigators used endoscopy to insert intragastric balloons in trials of surgical treatments for obesity (Lindor et al. 1987; Meshkinpour et al. 1988; Mathus-Vliegen et al. 1990).

Increased use of placebo controlled surgical trials in the 21st century

By the beginning of the current century the importance of blinding in assessing the effects of surgical treatments had become more widely accepted and there has been a rapid increase in the number of placebo-controlled surgical trials. Some of these trials have sparked heated debates about the role and ethical aspects of placebos in assessing efficacy of surgery, for example, the trials on the effects of dopaminergic neurone transplants in patients with advanced Parkinson’s disease (Freed et al. 2001; Olanow et al. 2003; Gross et al. 2011). These studies used a skin incision and burr holes in the skull as well as several additional diagnostic and therapeutic procedures, some of them clinically necessary and some performed to maintain blinding. For example, patients in both study arms underwent magnetic resonance imaging and positron emission tomography scans (Freed et al. 2001), or received drugs to prevent rejection of the transplanted cells (Olanow et al. 2003) or to prevent epilepsy after the transplantation procedure (Freed et al. 2001).

Placebo-controlled trials undertaken to assess the effects of arthroscopic debridement (Moseley et al. 2002) or tidal irrigation (Bradley et al. 2002) in patients with knee osteoarthritis have also been controversial. The blinded trial reported by Moseley and colleagues failed to detect the beneficial effects suggested by the results of earlier, unblinded trials of debridement (Merchan and Galindo 1993) and lavage (Ike et al. 1992).

Increasing use of implants, lasers, or other energy-emitting devices to alter tissues has meant that, in some cases, it has been possible to blind surgeons. For example, in a study reported by Freeman and colleagues (2006), the surgeon inserted a catheter under fluoroscopic guidance and handed over the subsequent procedure to a technician, who delivered or did not deliver the radio-frequency energy. In other trials, the delivery system for palatal implants to reduce obstructive sleep apnoea was prepared by the manufacturer to either contain the implant or not, so that surgeons did not know whether they had performed an actual implantation or a sham procedure (Gillespie et al. 2011; Maurer et al. 2012).

Increasingly sophisticated efforts to maintain blinding in surgical trials

Preparation for the placebo procedure has usually been done in the same way as for the active procedure. Steps have been taken to imitate visual, auditory and physical cues (Moseley et al. 2002; Pauza et al. 2004; Kallmes et al. 2009; Kapural et al. 2013). To mimic sounds, surgeons have been required to talk through the steps of the procedure (Rothstein et al. 2006), to ask for the instruments (Koutsourelakis et al. 2008; Sihvonen et al. 2013), to use suction (Sihvonen et al. 2013), or ask for a laser or other device to be activated; although, it was not used in the placebo group (Fleischer 1985; Mathus-Vliegen et al. 1990; Nease and Krempl 2004; Leon et al. 2005; Roehrborn et al. 2013).

Both the people performing interventions and endoscopic images have been screened from patients (Salem et al. 2004), either using a surgical drape (Roehrborn et al. 2013) or by arranging the operating room in such a way that the patient could not see the procedure (Kapural et al. 2013). In one trial patients were heavily sedated and required to wear dark goggles (Stone et al. 2002).

Surgeons also attempted to imitate sensory cues, for example, by manipulating the knee as if an actual arthroscopy was being performed (Sihvonen et al. 2013), injecting saline subcutaneously to mimic tidal irrigation (Bradley et al. 2002), or by splashing saline on the knee to simulate lavage (Moseley et al. 2002). In a trial of meniscectomy, surgeons used a mechanised shaver (without the blade) pushing it firmly against the patella to simulate the sensations the patient would experience during the surgery (Sihvonen et al. 2013). In a trial of intragastric balloon insertion for obesity, operators manipulated the endoscope as it would be manipulated during the balloon insertion and created the sensation of resistance in the stomach (Mathus-Vliegen et al. 1990).

Even smell during the surgery was imitated to make the placebo procedure indistinguishable from surgery. For example, in the trial by Deviere and colleagues there were concerns that patients could have known the allocation because the copolymer used in the active arm had a distinct smell (Deviere et al. 2005). Therefore, in trials of vertebroplasty, a container with cement was opened during the placebo procedure to help with blinding (Kallmes et al. 2009; Buchbinder et al. 2009).

Reports of several trials stated that the duration of procedures in the surgical and the control arms was matched, either by imitating the elements of the surgical procedure or by keeping all patients in the operation room for the same length of time (Moseley et al. 2002; Montgomery et al. 2006; Koutsourelakis et al. 2008; Castro et al. 2010; Gross et al. 2011; Sihvonen et al. 2013; Eid et al. 2014). In some trials, however, the placebo procedure was shorter than the surgical intervention, because it was judged unethical to prolong placebo treatment (Fleischer 1985; Deviere et al. 2005).

Most trials blinded assessors but the surgeon and other staff in the operating room knew the group assignment; however, they did not participate in further treatment, post-operative care or follow-up of the patient (Abbott et al. 2004; Sihvonen et al. 2013; Cotton et al. 2014). In a trial by Thomsen, the post-operative care and assessment was done at a different hospital from the surgery (Thomsen et al. 1981). In the trial by Cotton and colleagues, post-operative care was provided by physicians who were blinded when deciding on treatment, and when this was not sufficient, by an evaluating physician at the study site who was also blinded (Cotton et al. 2014).

The importance of placebo controlled surgical trials

Comparisons of surgical treatments believed to be active with placebo treatments are important because they reduce biased estimates of treatment effects (Wartolowska et al. 2016). Although double-blind randomised trials are costly and often difficult to perform, their cost is a fraction of the costs to patients and health services of ineffective, invasive and harmful treatments.

Half a century after surgeons and others began to recognise the value of placebo controlled trials of surgery (Thomsen et al. 1981), there are still no regulations requiring surgical procedures to be shown to be efficacious and safe in the same way there are for drugs. The consequence is much unjustified surgery being used at inevitable risk to the wellbeing of patients and expense to health services. Sixty-six placebo-controlled trials in surgery were reported in the first 15 years of the 21st century compared with only 19 during the whole of the second half of the 20th century. This recent increase in the use of placebo controlled trials of surgery is an encouraging sign that biased estimates of the effects of surgical treatment are being recognised and addressed.

Lind Library article has been republished in the Journal of the Royal Society of Medicine 2018;111:177-182. Print PDF

References

Abbott J, Hawe J, Hunter D, Holmes M, Finn P, Garry R (2004). Laparoscopic excision of endometriosis: a randomized, placebo-controlled trial. Fertility & Sterility 82: 878–884.

Alanson E (1782). Practical observations on amputation, and the after-treatment, 2nd edn. London: Joseph Johnson.

Bilguer JU (1764). A dissertation on the inutility of the amputation of limbs. London.

Bradley JD, Heilman DK, Katz BP, Gsell P, Wallick JE, Brandt KD (2002). Tidal irrigation as treatment for knee osteoarthritis: a sham-controlled, randomized, double-blinded evaluation. Arthritis Rheum 46:100–108.

Buchbinder R, Osborne RH, Ebeling PR, Wark JD, Mitchell P, Wriedt C, Graves S, Staples MP, Murphy B (2009). A randomized trial of vertebroplasty for painful osteoporotic vertebral fractures. New England Journal of Medicine 361:557–568.

Burton MJ (2015). Astley Cooper’s dramatically effective treatment of deafness. JLL Bulletin: Commentaries on the history of treatment evaluation. (https://www.jameslindlibrary.org/articles/astley-coopers-dramatically-effective-treatment-of-deafness/)

Castro M, Rubin AS, Laviolette M, Fiterman J, De Andrade Lima M, Shah PL, Fiss E, Olivenstein R, Thomson NC, Niven RM, Pavord ID, Simoff M, Duhamel DR, McEvoy C, Barbers R, Ten Hacken NH, Wechsler ME, Holmes M, Phillips MJ, Erzurum S, Lunn W, Israel E, Jarjour N, Kraft M, Shargill NS, Quiring J, Berry SM, Cox G; AIR2 Trial Study Group. (2010). Effectiveness and safety of bronchial thermoplasty in the treatment of severe asthma: a multicenter, randomized, double-blind, sham-controlled clinical trial. American Journal of Respiratory and Critical Care Medicine 181:116–24.

Chalmers I (2010). Why the 1948 MRC trial of streptomycin used treatment allocation based on random numbers. JLL Bulletin: Commentaries on the history of treatment evaluation. (https://www.jameslindlibrary.org/articles/why-the-1948-mrc-trial-of-streptomycin-used-treatment-allocation-based-on-random-numbers/)

Chalmers I, Dukan E, Podolsky SH, Davey Smith G (2011). The advent of fair treatment allocation schedules in clinical trials during the 19th and early 20th centuries. JLL Bulletin: Commentaries on the history of treatment evaluation. (https://www.jameslindlibrary.org/articles/the-advent-of-fair-treatment-allocation-schedules-in-clinical-trials-during-the-19th-and-early-20th-centuries/)

Cheselden W (1740). The anatomy of the human body. 5th edition. London: William Bowyer.

Cobb LA, Thomas GI, Dillard DH, Merendino KA, Bruce RA (1959). An evaluation of internal-mammary-artery ligation by a double-blind technic. New England Journal of Medicine 260:1115–1118.

Commission Royale. Bailly A (1784). Rapport des commissaires chargés par le Roi, de l’examen du magnétisme animale. Imprimé par ordre du Roi. Paris: A Paris, de L’Imprimerie Royale.

Cook JA (2009). The challenges faced in the design, conduct and analysis of surgical randomised controlled trials. Trials 10:9.

Cooper A (1801). Farther observations on the effects which take place from the destruction of the Membrana Tympani of the ear; with an account of the operation for the removal of a particular species of deafness. Philosophical Transactions of the Royal Society, Part 1:435-451.

Cotton PB, Durkalski V, Romagnuolo J, Pauls Q, Fogel E, Tarnasky P, Aliperti G, Freeman M, Kozarek R, Jamidar P, Wilcox M, Serrano J, Brawman-Mintzer O, Elta G, Mauldin P, Thornhill A, Hawes R, Wood-Williams A, Orrell K, Drossman D, Robuck P (2014). Effect of endoscopic sphincterotomy for suspected sphincter of Oddi dysfunction on pain-related disability following cholecystectomy: the EPISOD randomized clinical trial. JAMA 311:2101-09.

Devière J, Costamagna G, Neuhaus H, Voderholzer W, Louis H, Tringali A, Marchese M, Fiedler T, Darb-Esfahani P, Schumacher B (2005). Nonresorbable copolymer implantation for gastroesophageal reflux disease: A randomized sham-controlled multicenter trial. Gastroenterology 128:532–540.

Dimond EG, Kittle CF, Crockett JE (1960). Comparison of internal mammary artery ligation and sham operation for angina pectoris. American Journal of Cardiology 5:483-486.

Donaldson IML (2016). Antoine de Lavoisier’s role in designing a single-blind trial to assess whether ‘Animal Magnetism’ exists. JLL Bulletin: Commentaries on the history of treatment evaluation. (https://www.jameslindlibrary.org/articles/antoine-de-lavoisiers-role-in-designing-a-single-blind-trial-to-assess-whether-animal-magnetism-exists/)

Eid GM, McCloskey CA, Eagleton JK, Lee LB, Courcoulas AP (2014). StomaphyX vs a sham procedure for revisional surgery to reduce regained weight in Roux-en-Y gastric bypass patients : a randomized clinical trial. JAMA Surgery 149:372–9.

Ergina PL, Cook JA, Blazeby JM, Boutron I, Clavien PA, Reeves BC, Seiler CM (2009). Challenges in evaluating surgical innovation. Lancet 374:1097–1104.

Fairbank J (1999). Randomised controlled trials in surgery. Lancet 354:257.

Fleischer D (1985). Endoscopic Nd:YAG laser therapy for active esophageal variceal bleeding. A randomized controlled study. Gastrointestinal Endoscopy 31:4–9.

Freed CR, Greene PE, Breeze RE, Tsai WY, DuMouchel W, Kao R, Dillon S, Winfield H, Culver S, Trojanowski JQ, Eidelberg D, Fahn S (2001). Transplantation of embryonic dopamine neurons for severe Parkinson’s disease. New England Journal of Medicine 344:710–719.

Freeman BJC, Fraser RD, Cain CMJ, Hall DJ, Chapple DCL (2006). A randomized, double-blind, controlled trial: intradiscal electrothermal therapy versus placebo for the treatment of chronic discogenic low back pain. Spine 31:1637–8.

Freitas D, Donato A, Monteiro JG (1985). Controlled trial of liquid monopolar electrocoagulation in bleeding peptic ulcers. American Journal of Gastroenterology 80:853–7.

Gillespie MB, Wylie PE, Lee-Chiong T, Rapoport DM (2011). Effect of palatal implants on continuous positive airway pressure and compliance. Otolaryngology – Head and Neck Surgery 144:230–6.

Gøtzsche PC, Hrobjartsson A (2004). Is the placebo powerless ? Update of a systematic review with 52 new randomized trials comparing placebo with no treatment. Journal of Internal Medicine 256:91–100.

Gross RE, Watts RL, Hauser RA, Bakay RAE, Reichmann H, von Kummer R, Ondo WG, Reissig E, Eisner W, Steiner-Schulze H, Siedentop H, Fichte K, Hong W, Cornfeldt M, Beebe K, Sandbrink R (2011). Intrastriatal transplantation of microcarrier-bound human retinal pigment epithelial cells versus sham surgery in patients with advanced Parkinson’s disease: A double-blind, randomised, controlled trial. Lancet Neurology 10:509–519.

Guthrie GJ (1815). A treatise on gun-shot wounds. London: Burgess and Hill.

Hartigan P (1994). Sclerotherapy for male alcoholic cirrhotic patients who have bled from esophageal varices: Results of a randomized, multicenter clinical trial. Hepatology 20:618–625.

Haygarth J (1800). Of the imagination, as a cause and as a cure of disorders of the body: exemplified by fictitious tractors, and epidemical convulsions. Bath: R. Crutwell.

Hidalgo de Agüero B (1604). Thesoro de la verdadera cirugía y vía particular contra la común. Sevilla: Francisco Pérez, 1604, pp. 32v.

Hill B (1879). Clinical lectures on stricture of the urethra. BMJ 2:883-886.

Hrobjartsson A, Emanuelsson F, Thomsen ASS, Hilden J, Brorson S, (2014). Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. International Journal of Epidemiology 43:1272–1283.

Hunter J (1794). A treatise on the blood, inflammation and gun-shot wounds. London: G Nichol.

Hutchinson AC (1817). Some further observations on the subject of the proper period for amputating in gun-shot wounds. London: Callow, p 16.

Ike RW, Arnold WJ, Rothschild EW, Shaw HL (1992). Tidal irrigation versus conservative medical management in patients with osteoarthritis of the knee: a prospective randomized study. Tidal Irrigation Cooperating Group. Journal of Rheumatology 19:772–9.

Kallmes DF, Comstock BA, Heagerty PJ, Turner JA, Wilson DJ, Diamond TH, Edwards R, Gray LA, Stout L, Owen S, Hollingworth W, Ghdoke B, Annesley-Williams DJ, Ralston SH, Jarvik JG (2009). A randomized trial of vertebroplasty for osteoporotic spinal fractures. New England Journal of Medicine 361:569–79.

Kaptchuk TJ (2011). A brief history of the evolution of methods to control observer biases in tests of treatments. JLL Bulletin: Commentaries on the history of treatment evaluation. (https://www.jameslindlibrary.org/articles/a-brief-history-of-the-evolution-of-methods-to-control-of-observer-biases-in-tests-of-treatments/)

Kapural L, Vrooman B, Sarwar S, Krizanac-Bengez L, Rauck R, Gilmore C, North J, Girgis G, Mekhail N (2013). A Randomized, Placebo-Controlled Trial of Transdiscal Radiofrequency, Biacuplasty for Treatment of Discogenic Lower Back Pain. Pain Medicine. (United States) 14:362–373.

Keith T (1878). Results of ovariotomy before and after antiseptics. BMJ 2:590-593. doi:10.1136/bmj.2.929.590

Koutsourelakis I, Georgoulopoulos G, Perraki E, Vagiakis E, Roussos C, Zakynthinos SG (2008). Randomised trial of nasal surgery for fixed nasal obstruction in obstructive sleep apnoea. European Respiratory Journal 31:110–7. doi:10.1183/09031936.00087607

Langton J (1899). A discussion on the treatment of hernia in children. BMJ 2:470-472.

Lavoisier A-L de (1784). Undated documents contained in: Mémoires de Lavoisier, Oeuvres, Tome III. Paris: Imprimerie Impériale, 1865. p 509.

Leon MB, Kornowski R, Downey WE, Weisz G, Baim DS, Bonow RO, Hendel RC, Cohen DJ, Gervino E, Laham R, Lembo NJ, Moses JW, Kuntz RE (2005). A blinded, randomized, placebo-controlled trial of percutaneous laser myocardial revascularization to improve angina symptoms in patients with severe coronary disease. Journal of the American College of Cardiology 46:1812–9.

Lindor KD, Hughes RW, Ilstrup DM, Jensen MD (1987). Intragastric balloons in comparison with standard therapy for obesity – a randomized, double-blind trial. Mayo Clinic Proceedings 62:992–6.

Livingston KE (1953). Cingulate cortex isolation for the treatment of psychoses and psychoneuroses. Research Publications- Association for Research in Nervous and Mental Diseases 31:374-8.

MacLeod IA, Mills PR, MacKenzie JF, Joffe SN, Russell RI, Carter DC (1983). Neodymium yttrium aluminium garnet laser photocoagulation for major haemorrhage from peptic ulcers and single vessels: a single blind controlled study. BMJ (Clin Res Ed) 286:345-8.

McLeod RS (1999). Issues in surgical randomized controlled trials. World Journal of Surgery 23:1210–4.

Mathus-Vliegen EM, Tytgat GN, Veldhuyzen-Offermans EA (1990). Intragastric balloon in the treatment of super-morbid obesity. Double-blind, sham-controlled, crossover evaluation of 500-milliliter balloon. Gastroenterology 99:362–9.

Maurer JT, Sommer JU, Hein G, Hörmann K, Heiser C, Stuck BA (2012). Palatal implants in the treatment of obstructive sleep apnea: a randomised, placebo-controlled single-centre trial. European Archives of Otorhinolaryngology 269:1851–6.

Medical Research Council (1948). Streptomycin treatment of pulmonary tuberculosis: a Medical Research Council investigation. BMJ 2:769-782.

Merchan EC, Galindo E (1993). Arthroscope-guided surgery versus nonoperative treatment for limited degenerative osteoarthritis of the femorotibial joint in patients over 50 years of age: a prospective comparative study. Arthroscopy 9:663–7.

Meshkinpour H, Hsu D, Farivar S (1988). Effect of gastric bubble as a weight reduction device: a controlled, crossover study. Gastroenterology 95:589–92.

Monro A (1737). Remarks on the amputations of the larger extremities. In: Medical essays and observations. Edinburgh: T & W Ruddimans.

Montgomery M, Hakanson B, Ljungqvist O, Ahlman B, Thorell A, Håkanson B (2006). Twelve months’ follow-up after treatment with the EndoCinch endoscopic technique for gastro-oesophageal reflux disease: A randomized, placebo-controlled study. Scandinavian Journal of Gastroenterology 41:1382–1389.

Moseley J.B, O’Malley K, Petersen NJ, Menke TJ, Brody BA. Kuykendall DH, Hollingsworth JC, Ashton CM, Wray NP (2002). A Controlled Trial of Arthroscopic Surgery for Osteoarthritis of the Knee. New England Journal of Medicine 347:81–88.

Nease CJ, Krempl GA (2004). Radiofrequency treatment of turbinate hypertrophy : A A randomized, blinded, placebo-controlled clinical trial. Otolaryngol Head Neck Surg 130: 291–299.

Olanow WC, Goetz CG, Kordower JH, Stoessl JA, Sossi V, Brin MF, Shannon KM, Nauert MG, Perl DP, Godbold J, Freeman TB (2003). A double-blind controlled trial of bilateral fetal nigral transplantation in Parkinson’s disease. Annals of Neurology 54:403–14.

Pauza KJ, Howell S, Dreyfuss P, Peloza JH, Dawson K, Bogduk N (2004). A randomized, placebo-controlled trial of intradiscal electrothermal therapy for the treatment of discogenic low back pain. Spine 4:27–35.

Poisson, Dulong, Larrey et Double (1835). Recherches de statistique sur l’affection calculeuse, par M. le docteur Civiale. Comptes rendus hebdomadaires des séances de l’Académie de Sciences [Statistical research on (urinary) stone, by M. Dr Civiale]. Paris: Bachelier, 167-177.

Roehrborn CG, Gange SN, Shore ND, Giddens JL, Bolton DM, Cowan BE, Brown BT, McVary KT, Te AE, Gholami SS, Rashid P, Moseley WG, Chin PT, Dowling WT, Freedman SJ, Incze PF, Coffield KS, Borges FD, Rukstalis DB (2013). The prostatic urethral lift for the treatment of lower urinary tract symptoms associated with prostate enlargement due to benign prostatic hyperplasia: The L.I.F.T. study. Journal of Urology, 190:2161–2167

Rothstein R, Filipi C, Caca K, Pruitt R, Mergener K, Torquati A, Haber G, Chen Y, Chang K, Wong D, Deviere J, Pleskow D, Lightdale C, Ades A, Kozarek R, Richards W, Lembo A (2006). Endoscopic Full-Thickness Plication for the Treatment of Gastroesophageal Reflux Disease: A Randomized, Sham-Controlled Trial. Gastroenterology 131:704–712.

Sackett DL (2007). Commentary: Measuring the success of blinding in RCTs: Don’t, must, can’t or needn’t? International Journal of Epidemiology 36:664–665.

Salem M, Rotevatn S, Stavnes S, Brekke M, Vollset SE, Nordrehaug JE (2004). Usefulness and safety of percutaneous myocardial laser revascularization for refractory angina pectoris. American Journal of Cardiology 93:1086–1091.

Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, Ioannidis JPA, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JAC (2012). Influence of reported study design characteristics on intervention effect estimates from randomized controlled trials. Annals of Internal Medicine 157:429-438.

Sihvonen R, Paavola M, Malmivaara A, Itälä A, Joukainen A, Nurmi H, Kalske J, Järvinen, TLN (2013). Arthroscopic Partial Meniscectomy versus Sham Surgery for a Degenerative Meniscal Tear. New England Journal of Medicine 369:2515–24.

Solis C (2011). Bartolomé Hidalgo de Agüero’s 16th century, evidence-based challenge to the orthodox management of wounds. JLL Bulletin: Commentaries on the history of treatment evaluation. (https://www.jameslindlibrary.org/articles/bartolome-hidalgo-de-agueros-16th-century-evidence-based-challenge-to-the-orthodox-management-of-wounds/)

Stone GW, Teirstein PS, Rubenstein R, Schmidt D, Whitlow PL, Kosinski EJ, Mishkel G, Power JA (2002). A prospective, multicenter, randomized trial of percutaneous transmyocardial laser revascularization in patients with nonrecanalizable chronic total occlusions. Journal of the American College of Cardiology 39:1581–1587.

Sutton CJ, Ewen SP, Whitelaw N, Haines P (1994). Prospective, randomized, double-blind, controlled trial of laser laparoscopy in the treatment of pelvic pain associated with minimal, mild, and moderate endometriosis. Fertility and Sterility 62:696–700.

Thomsen J, Bretlau P, Tos M, Johnsen NJ (1981). Meniere’s disease: Endolymphatic sac decompression compared with sham (placebo) decompression. Annals of the New York Academy of Sciences 374:820–830.

Tröhler U (2013). William Cheselden’s 1740 presentation of data on age-specific mortality after lithotomy. JLL Bulletin: Commentaries on the history of treatment evaluation. (https://www.jameslindlibrary.org/articles/william-cheseldens-1740-presentation-of-data-on-age-specific-mortality-after-lithotomy/)

Ware JE (1993). Measuring patients’ views: the optimum outcome measure. SF-36: a medical valid, reliable assessment of health from the patient’s point of view. BMJ 306:1429–30.

Wartolowska K, Judge A, Hopewell S, Collins GS, Dean BJ, Rombach I, Brindley D, Savulescu J, Beard DJ, Carr AJ (2014). Use of placebo controls in the evaluation of surgery: systematic review. BMJ 348:g3253.

Wartolowska KA, Collins GS, Hopewell S, Judge A, Dean BJF, Rombach I, Beard DJ, Carr AJ (2016). Feasibility of Surgical Randomised Controlled Trials with a Placebo Arm: A Systematic Review. BMJ Open 6:e010194.

Webster Fox L (1894). Immediate capsulotomy following the removal of cataract. JAMA 22:837-39.

Wente MN, Seiler CM, Uhl W, Büchler MW (2003). Perspectives of evidence-based surgery. Digestive Surgery 20:263–9.

Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA (2008). Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336: 601–605.