and dealing with incomplete and biased reporting of clinical research: from Francis Bacon to the World Health Organisation.
Cite as: Dickersin K, Chalmers I (2010). Recognising, investigating and dealing with incomplete and biased reporting of clinical research: from Francis Bacon to the World Health Organisation. James Lind Library (www.jameslindlibrary.org).
Author contact details: Dr Kay Dickersin, Dept of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Mail Rm W5010, Baltimore, Maryland 21205, USA. Email: email@example.com
Why is incomplete reporting of research a problem?
Failure to publish is also unethical. Participants in clinical research are usually assured that their involvement will contribute to knowledge; but this does not happen if the research is not reported publically and accessibly. Moreover, failure to publish is simply a waste of precious research and other resources (Chalmers and Glasziou 2009a). Every year an estimated 12,000 clinical trials which should have been fully reported are not, wasting just under a million tonnes of carbon dioxide annually - the carbon emission equivalent of about 800,000 round trip flights between London and New York (Chalmers and Glasziou 2009b).
In brief, failure to report research findings is not only unscientific but also unethical (Chalmers 1985; 1990; Antes and Chalmers 2003; World Medical Association 2008). How did this problem come to be recognised and investigated, and what steps are being taken today to deal with it?
Evidence of biased reporting of studies
Failure to publish research findings is pervasive (Dickersin 2005; Song et al. 2010). Studies demonstrating failure to publish have included research conducted in many countries, including Australia, France, Germany, Spain, Switzerland, the United Kingdom and the United States. For example, an analysis of follow-up studies based on 29,729 reports of research made available only in abstract form found that fewer than half of the studies went on to full publication, and that positive results were positively associated with full publication, regardless of whether 'positive' results had been defined as any 'statistically significant' result or as ‘a result favoring the experimental treatment’ (Scherer et al. 2007).
For example, the bronze statue of Albert Einstein is inscribed with a quotation from a letter that he wrote on 3 March 1954, for a conference of the Emergency Civil Liberties Committee:
In 1959, the father of medical statistics in Britain, Austin Bradford Hill, wrote:
And in the same year, Seymour Kety, an American psychiatrist wrote:
Although the importance of reporting biases had been recognised for centuries, it was not until the second half of the 20th century that researchers began to investigate the phenomenon. The impetus for these investigations came from the development of research synthesis, first by social scientists, then by health researchers (Hunt 1997; Chalmers I et al. 2002; O’Rourke 2006). Unsurprisingly, researchers who have exposed reporting biases are often those who have also been involved in the application of methods for research synthesis.
Investigations of biased reporting of research began with surveys of journal articles, which revealed improbably high proportions of published studies showing statistically significant differences (Sterling 1959; Smart 1964; Chalmers TC et al. 1965). Subsequent surveys of authors and peer reviewers showed that research that had yielded ‘negative’ results were less likely than others to be submitted or recommended for publication (Greenwald 1975; Coursol and Wagner 1986; Shadish et al. 1989; Dickersin et al. 1987). These findings were reinforced by the results of experimental studies, which showed that studies with no reported statistically significant differences were less likely to be accepted for publication (Mahoney 1977; Peters and Ceci 1982; Epstein 1990).
The most direct evidence of publication bias in the medical field has come from following up cohorts of studies identified at the time of funding (Dickersin and Min 1993), ethics approval (Easterbrook et al. 1991; Dickersin et al. 1992) submission for drug licences (Hemminki 1980; Melander et al. 2003; Turner et al. 2008), or when they were reported in summary form, for example in conference abstracts (Scherer et al. 1994; 2007). Systematic reviews of this body of evidence have shown that ‘positive findings’ are the principal factor associated with subsequent publication: a systematic review of data from five cohort studies following research projects from inception found that, overall, the odds of publication for studies with ‘positive’ findings was about two and a half times greater than the odds of publication of studies with ‘negative’ or ‘null’ results, and that study results were the principal factor explaining these differences in reporting (Dwan et al. 2008; Hopewell et al. 2009; Song et al. 2009).
Even when studies are eventually reported in substantive publications, ‘negative’ findings take longer to appear in print (Stern and Simes 1997; Ioannidis 1998; Misakian and Bero 1998; Hopewell et al. 2007a): on average, clinical trials with ‘positive results’ are published about a year sooner than trials with ‘null or negative results’. There is also evidence that, compared to negative or null results, statistically significant results tend to be published in journals with higher impact factors (Easterbrook et al. 1991), and that publication in the mainstream (‘non-grey’) literature is associated with an overall 9 per cent larger estimate of treatment effects compared to reports in the grey literature (Hopewell et al. 2007b). Articles reporting negative findings for efficacy, or reporting adverse events associated with an exposure, may be published but ’hidden’ in harder to access sources (Bero et al. 1996). Furthermore, even when studies initially published in abstract form are published in full, ‘negative’ results are less likely to be published in high impact journals than ‘positive’ results (Timmer et al. 2002).
Selective reporting of suspected or confirmed adverse treatment effects is an area for particular concern because of the potential for patient harm. In a study of adverse drug events submitted to Scandinavian drug licensing authorities, subsequently published studies were less likely than unpublished studies to have recorded adverse events (Hemminki 1980). The lay and scientific media have drawn attention to failure to accurately report adverse events for drugs, for example, of selective serotonin uptake inhibitors for depression (Healy 2006; Bass 2008), rosiglitazone for diabetes (Drazen et al. 2007), and rofecoxib for arthritis pain (de Angelis and Fontanarosa 2008).
Biased reporting of data within studies
Biased outcome reporting has also been shown in a comparison with subsequent publications of data about 12 antidepressant agents submitted for review to the Food and Drug Administration (FDA) (Turner et al. 2008). Only 31 per cent of the 74 FDA-registered studies had been published, and publication was associated with a ‘positive’ outcome (as determined by the FDA). Studies that the FDA had considered ‘negative’ or ‘questionable’ (n=36) were either not published (22 studies), reported with a positive interpretation (11 studies), or reported in a manner consistent with the FDA interpretation (3 studies). In summary, evidence from the published literature suggested that 94 per cent of studies had positive findings, while the FDA analysis concluded that only 51 per cent had positive findings.
It is now also well-established that biased reporting of research studies is associated with the sources of funding. In particular, research funded by the pharmaceutical industry has been shown to be less likely to be published than research funded from other sources (Lexchin et al. 2003; Sismondo 2008), and that studies sponsored by pharmaceutical companies are more likely to have outcomes favoring the sponsor than studies with other sponsors (Als-Neilsen et al. 2003; Bhandari et al. 2004). There are several possible explanations for the association between industry support and failure to publish ‘negative’ results. Industry may selectively publish findings supporting a product’s efficacy. It is also possible that industry is more likely to undertake studies with a high likelihood of a positive outcome, for example, by selecting a comparison population likely to yield results favoring the product (Djulbegovic et al. 2000; Mann and Djulbegovic 2004). Neither of these actions is ethical.
The practice of hiring a commercial firm to write up the results from a clinical trial is common in industry trials (Sismondo 2007). It has been estimated that 75% of industry-initiated studies approved by two ethics committees in Denmark had ghost authors (Gøtzsche et al. 2007). In these cases, the named authors listed rarely included the hired writer. The World Association of Medical Editors has made it clear it considers such ghost authorship to be dishonest (http://www.wame.org/resources/policies accessed August 1, 2008). Unnamed, paid medical writers may be asked to address commercial interests in the way that research methods and results are presented. When the proportion of paid medical writers is sufficiently large, the literature, and thus opinion about the drug, may be influenced (Healy and Cattell 2003).
Because industry is the main funder of clinical research, it must inevitably shoulder a high proportion of the blame for this unscientific and unethical behaviour. The responsibility for biased reporting of clinical research does not lie solely with industry, however. As long ago as 1998, the Ethics Committee of the Faculty of Pharmaceutical Medicine, which represents physicians working in industry in particular, declared that: “Pharmaceutical physicians … have a particular ethical responsibility to ensure that the evidence on which doctors should make their prescribing decisions is freely available…the outcome of all clinical trials on a medicine should be reported" (Faculty of Pharmaceutical Medicine 1998).
Ad Hoc Working Party of the International Collaborative Group on Clinical Trials Registries (1993). International Collaborative Group on Clinical Trials Registries. Position paper and consensus recommendations on clinical trial registries. Clinical Trials and Meta-analysis 29:255-266.
Alanson E (1782). Practical observations on amputation, and the after-treatment, 2nd Ed. London: Joseph Johnson.
Al-Marzouki S, Roberts I, Evans S, Marshall T (2008). Selective reporting in clinical trials: analysis of trial protocols accepted by The Lancet. Lancet 372:201.
Als-Nielsen B, Chen W, Gluud C, Kjærgaard LL (2003). Association of funding and conclusions in randomized drug trials: A reflection of treatment effects or adverse events? JAMA 290:921-928.
Antes G, Chalmers I (2003). Under-reporting of clinical trials is unethical. Lancet 361:978- 979.
Bacon F (1645). Franc. Baconis de Verulamio / Summi Angliae Cancellarii /Novum organum scientiarum. [Francis Bacon of St. Albans Lord Chancellor of England. A 'New Instrument' for the sciences] Lugd. Bat: apud Adrianum Wiingaerde, et Franciscum Moiardum. Aphorism XLVI (pages 45-46).
Bass A (2008). Side Effects: A Prosecutor, a Whistleblower, and a Bestselling Antidepressant on Trial. Boston: Algonquin.
Bero LA, Rennie D (1996). Influences on the quality of published drug studies. Int J Technol Assess Health Care 12:209-237.
Bhandari M, Busse JW, Jackowski D, Montori VM, Schünemann H, Sprague S, Mears D, Schemitsch EH, Heels-Ansdell D, Devereaux PJ (2004). Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ 170:477-480.
BioMedCentral. Information for authors: Publish your study protocols. Available at http://www.biomedcentral.com/info/authors/protocols. Accessed 8 March 2010.
BiomedCentral (2002). Journal of Negative Results in Biomedicine. http://www.jnrbm.com/info/about/
Boyle R (1661). Cited in Hall MB. In defense of experimental essays. In: Robert Boyle on natural philosophy. Bloomington: Indiana University Press:119-131. [Boyle R (1661). In defense of Experimental Essays. In: Certain Physiological Essays.]
Chalmers I (1985). Proposal to outlaw the term 'negative trial'. BMJ 1985;290:1002.
Chalmers I (1990). Underreporting research is scientific misconduct. JAMA 263:1405-1408.
Chalmers I (2006). From optimism to disillusion about commitment to transparency in the medico-industrial complex. Journal of the Royal Society of Medicine 2006;99:337-341.
Chalmers I, Hedges LV, Cooper H (2002). A brief history of research synthesis. Evaluation and the Health Professions 25:12-37.
Chalmers I, Glasziou P (2009a). Avoidable waste in the production and reporting of research evidence. Lancet 2009;374:86-89. doi:10.1016/S0140-6736(09)60329-9.
Chalmers I, Glasziou PG (2009b). The environmental, scientific and ethical scandal of biased under-reporting of research. http://www.bmj.com/cgi/eletters/339/oct30_1/b4187#224821.
Chalmers TC, Koff RS, Grady GF (1965). A note on fatality in serum hepatitis. Gastroenterology 49:22-26.
Chan A-W (2008). Bias, spin, and misreporting: time for full access to trial protocols and results. PLoS Medicine 5(11): e230. DOI:10.1371/journal.pmed.0050230.
Chan AW, Krleža-Jeric K, Schmid I, Altman D (2004a). Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 171:735-40.
Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG (2004b). Empirical Evidence for selective reporting of outcomes in randomized trials. Comparison of protocols to published articles. JAMA 291:2457-2465.
Coursol A, Wagner EE (1986). Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias. Professional Psychology: Research and Practice 17:136-137.
Cowley AJ, Skene A, Stainer K, Hampton JR (1993). The effect of lorcainide on arrhythmias and survival in patients with acute myocardial infarction: an example of publication bias. International Journal of Cardiology 40:161-166.
DeAngelis CD, Fontanarosa PB (2008). Impugning the integrity of medical science, the adverse effects of industry influence. JAMA 299:1833.
Dickersin K (2005). Publication bias: Recognizing the problem, understanding its origins and scope, and preventing harm. In: Rothstein H, Sutton A, Borenstein M (eds). Publication bias in meta-analysis: prevention, assessment, and adjustments. London: Wiley, pp 11-33.
Dickersin K (1997). How important is publication bias? A synthesis of available data. AIDS Educ Prev 9 (1 Suppl):15-21.
Dickersin K, Min Y-I (1993). NIH clinical trials and publication bias. Online Journal of Current Clinical Trials, 28 April [Doc No. 50]
Dickersin K, Rennie D (2003). Registering clinical trials. JAMA 290:516-523.
Dickersin K, Chan S, Chalmers TC, Sacks HS, Smith H (1987). Publication bias and clinical trials. Control Clin Trials 8:343-53.
Dickersin K, Min YI, Meinert CL (1992). Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 267:374-8.
Dickersin K, Olson CM, Rennie D, Cook D, Flanagin A, Zhu Q, Reiling J, Pace B (2002). Association between time interval to publication and statistical significance. JAMA 287:2829-2831.
Dickersin K, Ssemanda E, Mansell C, Rennie D (2007). What do JAMA editors say when they discuss manuscripts that they are considering for publication? Developing a schema for classifying the content of editorial discussion. BMC Medical Research Methodology 7:44.
Djulbegovic B, Lacevic M, Cantor A, Fields KK, Bennett CL, Adams JR, Kuderer NM, Lyman GH (2000). The uncertainty principle and industry-sponsored research. Lancet 356:635-638.
Drazen JM, Morrissey S, Curfman GD (2007). Rosiglitazone--continued uncertainty about safety. N Engl J Med 357:63-64.
Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A-W, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, Ghersi D, Ioannidis JPA, Simes J, Williamson PR (2008). Systematic Review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE 3:e3081.
Earp JR (1927). The need for reporting negative results. JAMA 88:119.
Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR (1991). Publication bias in clinical research. Lancet 337:867-872.
Editorial (1909). The reporting of unsuccessful cases. Boston Medical and Surgical Journal 161:263-264.
Editorial (1962). Negative results section. JAMA 181:42-43.
Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G (1997). Language bias in randomised controlled trials published in English and German. Lancet 350:326–329.
Einstein A (1954). Statement for a conference of the Emergency Civil Liberties Committee, 3 March. Albert Einstein Archives, Hebrew University of Jerusalem, 28-1025.
Epstein WM (1990). Confirmational response bias among social work journals. Science, Technology, & Human Values 15:9-37.
Faculty of Pharmaceutical Medicine (1998). Ethical Issues Working Group. Ethics in pharmaceutical medicine. International Journal of Pharmaceutical Medicine 12:193-8.
Feynman RP (1985). Surely You’re Joking Mr. Feynman. New York: Norton.
Ghersi D, Pang T (2008). En route to international clinical trial transparency. Lancet 372:1531-2.
Godlee F, Dickersin K (2003). Bias, subjectivity, chance, and conflict of interest in editorial decisions. In: Godlee F, Jefferson T, eds. Peer review in health sciences, 2nd edition. London: BMJ Books.
Gøtzsche PC (1987). Reference bias in reports of drug trials. BMJ 195:654-656.
Gøtzsche PC, Hrobjartsson A, Johansen HK, Haahr MT, Altman DG, Chan A-W (2007). Ghost authorship in industry-initiated randomised trials. PloS Medicine 4:47-52.
Gould SJ (1987). Urchin in the Storm. Essays about Books and Ideas. New York: Norton.
Greenwald AG (1975). Consequences of prejudice against the null hypothesis. Psychol Bull 82:1-20.
Hahn S, Williamson PR, Hutton JL (2002). Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee. Journal of Evaluation in Clinical Practice 8:353-359.
Healy D, Cattell D (2003). Interface between authorship, industry and science in the domain of therapeutics. Br J Psychiatry 183:22-27.
Healy D (2006). Did regulators fail over selective serotonin reuptake inhibitors? BMJ 333:92-95.
Hemminki E (1980). Study of information submitted by drug companies to licensing authorities. BMJ 280:833–836.
Hetherington J, Dickersin K, Chalmers I, Meinert CL (1989). Retrospective and prospective identification of unpublished controlled trials: lessons from a survey of obstetricians and pediatricians. Pediatrics 84:374-380.
Hill AB (1959). Discussion of a paper by DJ Finney. Journal of the Royal Statistical Society, Series A 119:19-20.
Hopewell S, Clarke M, Stewart L, Tierney J (2007a). Time to publication for results of clinical trials. Cochrane Database of Systematic Reviews, Issue 2. Art. No.: MR000011. DOI: 10.1002/14651858.MR000011.pub2.
Hopewell S, McDonald S, Clarke M, Egger M (2007b). Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database of Systematic Reviews, Issue 2. Art. No.: MR000010. DOI: 10.1002/14651858.MR000010.pub3.
Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K (2009). Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews, Issue 1. Art. No.: MR000006. DOI: 10.1002/14651858.MR000006.pub3.
Horton R (1997). Pardonable revisions and protocol reviews. Lancet 349:6.
Ioannidis JP (1998). Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 279:281-286.
Juni P, Holenstein F, Sterne J, Bartlett C, Egger M (2002). Direction and impact of language bias of controlled trials: An empirical study. International Journal of Epidemiology 31: 115-123.
Kety S (1959). In: Cole JO, Gerard RW, eds. Psychopharmacology. Problems in Evaluation. Publication 583. Washington DC: National Academy of Sciences, pp 651-2.
Kirkham J, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR (2009). The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ;340:c365. DOI: 10.1136/BMJ.c365.
Kjaergaard LL, Gluud C (2002). Citation bias of hepato-biliary randomized clinical trials. J Clin Epidemiol 55:407-410.
Krleža-Jeric K, Chan A-W, Dickersin K, Sim I, Grimshaw J, Gluud C, for the Ottawa Group (2005). Principles for international registration of protocol information and results from human trials of health-related interventions. Ottawa Statement (Part 1) BMJ 330:956-958.
Lexchin J, Bero LA, Djulbegovic B, Clark O (2003). Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 326:1-10.
Light RJ, Pillemer DB (1984). Summing up. Cambridge:Harvard University Press.
Mahoney MJ (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research 1:161-175.
Mann H, Djulbegovic B (2004). Why comparisons must address genuine uncertainties. James Lind Library (www.jameslindlibrary.org).
Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P (2009). Comparison of registered and published primary outcomes in randomized controlled trials. JAMA 302:977-984.
Meinert CL (1988). Toward prospective registration of clinical trials. Controlled Clin Trials. 9:1-5.
Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B (2003). Evidence-b(i)ased medicine – selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 326:1171-1173.
Miller JD (2010). Registering clinical trial results: the next step. JAMA 303: 773-774.
Misakian AL, Bero LA (1998). Publication bias and research on passive smoking. Comparison of published and unpublished studies. JAMA 280:250-253.
Nieminen P, Rucker G, Miettunen J, Carpenter J, Schumacher M (2007). Statistically significant papers in psychiatry were cited more often than others. J Clin Epidemiol 60:939-946.
Olson CM, Rennie D, Cook D, Dickersin K, Flanagin A, Hogan J, Zhu Q, Reiling J, Pace B (2002). Publication bias in editorial decision making. JAMA 287:2825-2828.
O’Rourke K (2006). An historical perspective on meta-analysis: dealing quantitatively with varying study results. The James Lind Library (www.jameslindlibrary.org).
Peters D, Ceci S (1982). Peer review practice of psychologic journals: The fate of published articles submitted again. Behav Brain Sci 5:187-195.
Ravnskov U (1992). Frequency of citation and outcome of cholesterol lowering trials. BMJ 305:717.
Ravnskov U (1995). Quotation bias in reviews of the diet heart idea. J Clin Epidemiol 48:713-719.
Roberts I (1998). An amnesty for unpublished trials BMJ 317:763-764
Rochon PA, Gurwitz JH, Simms RW, Fortin PR, Felson DT, Minaker KL, Chalmers TC (1994). A study of manufacturer supported trials of non-steroidal anti-inflammatory drugs in the treatment of arthritis. Arch Intern Med 154:157-163.
Rosenthal R (1979). The ‘file drawer problem’ and tolerance for null results. Psychological Bulletin 86:638-641.
Ross MG, Hines EM, Nissen SE, Krumholz HM (2006). Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med 6(9): e1000144. DOI:10.1371/journal.pmed.1000144.
Scherer RW, Dickersin K. Langenberg P (1994). Full publication of results initially presented in abstracts. JAMA 272:158-162.
Scherer RW, Langenberg P, Von Elm E (2007). Full publication of results initially presented in abstracts. In: The Cochrane Library. Issue 2. Art. No.: MR000005. DOI: 10.1002/14651858.MR000005.pub3.
Schmidt LM, Gøtzsche PC (2005). Of mites and men: reference bias in narrative review articles: a systematic review. J Fam Practice 54:334-8.
Shadish WR, Doherty M, Montgomery LM (1989). How many studies are in the file drawer? An estimate from the family/marital psychotherapy literature. Clin Psychol Rev 9:589-603.
Shields PG (2000). Publication bias is a scientific problem with adverse ethical outcomes: the case for a section for null results. Cancer Epidemiology, Biomarkers and Prevention 9:771-772.
Siegel J (1990). Editorial review of protocols for clinical trials. N Engl J Med 323: 1355.
Simes RJ (1986). Publication bias: the case for an international registry of clinical trials. J Clin Oncol 4:1529-1541.
Sismondo S (2008). Pharmaceutical company funding and its consequences: A qualitative systematic review. Contemp Clin Trials 29:109-113.
Smart RG (1964). The importance of negative results in psychological research. Canadian Psychologist 5:225-232.
Smith R, Roberts I (1997). An amnesty for unpublished trials. BMJ 315:622.
Song F, Parekh-Bhurke S, Hooper L, Loke YK, Ryder JJ, Sutton AJ, Hing CB, Harvey I (2009). Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies. BMC Med Res Methodol. 26;9:79.
Song F, Parekh S, Hooper L, Loke YK, Ryder JJ, Sutton AJ, Hing CB, Kwok CS, Pang C, Harvey I (2010). Dissemination and publication of research findings: an updated review of related biases. Health Technology Assessment 14(8).
SPIRIT Initiative. http://www.equator-network.org/resource-centre/library-of- health-research-reporting/reporting-guidelines-under-development/ (Accessed February 15, 2010).
Sterling TD (1959). Publication decisions and their possible effects on inferences drawn from tests of significance - or vice versa. Journal of the American Statistical Association 54:30-34.
Stern JM, Simes RJ (1997). Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ 315:640-645.
Sterne J, Egger M, Moher D on behalf of the Cochrane Bias Methods Group, eds (2008). Chapter 10. Addressing reporting biases. In: Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.0 [updated February 2008]. The Cochrane Collaboration, 2008. Available from www.cochrane-handbook.org.
Tramèr M, Reynolds DJ, Moore RA, McQuay HJ (1997). Impact of covert duplicate publication on meta-analysis: A case study. BMJ 315:635-640.
Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358:252-260.
Vandenbroucke JP (1988). Passive smoking and lung cancer: a publication bias? BMJ 296:391-392.
Vedula S, Bero L, Scherer RW, Dickersin K (2009). Outcome reporting in industry-sponsored trials of gabapentin for off-label use. New England Journal of Medicine 361:1963-1971.
Von Elm E, Poglia G, Walder B, Tramèr MR (2004). Different patterns of duplicate publication. An analysis of articles used in systematic reviews. JAMA 291:974-980.
Wager E, Field EA, Grossman L (2003). Good Publication Practice for pharmaceutical companies. Current Medical Research and Opinion 19:149-54.
Whittington CJ, Kendall T, Fonagy P, Cottrell D, Cotgrove A, Boddington E (2004). Selective serotonin reuptake inhibitors in childhood depression: sysyrtamtycm review of published versus unpublished data. Lancet 363:1341-1345.
World Medical Association (2008). World Medical Association. Declaration of Helsinki: ethical principles for medical research involving human subjects.