Introduction

‘Trials of therapy’, in which physicians ‘try out’ treatments and assess patients’ responses, are long-established, common elements of routine medical practice. Because ‘trials of therapy’ are usually informal, they may only be reported if treatments are associated with dramatic changes in a patient’s condition – whether by improvement or deterioration.

Our understanding of bias suggests that informal ‘trials of therapy’ – comparisons of patients’ condition before and after treatment – do not provide a trustworthy basis for inferring treatment effects. More sophisticated comparisons are usually needed: for example, comparing a patient’s responses when treatments are given or withheld (‘crossed over’) and conducting formal assessment of outcomes.

In 1676, Richard Wiseman (a surgeon to King Charles II) reported an unplanned experiment. He had prescribed a pair of laced stockings for a patient suffering from leg oedema. The stockings had reduced the oedema to the extent that the patient “was able to walk to his closet, and take the air in his coach, and was well pleased with them” (Wiseman 1676). However, someone suggested to the patient that the stockings might do him harm, and persuaded him to remove them. His legs swelled up, he became confined to bed again, and developed leg ulcers. Dr Wiseman waited six weeks for the ulcers to heal, restored the laced stockings, with the result that the patient recovered.

A century after Wiseman’s crude crossover trial of laced stockings, Caleb Parry (Rolls 2003), a doctor in Bath, England, published a more formal, planned use of between 2 and 6 crossover periods of variable duration in 13 patients, to compare the purgative effects of three varieties of rhubarb (Parry 1786). Parry was unable to find any advantage of the more costly Turkish rhubarb compared with English rhubarb.

Parry’s ‘trials of therapy’ were important in having used at least 2 crossovers, but he took no steps to ensure that his and his patients’ assessments of the treatment effects was not influenced by his or the patients’ knowledge of the type of rhubarb being given. Fourteen years later, also in Bath, John Haygarth (Booth 2002) compared the effects on rheumatism of a metal ‘tractor’ with a matched wooden (placebo) tractor. This demonstrated that the assumed treatment effects of the metal tractor resulted from patients’ imagination (Haygarth 1800).

Haygarth’s study made clear that informal ‘trials of therapy’ can be plagued by false positives (due to placebo effects, physicians’ and patients’ desires to please, the pre-existing expectations of both parties, and natural history). And they can also result in false negatives (patients destined to deteriorate, and the intervention resulting in them remaining stable). Although, more than a century passed after Haygarth before Paul Martini set out principles for designing unbiased crossover trials in his 69-page book (Martini 1932; Stoll 2004), it appears that it was not until 1953 that serious scientific consideration was given to how controlled trials in individual patients could complement traditional parallel group trials. In 1953, Hogben and Sim (1953) recognized that:

The now current recipe for a clinical trial based on group comparison sets out a balance sheet in which individual variability with respect both to nature and to previous nurture does not appear as an explicit item in the final statement of the account; but such variability of response to treatment may be of paramount interest in practice. (Hogben and Sim 1953).

Trialists conducting parallel group trials using alternate or random allocation had been trying for half a century to deal with the challenge of deducing how to treat individual patients by using estimates of effects in subgroups of participants, but this was only a partial way of addressing the fundamental underlying issue – ascertaining individual responses (Hamburg and Collins 2010).

The experiment reported by Hogben and Sim is a methodological landmark (Appendix), celebrated more than half a century later by republication and commentaries in the International Journal of Epidemiology (Ebrahim 2011; Tabery 2011; Glasziou 2011). One of the commentaries (Glasziou 2011) summarizes the features of the study:

Because they used patient’s self-reported symptoms, they put a particular emphasis on careful blinding: the use of a placebo and keeping both clinical and patient unaware of the sequence of treatments. They were also concerned about the non-specific response to prostigmine so they used two comparators: dexamphetamine (a stimulant) and lactose (as an inert placebo). Their weighted analysis, based on concerns about wash-out and wash-in effects, also appears to be novel. Finally, with a minimum of eight periods for each treatment, they seemed to have set a new record for the number of crossovers in any crossover trial in an individual patient.

Hogben’s and Sim’s paper does not appear to have had an impact – possibly because it was published in a non-clinical journal. Glasziou (2011) identified only 12 citations, and only one of those reported a replication of their methods (in 30 patients in a neurosis unit) (Hare 1955). Thereafter, these two studies and developments in the application of single subject design methodology in the social sciences (Kazdin 2011) appear to have gone unnoticed in the medical community until 1986.

Baskerville et al. (1984) were the first to apply principles of adaptive design to the N of 1 model. Instead of fixed treatment periods, length was determined by adverse events, clinical deterioration, and patient preference. Their model was further expanded to account for typical crossover features, including carry-over effects (Lindsey and Jones, 1996).

N of 1 trials come of age

In 1986, in the New England Journal of Medicine, a group of clinical investigators at McMaster University, Canada, published a paper entitled ‘Determining optimal therapy – randomized trials in individual patients’, in which they labelled such studies ‘N of 1 randomized control trials’ (Guyatt et al. 1986). Their interest had been prompted by a poorly controlled asthmatic patient treated with inhaled beta agonists, theophylline, and prednisone. The N of 1 trial they designed addressed the utility of the theophylline the patient was using. After the second paired block of theophylline and placebo, the patient ended the trial early: the results were clear to him, and, from the symptom diary he had been keeping, to the clinician who instituted the trial. When the blind was broken, it was clear that during the periods when the patient had been using theophylline his symptoms were much worse. Improvement was sustained when theophylline was withheld after the trial ended, with much better asthma control despite a reduced dose of steroids. The trial proved spectacularly helpful: improved symptom control, reduced drug burden, and decreased costs.

Among the class of single patient/person study designs (Tate et al. 2008; Perdices and Tate 2009; Gagnier et al. 2013), N of 1 trials are unique as rigorously controlled intervention studies that can provide a basis for inferring cause and effect. Though many variations exist, the work that originated at McMaster University focused on single patient trials with two or more pairs of treatment periods, one for the intervention and one for the comparator, ideally with blinding of both patients and health care providers (Figure 1). The outcome measures in such trials are the experiences of the patients, recorded using individualized, patient-reported outcomes.

Clinicians have now formally reported on hundreds if not thousands of N of 1 trials, exploring their utility in avoiding unnecessary treatment and improving patient outcomes, and also in facilitating drug development. Despite these reports, and the enormous potential that the originators saw for use of N of 1 trials, their uptake has remained limited in the decades since 1986, although there have been recent signs of renewed interest (Scuffam et al. 2010; Lillie et al. 2011; Kravitz et al. 2014, Schork 2015).

The N of 1 niche

The N of 1 trial identifies whether an intervention is likely to benefit or cause unwanted effects in an individual patient. The design is most suited to assessing interventions that act and cease to act quickly. It is particularly useful in clinical contexts in which variability in patient responses is large; when the evidence is limited; and/or when the patient differs in important ways from the people who have participated in conventional RCTs. Examples include: conditions with quickly acting symptomatic treatment, in which variability in response is large (for example, chronic pain, obstructive lung disease); conditions with a prevalence too low for large, parallel group RCTs; medically complex patients who differ substantially from patients who have participated in existing trials; and patients who have been treated over a long time when there is uncertainty about ongoing need for treatment (for example, proton pump inhibitors in longstanding dyspepsia). Indeed, the applicability of the results of parallel group randomized clinical trials (RCTs) to individual patients (i.e. external validity) may sometimes be limited by narrow inclusion criteria and the exclusion of patients with comorbidities and/or concurrent treatment. Reviews of RCTs have found average exclusion rates of 73% and recruitment of less than 10% of patients with the primary diagnosis (Rothwell 2005). These concerns, however, should be tempered by knowledge that true subgroup effects are very unusual (Sun et al. 2010). The real issue of importance to N of 1 trials is the likelihood, in many instances, of large variability in responses among patients (Larson 1990).

N of 1 trial services

The result of their first N of 1 trial inspired the team at McMaster to develop a full N of 1 referral service to address patient dilemmas that met criteria for our N of 1 designs: therapeutic impact was uncertain, the treatment target was to reduce daily or otherwise frequent symptoms, the intervention (typically a drug) worked quickly, and it quickly ceased acting. Within two years, the group had completed 57 N of 1 trials. Results had provided a definite therapeutic answer in 88% of the patients studied, and these results prompted 39% of physicians to change their prior-to-trial treatment plan. This experience led the McMaster team to offer guides for clinicians wishing to apply the N of 1 concept in their own practice (Guyatt et al. 1988). Ultimately, however, the clinical communities interest in conducting N of 1 trials diminished, and the service was terminated.

Eric Larson was in the audience at a presentation of the McMaster work at the American Federation for Clinical Research (Kravitz et al. 2008). Appreciating the utility of the design, Larson developed an N of 1 clinical service at the University of Washington. Over two years, Larson’s group completed 34 trials, again demonstrating that N of 1 trials could provide physicians with useful treatment guidance in uncertain cases, and improve patient satisfaction (Larson et al. 1993). Unfortunately, funding for the service ran dry and it was discontinued.

In 1999, the University of Queensland in Australia created the first national N of 1 research service, referred to as a ‘single patient trial service’ (Nikles et al. 2000). The service was designed to acquaint general practitioners with research methodology and to introduce research-derived data into clinical decision making for conditions where treatment effectiveness was uncertain. Physicians could refer their patients to the service, which was centrally located, and so used mail and telephone communication only. The service managed all major components of trial management: randomization, preparing tablets, sending all materials to patients, following-up, and relaying results to clinicians. Of the N-of-1 trials carried out by this service and which had available data, post-trial management decisions were consistent with trial results at 12 months in approximately 70% of ADHD trials (Nikles et al. 2006), 45% of osteoarthritis trials (Yelland et al. 2007), and 32% of neuropathic pain trials (Yelland et al. 2009). This is a successful example of how N of 1 trials can be implemented at a national level, though again, only as a temporary research initiative.

Another example of the versatility of N of 1 trials began when the Complementary and Alternative Research and Education (CARE) program at the University of Alberta established the first academic pediatric integrative medicine program in Canada. In 2006, as part of this program, a pediatric N of 1 service responded to the increased use of complementary therapies in children with chronic conditions. The goal of this service is to offer an objective, evidence-based approach to assessing whether a given complementary therapy is effective for a specific patient. The service is designed to assist patients, their parents and referring physicians throughout all stages of the N of 1 trial, including the design and implementation of the N of 1 evaluation. For example, this service has assessed natural health products (e.g. melatonin, probiotics, micronutrients) and acupuncture for conditions including ADHD, eczema, sleep disturbances, chemo-induced nausea and vomiting, irritable bowel syndrome, and autism.

N of 1 in drug development

The McMaster group speculated that drug development might also benefit from use of the N of 1 methodology. The reasoning was that pre-approval drug development costs are high (average $479-936 million USD (DiMasi et al. 2003; Adams et al. 2006) and rising (Dickson and Gagnon 2004)). Conducting N of 1 trials before a costly large-scale RCT could: a) help to assess early efficacy, b) be less expensive than traditional approaches, and c) identify predictors of response (Guyatt et al. 1990a).

The idea of applying the N of 1 approach to early drug development arose from experience with multiple N of 1 trials in specific conditions. For instance, when what is now termed myofascial pain syndrome was labeled fibrositis and there had been one apparently positive RCT of amitriptyline, the condition provided a framework for N of 1 trials in early drug development. The McMaster team conducted 14 N of 1 trials which demonstrated substantial benefit from amitriptyline at doses far lower than had been used for the primary indication for the drug, depression (Guyatt et al. 1990a). The McMaster team also demonstrated the utility of multiple N of 1 trials in Alzheimer’s disease (Molloy et al. 1991), and in the use of home oxygen in patients with chronic obstructive pulmonary disease (Nonoyama et al. 2007). In each of these situations the process appeared to be efficient, requiring limited cost and time investment. Nevertheless, subsequent attempts to apply the reasoning in drug development have been sporadic and unsuccessful.

Failure to revolutionize clinical practice: were N of 1 trials ahead of their time?

Early experience was disappointing, shattering the initial optimism that N of 1 trials would quickly revolutionize clinical practice. There had been some tantalizing results (Mahon et al. 1996), but RCTs in which patients were randomized to conventional care or to N of 1 trials generally failed to show dramatically convincing benefits of participation in the N of 1 trials (Pope et al. 2004; Mahon et al. 1999).

At McMaster University, despite educating local clinicians, playing cheerleader, and succeeding in conducting 73 N of 1 trials over 3 years (Guyatt et al. 1990b) and inspiring other “N of 1 services”, interest still faded. An attempt to use venture capital to create an efficient, marketable service went nowhere. Thirty years after our initial publication, few clinicians have even heard of N of 1 trials.

Sporadic reports of success with N of 1 continue. For instance, Joy et al. (2014) reported findings consistent with ‘the nocebo phenomenon’ – patients sometimes report side effects to placebo (Barsky et al. 2002): in 7 patients with suspected but uncertain statin-associated myalgia, N of 1 trials failed to detect any statin-related symptoms in any of the patients, allowing patients to continue the drugs. Despite such isolated reports of successes, clinicians seldom use N of 1 trials, and most remain unaware of the design.

Renewed interest in N of 1 trials

At the University of Alberta, recent efforts have focused on methodological issues related to N-of-1 trial design and reporting. For example, N of 1 trials have been criticized for their lack of generalizability. The Alberta group recently partnered with the Journal of Clinical Epidemiology to publish a series dedicated to N of 1 trials and included papers to address this concern. A comprehensive systematic review of the design, analysis, and meta-analysis of N of 1 trials found that the majority (60%) of published N of 1 trials are published as a series (i.e. one report publishing N of 1 trial data about more than one participant for the same condition-intervention pair), suggesting their value beyond assessing individual treatment effects and their potential to provide more generalizable treatment effects. Indeed, the Oxford Centre for Evidence-Based Medicine has classified N-of-1 trials as Level 1 evidence, comparable to systematic reviews of randomized controlled trials (OCEBM 2011a).

By virtue of their methods (i.e. use of randomization, blinding, formal outcome assessment), the meta-analysis of N of 1 trials may provide a valuable source of population data for conditions that have little to no RCT evidence, and to help refine evidence when parallel group RCTs may exist.

Given the large number of published N of 1 trials in ADHD, the condition may serve as a clinical model to explore the applicability of N of 1 trials beyond the individual patient (Punja et al. 2016a). Investigators at the University of Alberta conducted a systematic review and meta-analysis of N of 1 trials and demonstrated the use of traditional RCT meta-analysis methods in N of 1 trials (Punja et al. 2016a). In another study, Punja and colleagues demonstrated the value of N of 1 trials in meta-analyses, by conducting a combined meta-analysis of N of 1 trial data with RCT data (Punja et al. 2016b). The inclusion of N of 1 data in RCT meta-analyses improved the precision of population treatment effects, suggesting their potential to provide a rich source of data allowing for more powerful and reliable assessments of treatment effects (Punja et al. 2016b). This example also highlights the relevance of N-of-1 trials in conditions for which there is also traditional RCT evidence.

Range of conditions assessed in N of 1 literature	n
Diseases of the nervous system	27
Diseases of the musculoskeletal system and connective tissue	20
Mental and behavioural disorders	17
Diseases of the digestive system	11
Diseases of the respiratory system	09
Diseases of the circulatory system	04
Endocrine, nutritional, metabolic diseases	02
Infections and parasitic diseases	02
Other (nonspecific)	08

 

N=100; number of published N of 1 studies that have assessed treatments for the respective condition category (adapted from Punja et al. 2016c).

Challenges and future directions

Methodological considerations for N of 1 trials differ from those for standard, parallel-group RCTs. When considering N of 1 trials as a research endeavor, investigators have proposed solutions to three major limitations among reported N of 1 trials: incomplete reporting, marked variability in quality, and unacceptably low rates of prospective protocol registration.

First, as is the case with parallel group RCTs, lack of complete and transparent reporting is a problem in the N of 1 trial literature. The Alberta group (Punja et al. 2016c) found that authors of N of 1 trials failed to report on a number of critical design and conduct elements: trial registration (97%), whether individuals with comorbid conditions (77%) or on concurrent therapies (69%) were included, and whether adverse events were assessed (64%). Another review confirmed that the quality of reporting of published N of 1 trials was highly variable (Li et al. 2016). The Alberta group led the development of the CONSORT Extension for N of 1 Trials (CENT) in response to the limitations and heterogeneity in reporting (Vohra et al. 2015, Vohra et al. 2016). Like the CONSORT guidelines for RCTs, CENT is a minimum checklist for reporting N of 1 trials.

Second, careful development and reporting of N of 1 protocols is necessary for researchers, ethics review boards, and funders. The Alberta group is currently developing a SPIRIT Extension for N of 1 Trials (SPENT). This will recommend essential elements in N of 1 trial protocols, in the expectation that this will help to improve the quality of published reports of N of 1 trials and promote the inclusion of N-of-1 trial protocols in trial registries.

Third, only 3% of published N of 1 trials are reported as having registered protocols prospectively. It is certain that not all N of 1 trials are published and readily available (nor, for those conducted as part of optimal routine clinical practice, should they be) – unpublished trials begun as part of the research endeavor may create a risk of bias for future systematic reviews and meta-analyses. One way of capturing these trials would be to establish an electronic repository (as is done for conventional RCTs with clinicaltrials.gov), and encourage authors to register their N of 1 trial protocols. This would help reviewers to identify selective outcome reporting and publication biases.

Beyond these challenges, emerging methodologies may facilitate optimal use of N of 1 principles. Bayesian and adaptive designs have potential applicability to N of 1 trials. Trials can be designed with preset points based on adverse effects or patient preferences to crossover, change dose, or discontinuation. These methods can be used both to analyze and to meta-analyze N of 1 trials (Zucker et al. 2006; Zucker et al 2010). The strength of Bayesian approaches lies in their ability to maximize the use of reliable available information from each participant, as well as the use of reliable prior information for incorporation in the statistical model so that each N of 1 trial can inform the next. Zucker et al. have demonstrated the use of Bayesian methods to aggregate N of 1 trials to yield estimates of population treatment effects (Zucker et al 2010). Combining Bayesian approaches with adaptive designs may prove to be a useful combination for future N of 1 trials.

Discussion

What explains the failure adopt and sustain N of 1 trials? The obstacles to conducting N of 1 trials as an element of routine clinical practice have been too great. For many pharmacists, preparing identical drug and placebo combinations proved too labour intensive. For clinicians, N of 1 trials take too much time, even with easy-to-use guidance (Guyatt et al. 1988): preparing questionnaires, instructing patients, and examining the results all require clinician commitment. By comparison, the simple question, “did the treatment help” is too easy, and has too much face validity, compared to the more onerous substitution of a formal N of 1 trial. The late Professor Charles Bridge-Webb proposed a workaround to the expensive, time-consuming process of arranging placebo (Smith et al 2015). He suggested a simplified N of 1, The Single Patient Open Trials (SPOTs), substituting the blinded trial for an open one. This trial trades pragmatism for rigour, particularly useful for independent practitioners without access to N of 1 services.

The advent of technological advances may help to overcome the operational complexity and costs that have hindered the uptake of the N of 1 methodology. The emergence of mobile electronic health devices makes it easier than ever for patients to engage in their own healthcare. The creation of an IT-based N of 1 trial platform would help clinicians and patients to collaborate in designing their own N of 1 trials, track health outcomes, and produce a report of results for patients and clinicians to discuss. Researchers from the University of California, Davis, have developed a mobile application called the ‘Trialist’ specifically to facilitate the conduct of N of 1 trials in clinical settings. They are testing the feasibility and efficacy of this application in an RCT comparing the effects on patient outcomes of participating in a mobile N of 1 trial versus usual care (Barr et al. 2015).

This potential for N of 1 trials as a way of providing clinical care differs from its use as a research endeavor. The distinction comes down to the intent behind conducting an N of 1 trial. If the objective is to inform treatment decisions for an individual patient, the trial is optimal clinical care, and should therefore not require formal ethics approval (Irwig et al. 1995), nor regulatory oversight from agencies monitoring clinical research. When choices from among two or more alternative treatments are being considered, patients should be informed about genuine uncertainties about their relative merits, and how treatment should be selected in these circumstances (Chalmers and Lindley 2001). Random allocation within formal treatment comparisons is one of the options that should be offered to patients.

If the primary purpose of N of 1 trials is to produce generalizable knowledge to inform treatment decisions for future patients, these N of 1 trials are more properly regarded as research. In these circumstances, compliance with methodological and ethics standards will be expected. In 2014, the Agency for Healthcare Research and Quality commissioned a user’s guide to N of 1 trials, which clarifies this distinction (Kravitz et al. 2014).

N of 1 trials may have a future both as a research endeavor complementing standard trials and as a strategy for improving clinical care outside of the research setting. Unlike conventional parallel group RCTs, which assess what is best on average for a given population, N-of-1 trials assess what is best for an individual patient (Zucker et al. 1997). They are thus particularly well suited to emerging interests in patient-centred research and ‘precision’ or ‘personalized’ medicine. N of 1 trials support the evolution of patient-centered research by offering an evidence-based approach for personalizing care, they help to answer, for example, which treatment options are most effective through a process that strengthens the clinician-patient relationship and ultimately empowers the patient to be more engaged with their healthcare. Furthermore, with the advent of ‘big data’, and its hoped-for potential to inform care, N of 1 trials can provide opportunities to learn how to improve care. The potential exists. The extent to which it will be realized remains uncertain.

Acknowledgements

The authors are grateful to Liam Smeeth, Paul Glasziou, and Chris Del Mar for comments on earlier drafts.

This James Lind Library article has been republished in the Journal of the Royal Society of Medicine 2017;110:330-340. Print PDF

 

Appendix

N of 1 timeline

Author, year	Citation	Description/significance
Hogben and Sim 1953	Hogben L, Sim M (1953). The self-controlled and self-recorded clinical trial for low-grade morbidity. Br J Prev Soc Med 7:163–179.
Hare 1955	Hare EH (1955). Comparative efficacy of hypnotics: a self-controlled, self-recorded clinical trial in neurotic patients. British Journal of Preventive and Social Medicine 9:140-146.
Guyatt, 1986	Guyatt G, Sackett D, Taylor DW, Ghong J, Roberts R, Pugsley S (1986). Determining Optimal Therapy – – Randomized Trials in Individual Patients. N Engl J Med 314:889-892.	N of 1 trials are first brought to the attention of a wide medical readership
Guyatt, 1988	Guyatt G, Sackett D, Adachi J, Roberts R, Chong J (1988). A clinician’s guide for conducting randomized trials in individual patients. CMAJ 139: 497–503.	The practical approach presented encourages clinicians to conduct N of 1 trials
Guyatt, 1990	Guyatt GH, Keller JL, Jaeschke R, Rosenbloom D, Adachi JD, Newhouse MT (1990b). The n-of-1 randomized controlled trial: clinical usefulness. Our three-year experience. Ann Intern 112:293-299.	Results of the first N of 1 trial clinical service
Guyatt, 1990	Guyatt GGH, Heyting A, Jaeschke R, Keller J, Adachi JD, Roberts RS (1990a). N of 1 randomized trials for investigating new drugs. Control Clin Trials 11:88-100.	A proposal that individual N of 1 RCTs could be used to elucidate drug effects at an early stage of development.
Molloy 1991	Molloy DW, Guyatt GH, Wilson DB, Duke R, Rees L, Singer J (1991). Effect of tetrahydroaminoacridine on cognition, function and behaviour in Alzheimer’s disease. CMAJ 144:29-34.	Early use of multiple N of 1 trials in a particular condition to address drug effectiveness.
Zucker 1997	Zucker DR, Schmid CH, McIntosh MW, D’Agostino RB, Selker HP, Lau J (1997). Combining single patient (N-of-1) trials to estimate population treatment effects and to evaluate individual patient responses to treatment. J Clin Epidemiol 50:401-410.	Using Bayesian technique to combine N of 1 trials
Guyatt et al. 2000	Guyatt G, Haynes RB, Jaeschke RZ, Cook DJ, Naylor CD, Wilson MC, Richardson WS (2000). Users; guide to the medical literature: XXV. Evidence-based medicine: principles for applying the users’ guides to patient care. Evidence-Based Medicine Working Group. JAMA 284:1290-96.	N-of-1 trial described as top of methodological hierarchy for informing treatment decisions
Guyatt, 2002	Guyatt G, Zhang Y, Jaeschke R, McGinn T (2002). Therapy and Validity: N of 1 Randomized Controlled Trials. Users’ Guides to the Medical Literature: A manual for Evidence-Based Clinical Practice. Chicago, IL:  American Medical Association 275-290.	Users’ Guide to N of 1 trials
Nikles 2006	Nikles C, Mitchell G, Mar C Del, Clavarino A, McNairn N (2006). An N of 1 trial service in clinical practice: Testing the effectiveness of stimulants for attention-deficit/ hyperactivity disorder. Pediatrics 2006;117:2040-2046.	One of the largest series of N of 1 trials carried out in children with ADHD -carried out by a National Service to support the conduct of N of 1 trials (the first ever) in Australia
Zucker, 2006	Zucker DR, Ruthazer R, Schmid CH, Feuer JM, Fischer PA, Kieval RI, Mogavero N, Rapoport RJ, Selker HP, Stotsky SA, Winston E, Goldenberg DL (2006). Lessons learned combining N-of-1 trials to assess fibromyalgia therapies. J Rheumatol 33:2069-2077.	Using Bayesian technique to combine N of 1 trials
Tate, 2008	Tate RL, McDonald S, Perdices M, Togher L, Schultz R, Savage S (2008). Rating the methodological quality of single-subject designs and N of 1 trials: Introducing the single-case experimental design (SCED) scale. Neuropsychol Rehabil 18:385-401.	Single-Case Experimental Design (SCED) Scale
Zucker, 2010	Zucker DR, Ruthazer R, Schmid CH (2010). Individual (N of 1) trials can be combined to give population comparative treatment effect estimates:  methodological considerations. Journal of Clinical Epidemiology 63:1312-1323.	Using Bayesian technique to combine N of 1 trials
Gabler, 2011	Gabler NB, Duan N, Vohra S, Kravitz RL (2011). N-of-1 Trials in the Medical Literature: a systematic review. Med Care 49:761-768.	Systematic overview of the N of 1 literature
OCEBM, 2011	OCEBM Levels of Evidence Working Group*. “The Oxford Levels of Evidence 2”. Oxford Centre for Evidence-Based Medicine. http://www.cebm.net/index.aspx?o=5653	N of 1 trials are considered level 1 evidence by the Oxford Center for Evidence-Based Medicine
Tate, 2013	Tate RL, Perdices M, Rosenkoetter U, Wakim D, Godbee K, Togher L, McDonald S (2013). Revision of a method quality rating scale for single-case experimental designs and N of 1 trials: The 15-item Risk of Bias in N of 1 Trials (RoBiNT) Scale. Neuropsychological Rehabilitation 23:619-38.	15-item Risk of Bias in N of 1 Trials (RoBiNT) Scale
DEcIDE Methods Center N of 1 Guidance Panel, 2014	Kravitz RL, Duan N, eds, and the DEcIDE Methods Center N of 1 Guidance Panel (Duan N, Eslick I, Gabler NB, Kaplan HC, Kravitz RL, Larson EB, Pace WD, Schmid CH, Sim I, Vohra S) (2014). Design and Implementation of N of 1 Trials: A User’s Guide. AHRQ Publication No. 13(14)-EHC122-EF. Rockville, MD: Agency for Healthcare Research and Quality; February 2014. www.effectivehealthcare.ahrq.gov/N-1-Trials.cfm.	N of 1 AHRQ Publication
Tate, 2014	Tate RL, Perdices M, McDonald S, Togher L, Rosenkoetter U (2014). The design, conduct and report of single-case research: Resources to improve the quality of the neurorehabilitation literature. Neuropsychological Rehabilitation 2014;24:315-31.	Reporting guidelines: Risk of Bias in N of 1 Trials (RoBiNT) Scale; Single-Case Reporting guideline In BEhavioural interventions (SCRIBE)
Shamseer, 2015	Shamseer L, Sampson M, Bukutu C, et al. CONSORT extension for reporting N of 1 trials (CENT) 2015: Explanation and elaboration. BMJ 2015;350:18-46	CENT explanation and elaboration
Vohra, 2015	Vohra S, Shamseer L, Sampson M, Bukutu C, Schmid CH, Tate R, Nikles J, Zucker DR, Kravitz R, Guyatt G, Altman DG, Moher D, CENT group (2015). CONSORT extension for reporting N-of-1 trials (CENT) 2015 Statement. BMJ 350:h1738	CENT checklist
Punja, 2016	Punja S, Bukutu C, Shamseer L, Sampson M, Hartling L, Urichuk L, Vohra S (2016c). N-of-1 trials are a tapestry of heterogeneity. J Clin Epidemiol 76:47-56	A systematic review of the methods of design, analysis, and meta–analysis used in published N of 1 trials
Punja, 2016	Punja S, Schmid CH, Hartling L, Urichuk L, Nikles CJ, Vohra S (2016). To meta-analyze or not to meta-analyze? A combined meta-analysis of N of 1 trial data with RCT data on amphetamines and methylphenidate for pediatric ADHD. J Clin Epidemiol 76:76-81.	Combined meta-analysis of N of 1 trials with RCT data
Punja, 2016	Punja S, Xu D, Schmid CH, Hartling L, Urichuk L, Nikles CJ, Vohra S (2016a). N of 1 trials can be aggregated to generate group mean treatment effects: A systematic review and meta-analysis. J Clin Epidemiol 76:65-75.	Meta-analysis of N of 1 trials
Tate, 2016	Tate RL, Perdices M, Rosenkoetter U, Shadish W, Vohra S, Barlow DH, Horner R, Kazdin A, Kratochwill T, McDonald S, Sampson M, Shamseer L, Togher L, Albin R, Backman C, Douglas J, Evans JJ, Gast D, Manolov R, Mitchell G, Nickels L, Nikles J, Ownsworth T, Rose M, Schmid CH and Wilson B (2016). The Single-Case Reporting Guideline In BEhavioural Interventions (SCRIBE) 2016 Statement. J Clin Epidemiol 73:142-152.	Provides authors, readers and reviewers of single case design studies with a tool to maximise clarity, transparency and completeness in reports of such trials

 

References

Adams C, Brantner VV (2006). Estimating the Costs of New Drug Development: Is it really $802m? Health Aff 25:420-428. doi:10.1377/hlthaff.25.2.420.

Barr C, Marois M, Sim I, Schmid CH, Wilsey B, Ward D, Duan N, Hays RD, Selsky J, Servadio J, Schwartz M, Dsouza C, Dhammi N, Holt Z, Baquero V, MacDonald S, Jerant A, Sprinkle R, Kravitz RL (2015). The PREEMPT study – evaluating smartphone-assisted n-of-1 trials in patients with chronic pain: study protocol for a randomized controlled trial. Trials 16:67. doi:10.1186/s13063-015-0590-8.

Barsky AJ, Saintfort R, Rogers MP, Borus JF (2002). Nonspecific Medication Side Effects and the Nocebo Phenomenon. JAMA 287:622–627.

Baskerville JC, Toogood JH, Mazza J, Jennings B (1984). Clinical trials designed to evaluate therapeutic preferences. Statistics in Medicine 3:45–55.

Booth CC (2002). John Haygarth FRS (1740-1827). JLL Bulletin: Commentaries on the history of treatment evaluation (https://www.jameslindlibrary.org/articles/john-haygarth-frs-1740-1827/)

Chalmers I, Lindley R (2001). Double standards on informed consent to treatment. In: Doyal L, Tobias JS, eds. Informed Consent in Medical Research. London: BMJ Publications, 2001:266-275.

Dickson M, Gagnon J (2004). Key factors in the rising cost of new drug discovery and development. Nat Rev Drug Discov 3:417-429.

DiMasi J, Hansen R, Grabowski H (2003). The price of innovation: new estimates of drug development costs. J Health Econ 22:151-185.

Ebrahim S (2011). Hogben on speed, paradoxes and strain. International Journal of Epidemiology 40:1429-30.

Gabler NB, Duan N, Vohra S, Kravitz RL (2011). N-of-1 Trials in the Medical Literature: a systematic review. Med Care 49:761-768. doi:10.1097/MLR.0b013e318215d90d.

Gagnier J, Kienle G, Altman D, Moher D (2013). The CARE guidelines: consensus-based clinical case reporting guideline development. BMJ Case Reports. http://casereports.bmj.com/content/2013/bcr-2013-201554.full. Accessed November 24, 2016.

Glasziou P (2011). The history and place of n-of-1 trials: a commentary on Hogben and Sim. International Journal of Epidemiology 40:1458-60.

Guyatt G, Sackett D, Taylor DW, Ghong J, Roberts R, Pugsley S (1986). Determining Optimal Therapy – – Randomized Trials in Individual Patients. N Engl J Med 314:889-892. doi:10.1056/NEJM198604033141406.

Guyatt G, Sackett D, Adachi J, Roberts R, Chong J (1988). A clinician’s guide for conducting randomized trials in individual patients. CMAJ 139: 497–503.
https://www.researchgate.net/publication/19747472

Guyatt GGH, Heyting A, Jaeschke R, Keller J, Adachi JD, Roberts RS (1990a). N of 1 randomized trials for investigating new drugs. Control Clin Trials 11:88-100. doi:10.1016/0197-2456(90)90003-K.

Guyatt GH, Keller JL, Jaeschke R, Rosenbloom D, Adachi JD, Newhouse MT (1990b). The n-of-1 randomized controlled trial: clinical usefulness. Our three-year experience. Ann Intern 112:293-299. http://www.ncbi.nlm.nih.gov/pubmed/2297206. Accessed November 3, 2016.

Guyatt G, Haynes RB, Jaeschke RZ, Cook DJ, Naylor CD, Wilson MC, Richardson WS (2000). Users; guide to the medical literature: XXV. Evidence-based medicine: principles for applying the users’ guides to patient care. Evidence-Based Medicine Working Group. JAMA 284:1290-96.

Guyatt G, Zhang Y, Jaeschke R, McGinn T (2002). Therapy and Validity: N of 1 Randomized Controlled Trials. Users’ Guides to the Medical Literature: A manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association 275-290.

Hamburg MA, Collins FS (2010).The Path to Personalized Medicine. N Engl J Med 363:301-304 DOI: 10.1056/NEJMp1006304

Hare EH (1955). Comparative efficacy of hypnotics: a self-controlled, self-recorded clinical trial in neurotic patients. British Journal of Preventive and Social Medicine 9:140-146.

Haygarth J (1800). Of the imagination, as a cause and as a cure of disorders of the body: exemplified by fictitious tractors, and epidemical convulsions. Bath: R. Crutwell

Hogben L, Sim M (1953). The self-controlled and self-recorded clinical trial for low-grade morbidity. Br J Prev Soc Med 7:163–179. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1058512/. Accessed November 15, 2016.

Irwig L, Glasziou P, March L (1995). Ethics of n-of-1 trials. Lancet 345:469.

Joy T, Monjed A, Zou G, Hegele R, McDonald C (2014). N-of-1 (Single-Patient) Trials for Statin-Related Myalgia. Ann Intern Med 161:531-532. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.689.1067&rep=rep1&type=pdf. Accessed November 15, 2016.

Kazdin EA (2011). Single-Case Research Designs: Methods for Clinical and Applied Settings, 2nd Ed. Oxford University Press.

Kravitz RL, Duan N, Niedzinski EJ, Hay MC, Subramanian SK, Weisner TS (2008). What ever happened to N-of-1 trials? Insiders’ perspectives and a look to the future. Milbank Q 86:533-555. doi:10.1111/j.1468-0009.2008.00533.x.

Kravitz RL, Duan N, eds, and the DEcIDE Methods Center N-of-1 Guidance Panel (Duan N, Eslick I, Gabler NB, Kaplan HC, Kravitz RL, Larson EB, Pace WD, Schmid CH, Sim I, Vohra S) (2014). Design and Implementation of N-of-1 Trials: A User’s Guide. AHRQ Publication No. 13(14)-EHC122-EF. Rockville, MD: Agency for Healthcare Research and Quality.  https://effectivehealthcare.ahrq.gov/topics/n-1-trials/research-2014-5/

Larson EB (1990). N-of-1 Clinical Trials: A Technique for Improving Medical Therapeutics. Western Journal of Medicine 152: 52-56.

Larson E, Ellsworth A, Oas J (1993). Randomized clinical trials in single patients during a 2-year period. JAMA 270:2708-2712. http://www.ncbi.nlm.nih.gov/pubmed/8133588. Accessed November 3, 2016.

Li J, Gao W, Punja S, Ma B, Vohra S, Duan N, Gabler N, Yang K, Kravitz RL (2016). Reporting quality of N-of-1 trials published between 1985 and 2013: a systematic review. Journal of clinical epidemiology. https://doi.org/10.1016/j.jclinepi.2015.11.016.

Lillie EO, Patay B, Diamant J, Issell B, Topol EJ, Schork NJ (2011). The n-of-1 clinical trial: the ultimate strategy for individualizing medicine? Per Med 8:161-173. doi:10.2217/pme.11.7.

Lindsey JK, Jones B (1996). A model for cross-over trials evaluating therapeutic preferences. Statistics in Medicine 15:443–7.

Mahon J, Laupacis A, Donner A, Wood T (1996). Randomised study of n of 1 trials versus standard practice. BMJ 312:1069. http://www.bmj.com/content/312/7038/1069?variant=full. Accessed November 15, 2016.

Mahon JL, Laupacis A, Hodder RV, McKim DA, Paterson NA, Wood TE, Donner A (1999). Theophylline for irreversible chronic airflow limitation: a randomized study comparing n of 1 trials to standard practice. Chest 115:38-48.

Martini P (1932). Methodenlehre der Therapeutischen Untersuchung [Methodological principles for therapeutic investigations]. Berlin: Springer.

Molloy DW, Guyatt GH, Wilson DB, Duke R, Rees L, Singer J (1991). Effect of tetrahydroaminoacridine on cognition, function and behaviour in Alzheimer’s disease. CMAJ 144:29-34. http://www.ncbi.nlm.nih.gov/pubmed/1984813. Accessed December 17, 2016.

Nikles C, Glasziou PP, Del Mar CB, Duggan CM, Mitchell G (2000). N of 1 trials. Practical tools for medication management. Australian Family Physician 29:1108-1112. https://www.ncbi.nlm.nih.gov/pubmed/11127076. Accessed January 2, 2017.

Nikles C, Mitchell G, Mar C Del, Clavarino A, McNairn N (2006). An n-of-1 trial service in clinical practice: testing the effectiveness of stimulants for attention-deficit/hyperactivity disorder. Pediatrics 117. http://pediatrics.aappublications.org/content/117/6/2040.short. Accessed December 17, 2016.

Nonoyama ML, Brooks D, Guyatt GH, Goldstein RS (2007). Effect of Oxygen on Health Quality of Life in Patients with Chronic Obstructive Pulmonary Disease with Transient Exertional Hypoxemia. Am J Respir Crit Care Med 176:343-349. doi:10.1164/rccm.200702-308OC.

OCEBM (2011a). The Oxford Levels of Evidence. http://www.cebm.net/ocebm-levels-of-evidence/.

OCEBM (2011b). The Oxford Levels of Evidence 2. http://www.cebm.net/index.aspx?o=5653.

Parry C (1786). Experiments relative to the medical effects of Turkey Rhubarb, and of the English
Rhubarbs, No. I and No. II made on patients of the Pauper Charity. Lett Pap Bath Soc 11:431-453.

Perdices M, Tate R (2009). Single-subject designs as a tool for evidence-based clinical practice: Are they unrecognised and undervalued? Neuropsychol Rehabil 20:939. https://www.tandfonline.com/doi/abs/10.1080/09602010903040691. Accessed August 23, 2023.

Pope J, Prashker M, Anderson J (2004). The efficacy and cost effectiveness of N of 1 studies with diclofenac compared to standard treatment with nonsteroidal antiinflammatory drugs in osteoarthritis. J Rheumatol 31:140-149. http://www.jrheum.org/content/31/1/140.short. Accessed November 15, 2016.

Punja S, Xu D, Schmid CH, Hartling L, Urichuk L, Nikles CJ, Vohra S (2016a). N-of-1 trials can be aggregated to generate group mean treatment effects: a systematic review and meta-analysis. J Clin Epidemiol. 76:65-75. doi:10.1016/j.jclinepi.2016.03.026.

Punja S, Schmid CH, Hartling L, Urichuk L, Nikles CJ, Vohra S (2016b). To meta-analyze or not to meta-analyze? A combined meta-analysis of N-of-1 trial data with RCT data on amphetamines and methylphenidate for pediatric ADHD. J Clin Epidemiol 76:76-81. doi:10.1016/j.jclinepi.2016.03.021.

Punja S, Bukutu C, Shamseer L, Sampson M, Hartling L, Urichuk L, Vohra S (2016c). N-of-1 trials are a tapestry of heterogeneity. J Clin Epidemiol 76:47-56. doi:10.1016/j.jclinepi.2016.03.023.

Rolls R (2003). Caleb Hillier Parry (1755-1822). JLL Bulletin: Commentaries on the history of treatment evaluation (https://www.jameslindlibrary.org/articles/caleb-hillier-parry-1755-1822/).

Rothwell PM (2005). External validity of randomized controlled trials: “to whom do the results of this trial apply?” Lancet 365:82-93.

Schork NJ (2015). Time for one-person trials. Nature 520:609-611.

Scuffham PA, Nikles J, Mitchell GK, Yelland MJ, Vine N, Poulos CJ, Pillans PI, Bashford G, del Mar C, Schluter PJ, Glasziou P (2010). Using N-of-1 trials to improve patient management and save costs. Journal of General Internal Medicine 25:906–913. doi:10.1007/s11606-010-1352-7. PMC 2917656. PMID 20386995.

Shamseer L, Sampson M, Bukutu C, et al. CONSORT extension for reporting N of 1 trials (CENT) 2015: Explanation and elaboration. BMJ 2015;350:18-46 http://espace.library.uq.edu.au/view/UQ:362980. Accessed November 28, 2016.

Smith J, Yelland M, Del Mar C (2015). Single Patient Open Trials (SPOTs). In: Nikles J, Mitchell G, editors. The Essential Guide to N-of-1 trials in Health 195-209.

Stoll S (2004). Paul Martini’s Methodology of therapeutic investigation. JLL Bulletin: Commentaries on the history of treatment evaluation (https://www.jameslindlibrary.org/articles/paul-martinis-methodology-of-therapeutic-investigation/)

Sun X, Briel M, Walter S, Guyatt G (2010). Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses. BMJ 340:c117. http://www.bmj.com/content/bmj/340/bmj.c117.full.pdf. Accessed November 15, 2016.

Tabery J (2011). Hogben vs the tyranny of averages. International Journal of Epidemiology 40:1454-58.

Tate RL, McDonald S, Perdices M, Togher L, Schultz R, Savage S (2008). Rating the methodological quality of single-subject designs and n -of-1 trials: Introducing the Single-Case Experimental Design (SCED) Scale. Neuropsychol Rehabil 18:385-401. doi:10.1080/09602010802009201.

Tate RL, Perdices M, Rosenkoetter U, Wakim D, Godbee K, Togher L, McDonald S (2013). Revision of a method quality rating scale for single-case experimental designs and n -of-1 trials: The 15-item Risk of Bias in N -of-1 Trials (RoBiNT) Scale. Neuropsychol Rehabil 23:619-638. doi:10.1080/09602011.2013.824383.

Tate RL, Perdices M, McDonald S, Togher L, Rosenkoetter U (2014). The design, conduct and report of single-case research: Resources to improve the quality of the neurorehabilitation literature. Neuropsychol Rehabil 24:315-331. doi:10.1080/09602011.2013.875043.

Tate RL, Perdices M, Rosenkoetter U, Shadish R, Vohra S, Barlow DH, Horner R, Kazdin A, Kratochwill T, McDonald S, Sampson M, Shamseer L, Togher L, Albin R, Backman C, Douglas J, Evans JJ, Gast D, Manolov P, Mitchell G, Nickels L, Nikles J, Ownsworth T, Rose M, Schmid CH, Wilson B (2016). The Single-Case Reporting Guideline In BEhavioural Interventions (SCRIBE) 2016 Statement. Aphasiology. 6;30(7):862-876. doi:10.1080/02687038.2016.1178022.

Vohra S, Shamseer L, Sampson M, Bukutu C, Schmid CH, Tate R, Nikles J, Zucker DR, Kravitz R, Guyatt G, Altman DG, Moher D, CENT group (2015). CONSORT extension for reporting N-of-1 trials (CENT) 2015 Statement. BMJ 350:h1738. doi:10.1136/BMJ.H1738.

Vohra S, Shamseer L, Sampson M, Bukutu C, Schmid CH, Tate R, Nikles J, Zucker DR, Kravitz R, Guyatt G, Altman DG, Moher D; CENT Group (2016). CONSORT extension for reporting N-of-1 trials (CENT) 2015 Statement. J Clin Epidemiol. 76:9-17. doi:10.1016/j.jclinepi.2015.05.004.

Wiseman R (1676). Eight chirugical treatises. London: R Royston, p 123

Yelland M, Nikles C, McNairn N, Mar C, Schluter P, Brown R (2007). Celecoxib compared with sustained-release paracetamol for osteoarthritis: a series of n-of-1 trials. Rheumatology. 46:135–140. doi: 10.1093/rheumatology/kel195.

Yelland MJ, Poulos CJ, Pillans PI, Bashford GM, Nikles CJ, Sturtevant JM, Vine N, Del Mar CB, Schluter PJ, Tan M, Chan J (2009). N‐of‐1 Randomized Trials to Assess the Efficacy of Gabapentin for Chronic Neuropathic Pain. Pain Medicine. 10:754-761.

Zucker DR, Schmid CH, McIntosh MW, D’Agostino RB, Selker HP, Lau J (1997). Combining single patient (N-of-1) trials to estimate population treatment effects and to evaluate individual patient responses to treatment. J Clin Epidemiol 50:401-410. doi:10.1016/S0895-4356(96)00429-5.

Zucker DR, Ruthazer R, Schmid CH, Feuer JM, Fischer PA, Kieval RI, Mogavero N, Rapoport RJ, Selker HP, Stotsky SA, Winston E, Goldenberg DL (2006). Lessons learned combining N-of-1 trials to assess fibromyalgia therapies. J Rheumatol 33:2069-2077. http://www.ncbi.nlm.nih.gov/pubmed/17014022. Accessed December 17, 2016.

Zucker DR, Ruthazer R, Schmid CH (2010). Individual (N-of-1) trials can be combined to give population comparative treatment effect estimates: Methodologic considerations. J Clin Epidemiol 63:1312-1323. doi:10.1016/j.jclinepi.2010.04.020.