Abstract

In designing a controlled trial, a potential tension may exist between (i) providing insights into therapeutic mechanisms and (ii) guiding decisions about “what works” in everyday practice. In their 1967 paper in the Journal of Chronic Diseases, Schwartz and Lellouch (1967) designated these aims as ‘explanatory’ and ‘pragmatic’, illustrating the potential tension with the different trial designs needed to assess the effect of an agent hypothesized to increase responsiveness to radiotherapy (a “radiosensitiser”) with immediate initiation of radiotherapy in the ‘pragmatic’ design versus a 30-day delay in initiating radiotherapy for the ‘explanatory’ design. This dilemma is manifest in real clinical situations, such as chemotherapy or radiotherapy prior to cancer surgery (“neoadjuvant therapy), ’pre-habilitation’ to prepare patients for joint surgery, and whether to use a placebo when comparing two versus four times daily drug dosing.

Most study design choices are not so clearly and unidimensionally ‘pragmatic’ or ’explanatory’. The PRagmatic-Explanatory Continuum Indicator Summary (PRECIS) tool (Thorpe 2009), was developed to provide a graphical assessment of this spectrum for 10 “domains” that include eligibility criteria, flexibility of interventions, practitioner expertise, intensity of follow-up, compliance with treatments, and scope of the analysis.

The ‘explanatory’ versus ‘pragmatic’ tensions can sometimes be resolved. We suggest three categories: (a) Sometimes both ‘explanatory’ and ‘pragmatic’ objectives of a trial can be achieved by aligning the design to address both; (b) Sometimes there are possible compromises, for example, using an ‘explanatory’ sub-study within the framework of a ‘pragmatic’ main trial; and (c) Rarely, there are incompatible objectives, as illustrated by Schwartz and Lellouch (1967).

Both the ‘explanatory’ and ‘pragmatic’ aims of trials are important. In general, as research moves from an understanding of the mode of therapeutic action towards clinical application, studies move increasingly from the ‘explanatory’ mode to the ‘pragmatic’ mode. Recognising when and how to focus on ‘explanatory’ versus ‘pragmatic’ reasons for research will remain an important element in that judgment.

Introduction

Controlled trials are the principal means of investigating the effects of therapeutic and prophylactic interventions. They can be designed to avoid biases, and so provide clear and reliable insights into “what works” under ideal circumstances. Controlled trials can, however, also generate evidence to inform decisions on the choice of methods to control biases in everyday circumstances. This distinction between different types of trials came to be characterised by using the terms ’explanatory’ and ‘pragmatic’ to distinguish different types of trials [Schwartz and Lellouch 1967; Thorpe et al. 2009].

The need for these or similar terms (see Table in section on “The emergence of alternative terms”) began to be appreciated during the 1950s when it was realised that different terms were needed to distinguish between trials being done with differing primary objectives. In 1959, under the aegis of WHO and UNICEF, the Council for International Organizations of Medical Sciences sponsored a landmark, 5-day conference in Vienna on the role of controlled clinical trials (Bird 2014).

Among the 23 papers presented at the conference was one by the British rheumatologist Eric Bywaters who reported his experience dealing with the challenges presented by loss of trial participants over a three-year follow-up period after initial randomization (Bywaters 1960). Non-random dropouts and other losses typical of real-life trials raised concern that bias would result and undermine the reliability of the trial outcome.

The challenges presented to clinical decision-making by this practical issue were raised in a ‘rapport interprétatif’ (in French) of the conference prepared by Daniel Schwartz, a senior statistician at Inserm in Paris, and colleagues (Schwartz et al. 1960). The Chapter V was dedicated to ‘Portée des conclusions’ (in French) and stressed that:

[Translation: There is close conditioning between the definition of the trial and the scope of the conclusions. This conditioning is mutual: if the scope of the conclusions depends on the definition of the trial, the latter must conversely be organised in such a way as to respond to the problem posed.

It is from this angle that the various choices that constitute the organisation of the trial should be considered. For example, when defining the disease or the patients, there is always a choice between homogeneity and heterogeneity (…) each of the two methods has an advantage and a disadvantage: with the same number of patients, the examination of a homogeneous group makes it possible to see things more clearly and gives a more accurate result: but this result is less general (…). The choice will have to be made according to the problem posed – or the conditions of opportunity.]

In 1966, Marvin Schneiderman, a statistician who had been involved in US trials of cancer therapy in the early 1950’s, more specifically suggested dual purposes of RCTs in a working paper prepared for a WHO Expert Committee on Cancer Treatment:

In [cancer] chemotherapy at least, there appear to be two different kinds of trials, conducted for distinctly different purposes. There are the patient-orientated trial, and the drug-orientated trial. Patient-orientated trials are designed to give answers to the question ‘How shall I treat the next patient with cancer who comes into my care?” The drug-orientated trials attempt to answer the questions ‘Has this drug enough promise that I can bring it into patient-orientated trials?’ and ‘If I were to bring it into a patient orientated trial, how is it best to give it? (Schneiderman 1966, p 5)

Though Schneiderman’s distinction between “drug-oriented trials” and “patient-oriented trials” uses outdated terminology, we might now see these are precursors to the distinction between phase 2 and phase 3 trials. Indeed, under Schneiderman’s later guidance, as Chief Statistician at the US National Cancer Institute, this division of the two types of trials into early and later stage trials led to the three phases of drug trial with which we are now familiar. [See Zwarenstein: https://www.jameslindlibrary.org/articles/pragmatic-and-explanatory-attitudes-to-randomized-trials/]

Seven years after the WHO/UNICEF meeting in Vienna (Bird 2014) and a year after Schneiderman’s 1966 article, Schwartz and his colleague Joseph Lellouch explored the issue of the different purposes of trials further, and coined the terms now widely used in clinical trial design: ’explanatory’ trials were designed to provide understanding of the mechanisms through which treatments might have their effects, and ‘pragmatic” trials were designed to inform real-life decision-making (Schwartz & Lellouch 1967). While this distinction has some overlaps with Schneiderman’s distinction between ‘drug-oriented trials’ and ‘patient-oriented trials’, as the ‘explanatory’ versus ‘pragmatic’ distinction is more encompassing, both in terms of the types of intervention considered (not only drugs), and in the implications for design and analysis of trials.

In what follows, we describe the evolution of this distinction and the recognition that it does not reflect a simple dichotomy but forms a continuum. Its importance lies in how decisions made at the design stage affect where a trial lies on the several dimensions of the ‘pragmatic’-‘explanatory’ continuum’ (Thorpe et al. 2009). This in turn has a direct bearing on whether the trial can give reliable and relevant answers to the questions being asked of it.

As such, the distinction has an important if largely unrecognised role in the debate about waste in clinical research (Chalmers and Glasziou 2009). In recent years systematic methods for determining whether a specific trial design is appropriate for its intended purpose have been proposed (Thorpe et al. 2009; Loudon et al 2015). These have served to underline the importance and subtlety of the distinction made by Schwartz and Lellouch over half a century ago, the implications of which remain a focus of active research, such as surveys of how trials labelled as ‘pragmatic’ differ from others (eg Palakshappa et al. 2022; Taljaard et al. 2022).

The original statement of the distinction

In their 1967 paper in the Journal of Chronic Diseases, Schwartz and Lellouch first state (p 638) their chosen terms for what they held to be two contrasting trial types. As noted by Armitage (Armitage 1998), trialists had already noted the potential tension in designing a single trial that could provide reliable insights into therapeutic mechanisms while also guiding decisions about “what works” in everyday circumstances. Schwartz and Lellouch characterised the two types of trial as follows:

“The first type corresponds to an ‘explanatory’ approach, aimed at understanding. It seeks to discover whether a difference exists between two treatments which are specified by strict and usually simple definitions. Their effects are assessed by biologically meaningful criteria, and they are applied to a class of patients which is rather arbitrarily defined, but which is as likely as possible to reveal any difference that may exist.

“The second type corresponds to a ‘pragmatic’ approach, aimed at decision. It seeks to answer the question – which of the two treatments should we prefer? The definition of the treatments is flexible and usually complex: it takes account of auxiliary treatments and of the possibility of withdrawals. The criteria by which the effects are assessed take into account the interests of patients and the costs in the widest sense. The class of patients is predetermined as that to which the results of the trial are to be extrapolated.”

To illustrate how this distinction impacts trial design, Schwartz and Lellouch gave as an example a comparison of immediate initiation of radiotherapy for treating a newly diagnosed cancer versus delayed initiation of radiotherapy for 30 days during which the patient would take a “radiosensitiser” – a drug believed to increase tumour-sensitivity and responsiveness to radiotherapy.  Figure 1a (modified from the original) shows how the design of the trial thus depends on its aims. This difference in design prevents the two trials being equally reliable in resolving scientific and practical questions, as the confounding effect of the delay in giving the radiotherapy is unknown and could affect the outcome in different ways. The 2-arm trials in Figure 1a cannot answer both the ‘explanatory’ and ‘pragmatic’ questions, though a 3-arm trial (incorporating both “standard” arms) could address both questions.

Figure 1a: Does a radiosensitizer help? The ‘pragmatic’ design (left) would compare immediate radiotherapy to radiosensitiser then (delayed) radiotherapy; the ‘explanatory’ design (right) would also delay radiotherapy in the standard group to match the radiotherapy + radiosensitizer group, removing “delay” as an ‘explanatory’ factor for differences in effect (but the ethics of any such delay will need to be considered – see section on examples below).

Schwartz and Lellouch summarised the distinction as follows:

 “We may say in general that the ‘explanatory’ approach will always give an answer to the scientific problem but only sometimes to the problem of immediate practicality (depending on the result of the trial); while the reverse is true for the ‘pragmatic’ approach”.  

Three years after their 1967 article, Schwartz and his colleagues expanded on the distinction between these two types of trial in a book entitled ‘L’essai thérapeutique chez l’homme’ (Schwartz et al. 1970), subsequently translated into English by the British statistician Michael Healy (1980).

Examples of the ‘explanatory’ vs ‘pragmatic’ dilemma

Schwartz and Lellouch used a hypothetical example to illustrate a dilemma in addressing ‘explanatory’ and ‘pragmatic’ trial designs concurrently (see Figure 1a). However, the dilemma is manifested in multiple real clinical situations. Two actual clinical problems with similar timing, structure, and dilemma, are (i) whether to give chemotherapy or radiotherapy or neither prior to cancer surgery (Koukourakis 2020); and (ii) whether or not to use ’pre-habilitation’ to prepare patients for joint surgery (McIsaac 2022).

In the first example, chemotherapy can be given for several weeks prior to surgery to reduce ‘tumour load’ (known as neoadjuvant therapy, which can also be radiotherapy), but this will obviously delay surgery. Should trials of chemotherapy versus no chemotherapy prior to surgery use the ‘explanatory’ approach by arranging equal delay in the two comparison arms, or the ‘pragmatic’ approach by using immediate surgery for those not allocated to chemotherapy? In a tabulation of studies of short-course versus long-course radiotherapy [Koukourakis 2020, and Supplementary Figure 1b], both designs occurred: some studies used equal delay in both arms, and others began surgery immediately upon completion of short duration radiotherapy.

A similar dilemma arises with the second example – whether or not to use ’pre-habilitation’ to prepare a patient physically and psychologically for joint surgery or replacement [McIsaac 2022]. Analogous to the pre-surgical chemotherapy example, the dilemma for those designing trials is whether to arrange for an equal delay in both arms, or to have surgery as soon as possible [Figure 1c]. Often this dilemma is avoided by treating patients on long surgical waitlists, so waiting time is equalised, which, in effect, is the equivalent of the ‘explanatory’ design.

Delaying treatment is just one of several examples illustrating the challenge of designing studies addressing ‘explanatory’ and ‘pragmatic’ trial analyses concurrently. For example, a common problem in drug dosing is whether to prescribe medication once, twice, or thrice daily. Less frequent medication may result in better adherence to assigned treatment; more frequent medication would tend to result in more stable drug levels.

Figure 2 shows options for comparing twice daily with four times daily antibiotics with two options for the twice daily design [Raz 1995]. Option A does not use a placebo but only simple twice daily administration. With this comparison, the clinical impact will reflect both the biological effect and the effects of differences in adherence, but we cannot know the relative contributions of these two effects.  Option B replaces every second dose of the drug with a placebo. This option addresses the ‘explanatory’ effect, namely the value of steadier versus more intermittent drug levels by using placebo to remove the adherence effect [Figure 1c]. We cannot answer both questions with a 2-arm trial, but we can do this by using all three arms, a potential solution to the challenges of applying ‘explanatory’ and ‘pragmatic’ designs concurrently.

Figure 2: Twice daily dosing options: ‘pragmatic’ (option A) or ‘explanatory’ (option B)

The emergence of alternative terms

Most study design choices are not so clearly ‘pragmatic’ or ‘explanatory’ as the example given by Schwartz and Lellouch. Moreover – and as so often in the terminology of trial design – their chosen nomenclature can be criticised as failing to capture the nuances of the concepts involved. Over the decades, this has led to proposals for alternative terms referring to the same basic idea (see Table), among them those proposed by Cochrane (1972), the US Congress Office of Technology Assessment (1978) and Sackett (2006). Nevertheless, the original choice of ’explanatory’ and ’pragmatic’ appears to have become the de facto nomenclature, and we will use these terms as the default during the rest of this article.

From simple dichotomy to continuum

In their seminal 1967 article, Schwartz and Lellouch begin by stating their belief that “most therapeutic trials are inadequately formulated…in that the trials may be aimed at the solution of one or other of two radically different kinds of problem.” [emphasis added]. In their summary section the authors qualified this apparent dichotomy, stating that, with the comparison of two treatments, “the ‘explanatory’ approach will always give an answer to the scientific problem but only sometimes to the problem of immediate practicability (depending on the result of the trial); while the reverse is true for the ‘pragmatic’ approach.” By the close of the article, they had moved away from their simple classification and towards a continuum, stating that “Most real problems contain both ‘explanatory’ and ‘pragmatic’ elements, for ethical reasons”.

Armitage elaborated on this in a 1998 article (Armitage, 1998) stating: “It may be more realistic to suggest that the two attitudes are likely to co-exist, and to compete for ascendency, in any one trial. The investigators planning a trial may well wish to ‘kill two birds with one stone’ to gain information about biological mechanisms, and to assess strategies of treatment.”

Perhaps surprisingly given its obvious importance, practical guidance on ensuring that a trial design was “fit for purpose” only emerged half a century after the Vienna conference. It took the form of a means of assessing a given design according to criteria which collectively capture the degree to which a trial is ‘pragmatic’ or ‘explanatory’ in nature. Devised and reported by Thorpe et al. (2009), the PRagmatic-Explanatory Continuum Indicator Summary (PRECIS) tool leads to a graphical assessment based on 10 “domains”:

1. The eligibility criteria of trial participants.
2. The flexibility with which the experimental intervention is applied.
3. The degree of practitioner expertise in applying and monitoring the experimental intervention.
4. The flexibility with which the comparison intervention is applied.
5. The degree of practitioner expertise in applying and monitoring the comparison intervention.
6. The intensity of follow-up of trial participants.
7. The nature of the trial’s primary outcome.
8. The intensity of measuring participants’ compliance with the prescribed intervention, and whether compliance improving strategies have been used.
9. The intensity of the measurement of practitioners’ adherence to the study protocol, and whether adherence-improving strategies are used.
10. The specification and scope of the analysis of the primary outcome.

Squaring the circle of ‘pragmatic’ and ‘explanatory’ designs

The domains suggested by Thorpe et al. (2009) open up the possibility of trial designs that can achieve both ‘explanatory’ and ‘pragmatic’ objectives:

1/ Aligned: Sometimes both ‘explanatory’ and ‘pragmatic’ objectives of a trial can be achieved by aligning the design to address both. For example, large sample sizes and strict adherence to randomly assigned treatment are desirable for both ‘explanatory’ and ‘pragmatic’ aims. An ‘explanatory study’ may accept a somewhat smaller sample size aimed at improving adherence, for example, by using a ‘run in period’ to check for tolerance or adherence to treatment (the FIELD trial of 10,000 randomized participants (FIELD 2004) did this, so it is not just for small trials). But such run-ins would also be acceptable for analysing the pragmatic component of the study.

2/ Compromise needed and possible: Sometimes there are possible compromises. For example, the ‘pragmatic’ trial might be large and simple, but use a sub-study with greater measurement options for achieving some ‘explanatory’ components (eg the LIPID trial with 9,000 randomized (MacMahon 1998) had a carotid substudy of 400 patients to look at the effect of statins on carotid thickening). Similarly, the large Covid vaccine trials had a broad population with a simple intervention (Voysey 2021), but most of these also included some ‘explanatory’ elements, such as identifying asymptomatic cases or taking blood for assessing antigenic response in a subsample of participants.

3/ Incompatible objectives: Sometimes pursuit of ‘explanatory’ and ‘pragmatic’ trial objectives concurrently within the same trial is impossible. The radiosensitiser example (see above) illustrates this problem. As noted earlier, for that example, no 2-arm trials (see Figure 1a) can address and answer both the ‘explanatory’ and ‘pragmatic’ questions, though a 3-arm trial (incorporating both ‘standard’ arms) could. Irrespective of size, inclusion criteria, etc. a single 2-arm trial design cannot address both the ‘explanatory’ (biological) and the ‘pragmatic’ (decisional) conceptualisation. [For more detail see Zwarenstein: https://www.jameslindlibrary.org/articles/pragmatic-and-explanatory-attitudes-to-randomized-trials/]

Relation to Intention-To-Treat (ITT) Analysis

‘Pragmatic’ and ‘explanatory’ features are different facets of the design and analysis of controlled trials. They do not necessarily compete. Indeed, they can co-exist within the same trial. The ‘pragmatic’ and ‘explanatory’ aims are about decisions at or before the start of a trial about the questions and purposes of the investigation. The purpose of ‘pragmatic’ trials is about  decisions, often for a broad group of patients who might benefit from an intervention. By contrast, the ‘explanatory’ route focuses on understanding mechanisms, and often requires highly selected patients whose condition is well understood and might benefit or become clearer by a treatment comparison. Both types of trial might require an intention-to-treat analysis, although this would be more likely to be needed in a pragmatic trial than in an ‘explanatory’ trial because of the heterogeneity of the patients in the former group.

In their highly-cited paper, Thorpe et al. (2009) note that, whether a trial is characterised as ‘explanatory’ or ‘pragmatic’, the primary, unbiased analyses should be by ‘Intention To Treat’ (ITT) (see article by Chalmers, Matthews, Glasziou, Boutron and Armitage 2023).

Thus, for analyses of ‘pragmatic’ trials, “the primary outcome should include all patients, regardless of eligibility and compliance, etc. (‘intention-to-treat’ analysis). In other words, the analysis attempts to see if the treatment works under usual conditions, with all the ‘noise’ inherent therein.”

For analyses of ‘explanatory’ trials, “an ‘intention-to-treat’ analysis is usually performed. However, this may be supplemented by secondary analyses, for example, an analysis restricted to ‘compliers’ or other subgroups in order to estimate maximum achievable treatment effect. Analyses are conducted that attempt to answer the narrowest, ‘mechanistic’ question (whether biological, educational or organizational” (Thorpe et al. 2009, p E48). If estimates of treatment effects from non-ITT and ITT analyses are indistinguishable it provides some reassurance that residual biases are likely to be small. 

Additional analyses, such as adjustments for non-compliance, may help to explain trial results, but the confusing term ‘per-protocol’ should be avoided.

In a 1998 paper, Armitage (1998) suggested that the ‘explanatory’ effect which, if present, should occur with 100% compliance, might be best done by one of several modelling approaches which adjust for non-compliance rather than by dismissing the data of non-compliant participants. Since then there have been many other such approaches developed, which use the ITT analysis but make an adjustment for non-compliance. Unlike a so-called “per protocol” analysis, these compliance-adjusted ITT estimates will not suggest an effect when there is none.

Conclusions

As randomised trials gained acceptance in the 1950s, some recognition emerged that trials may have different purposes and hence different design requirements. For example, Schneiderman identified the different requirements for examining the biological activity of a drug versus the assessment of its value in treating patients. This recognition resulted in the different aims and designs of phase 2 and phase 3 clinical trials. The 1967 paper by Schwartz and Lellouch drew the key distinction between the ‘explanatory’ and ‘pragmatic’ aims of a trial.

Both the ‘explanatory’ and ‘pragmatic’ aims are important. In general, as research moves from an initial understanding of a potential therapy towards its clinical application with patients, studies move increasingly from the ‘explanatory’ mode to the ‘pragmatic’ mode. That is, research moves progressively from small ‘explanatory’ trials which measures multiple surrogate outcomes to examine biological activity, to the larger scale, simple ‘pragmatic’ trial, which focuses much more on patient-relevant outcomes, both beneficial and harmful. As we have set out in the later sections of this paper, the ‘explanatory’ and ‘pragmatic’ tensions can sometimes be resolved, but as Schwartz and Lellouch illustrated with their radiotherapy example, the two aims are occasionally irreconcilable.

Since this watershed recognition, there have been ongoing developments. For example, Thorpe and colleagues (2009) describe multiple domains, and how within each domain there is a spectrum of ‘explanatory’ to ‘pragmatic’ characteristics. As we adopt trial designs beyond the simple parallel two-group trial – such as factorial, platform and adaptive trials – the ‘explanatory’ versus ‘pragmatic’ distinction will undoubtedly evolve further and be recognised as a key feature in design. That recognition would be an important step towards fulfilling Doug Altman’s call for for “… less research, better research and research done for the right reasons” [Altman 1994]. Recognising when and how to focus on ‘explanatory’ versus ‘pragmatic’ reasons for research will be an important element in pursuing the implied endeavour.

Acknowledgements

We thank John Simes for helpful comments on various drafts and/or questions.

This James Lind Library article has been republished in two parts in the Journal of the Royal Society of Medicine 2023;116:425-432. Print PDF

References    

Altman DG (1994). The scandal of poor medical research. BMJ 29;308(6924):283-4. doi: 10.1136/bmj.308.6924.283. PMID: 8124111; PMCID: PMC2539276.

Armitage P (1998). Attitudes in clinical trials. Stat Med 15;17(23):2675-83.

Bird SM (2014). The 1959 meeting in Vienna on controlled clinical trials – a methodological landmark. JLL Bulletin: Commentaries on the history of treatment evaluation (https://www.jameslindlibrary.org/articles/the-1959-meeting-in-vienna-on-controlled-clinical-trials-a-methodological-landmark/)

Bywaters E (1960). Rheumatoid arthritis. Treatment and illustrative answers. In: Hill AB (ed). Controlled clinical trials. Oxford: Blackwell, pp 75-83.

Chalmers I, Glasziou P (2009). Avoidable waste in the production and reporting of research evidence. Lancet 374:86-89. doi:10.1016/S0140-6736(09)60329-9.

Chalmers I, Matthews R, Glasziou P, Boutron I, Armitage P (2023). Analysis of clinical trial by Treatment Allocated or by Treatment Received? Applying ‘the intention-to-treat principle’. JLL Bulletin: Commentaries on the history of treatment evaluation (https://www.jameslindlibrary.org/articles/analysis-of-clinical-trial-by-treatment-allocated-or-by-treatment-received-applying-the-intention-to-treat-principle/)

Cochrane AL (1972). Effectiveness and Efficiency: random reflections on health services. Nuffield Provincial Hospital Trust. https://www.nuffieldtrust.org.uk/files/2017-01/effectiveness-and-efficiency-web-final.pdf

FIELD Study Investigators (2004). The need for a large-scale trial of fibrate therapy in diabetes: the rationale and design of the Fenofibrate Intervention and Event Lowering in Diabetes (FIELD) study. [ISRCTN64783481]. Cardiovasc Diabetol 1;3:9. doi: 10.1186/1475-2840-3-9. PMID: 15571637; PMCID: PMC1129022.

Karanicolas PJ, Montori VM, Devereaux PJ, Schünemann H, Guyatt GH (2009). A new ‘mechanistic-practical” framework for designing and interpreting randomized trials. J Clin Epidemiol 62:479-84. doi: 10.1016/j.jclinepi.2008.02.009. Epub 2008 May 12. PMID: 18468856.

Koukourakis G (2020). Which is the best neoadjuvant (pre-surgery) chemoradiation regimen for locally advanced rectal carcinoma? Short or long course of radiation therapy? Do we have new data? J BUON 25(1):51-61. PMID: 32277614.

Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M (2015). The PRECIS-2 tool: designing trials that are fit for purpose. BMJ 8;350:h2147. doi: 10.1136/bmj.h2147. PMID: 25956159.

MacMahon S, Sharpe N, Gamble G, Hart H, Scott J, Simes J, White H (1998). Effects of lowering average of below-average cholesterol levels on the progression of carotid atherosclerosis: results of the LIPID Atherosclerosis Substudy. LIPID Trial Research Group. Circulation 12;97(18):1784-90. doi: 10.1161/01.cir.97.18.1784.

McIsaac DI, Gill M, Boland L, Hutton B, Branje K, Shaw J, Grudzinski AL, Barone N, Gillis C; Prehabilitation Knowledge Network (2022). Prehabilitation in adult patients undergoing surgery: an umbrella review of systematic reviews. Br J Anaesth 128(2):244-257. doi: 10.1016/j.bja.2021.11.014. Epub 2021 Dec 16. PMID: 34922735.

Palakshappa JA, Gibbs KW, Lannan MT, Cranford AR, Taylor SP (2022). Systematic Review of the “Pragmatism” of Pragmatic Critical Care Trials. Crit Care Explor 22;4(7):e0738. doi: 10.1097/CCE.0000000000000738. PMID: 35923590; PMCID: PMC9312432.

Raz, R., et al. “Penicillin V twice daily vs. four times daily in the treatment of streptococcal pharyngitis.” Infectious Diseases in Clinical Practice 4.1 (1995): 50-54.

Schneidermann MA (1966). Therapeutic trials in cancer. Working paper prepared for WHO Expert Committee on Cancer Treatment, Geneva, Switzerland, 9-15 March 1965. WHO/CANC/66.66.

Schwartz D, Flamant R, Lellouch J, Rouquette C (1960). Les essais thérapeutiques cliniques. Paris: Masson.

Schwartz D, Lellouch J (1967). Explanatory and pragmatic attitudes in therapeutic trials. Journal of Chronic Diseases 20:637-648.

Schwartz D, Flamant R, Lellouch J (1970). L’essai thérapeutique chez l‘homme. Paris: Flammarion.

Taljaard M, Nicholls SG, Howie AH, Nix HP, Carroll K, Moon PM, Nightingale NM, Giraudeau B, Hey SP, Eldridge SM, Weijer C, Zwarenstein M (2022). An analysis of published trials found that current use of pragmatic trial labels is uninformative. J Clin Epidemiol 151:113-121. doi: 10.1016/j.jclinepi.2022.08.007. Epub 2022 Aug 18. PMID: 35987403.

Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg C, Altman DG, Tunis S, Bergel E, Haevwy I, Magid DG, Chalkidou K (2009). A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. CMAJ 180:E47-E57.

Voysey M, Clemens SAC, Madhi SA, Weckx LY, Folegatti PM, Aley PK, Angus B, Baillie VL, Barnabas SL, Bhorat QE, Bibi S, Briner C, Cicconi P, Collins AM, Colin-Jones R, Cutland CL, Darton TC, Dheda K, Duncan CJA, Emary KRW, Ewer KJ, Fairlie L, Faust SN, Feng S, Ferreira DM, Finn A, Goodman AL, Green CM, Green CA, Heath PT, Hill C, Hill H, Hirsch I, Hodgson SHC, Izu A, Jackson S, Jenkin D, Joe CCD, Kerridge S, Koen A, Kwatra G, Lazarus R, Lawrie AM, Lelliott A, Libri V, Lillie PJ, Mallory R, Mendes AVA, Milan EP, Minassian AM, McGregor A, Morrison H, Mujadidi YF, Nana A, O’Reilly PJ, Padayachee SD, Pittella A, Plested E, Pollock KM, Ramasamy MN, Rhead S, Schwarzbold AV, Singh N, Smith A, Song R, Snape MD, Sprinz E, Sutherland RK, Tarrant R, Thomson EC, Török ME, Toshner M, Turner DPJ, Vekemans J, Villafana TL, Watson MEE, Williams CJ, Douglas AD, Hill AVS, Lambe T, Gilbert SC, Pollard AJ; Oxford COVID Vaccine Trial Group (2021). Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet 9;397(10269):99-111. doi: 10.1016/S0140-6736(20)32661-1. Epub 2020 Dec 8.

Zwarenstein M (2016). ‘Pragmatic’ and ‘Explanatory’ attitudes to randomized trials. JLL Bulletin: Commentaries on the history of treatment evaluation (https://www.jameslindlibrary.org/articles/pragmatic-and-explanatory-attitudes-to-randomized-trials/). Also as Zwarenstein M (2017). ‘Pragmatic’ and ‘explanatory’ attitudes to randomised trials. Journal of the Royal Society of Medicine 110:208–218. DOI: 10.1177/0141076817706303ncet 2009;374:86-89. doi:10.1016/S0140-6736(09)60329-9.

Supplementary Figures

Figure 1 b and c: The design structures for ‘pragmatic’ versus ‘explanatory’ trials for the neo-adjuvant chemotherapy and ‘pre-habilitation’ examples – these mirror the issues in Figure 1.