Stjernswärd J (2009) Meta-analysis as a manifestation of ‘bondförnuft’ (‘peasant sense’).

© Jan Stjernswärd, Borringekloster, SE – 233 91 Svedala, Sweden.Email: janstjernsward@hotmail.com


Cite as: Stjernswärd J (2009) Meta-analysis as a manifestation of ‘bondförnuft’ (‘peasant sense’). JLL Bulletin: Commentaries on the history of treatment evaluation (https://www.jameslindlibrary.org/articles/meta-analysis-as-a-manifestation-of-bondfornuft-peasant-sense/)


Background

In the early 1970s I was a young radiotherapist/oncologist at the Radiumhemmet, Karolinska Institutet, Stockholm – once a ’Mecca’ for radiotherapy. Postoperative radiotherapy for breast cancer patients was routine there at that time. However, most breast cancer patients were dying of metastases and I questioned the effectiveness of routine use of local radiotherapy when so many patients were dying of disseminated cancer anyway. My peers in the hospital told me that “their long clinical experience had shown them that radiotherapy was effective” and that it should be given routinely.

Undeterred, I reviewed and analysed five published reports of controlled trials of local radiotherapy given after or before surgery (Stjernswärd 1974). None showed any survival benefit; indeed, compared to controls, mortality was higher in the irradiated groups in all of the trials. The increase ranged from 1 to 10 per cent, between 1 to 10 years postoperatively. In two of the studies with data allowing an analysis by age and menstrual status, mortality after radiotherapy was about 15 per cent higher in younger, premenopausal women.

I was never any good at mathematics and statistics but I know when and what I want them for. My Nordic ‘bondförnuft’ – literally, ‘peasant sense’ – told me that although the higher mortality following radiotherapy seen in the trials was not statistically significant in the individual studies, taken together they might become statistically significant and that this might be demonstrated if there was a clever method of combining the data. I had published most of the articles from my PhD research in the Journal of the National Cancer Institute and that, together with the Lancet, were my favorite journals at that time. I remembered that I had seen an article in the JNCI in which data from several similar retrospective studies had been combined and analyzed statistically. I therefore went to my statistical-epidemiological guru, Nick Day, at the International Agency for Research in Cancer, Lyon, and asked him if it could be done and, if so, if he could help. Nick immediately fished out the article by Mantel and Haenzel (1959) and did the statistical analysis. In spite of my insistence, however, he declined to co-author my Lancet article with me (Stjernswärd 1974). So if my article is one of the earliest uses of meta–analysis of controlled trials in medicine, this is mainly thanks to Nick Day.

In the article I clearly acknowledged as already proven that radiotherapy reduced the risk of local and regional recurrences of breast cancer, but I questioned routine use of postoperative radiotherapy, over several weeks, to even node-negative breast cancer patients, given that the treatment seemed to be associated with increased mortality. Indeed, as early as 1959 a report of the first randomized trial of radiotherapy in breast cancer noted that ”Something seemed to be causing death shortly after treatment in a small number of patients who would not have died then had they not been irradiated” (Paterson and Russell 1959).

Reactions to the article

The article outraged the radiotherapy establishment, especially in North America, where radiotherapists felt that the evidence I had presented threatened not only their lucrative earnings but also their professional prestige. The latter was coming under threat from the rapidly emerging specialty of medical oncology, which had overtaken radiotherapy by developing and testing treatments for breast cancer with curative intent, and by demonstrating this in controlled trials with positive results (see, for example, Bonadonna et al. 1977). As a result of my recent article, I found myself less frequently invited to radiotherapy meetings, but correspondingly more likely to be invited to cancer control and oncology meetings. An American surgeon, Bernard Fisher, was leading the largest breast cancer trials group in the world. Reflecting his own experience, he presented me with a hand-painted picture of a turtle with the legend “Only by sticking out your neck do you advance”. He made me a member of his Turtle Club, and this helped me to weather the storm that followed my article.

A year after my article had been published, Levitt and McHugh (1975) challenged my analysis, both in terms of the five clinical trials I had selected, and my use of the Mantel-Haenszel technique to analyze them. In a rebuttal (Stjernswärd et al. 1976), I and my colleagues suggested that, in the absence of any suggested alternative statistical technique, it was appropriate to use the Mantel-Haenszel procedure, in spite of the technical differences between the radiotherapy regimens used in the five studies, again stressing that radiotherapy’s reduction of local tumour recurrences was not in question. We stressed that therapeutic and economic priorities argued for a more critical approach to routine use of radiotherapy in early breast cancer, particularly as it seemed to be associated with increased mortality, an iatrogenic failure.

The following year, the same authors published two linked papers, one of which was entitled “A re-examination of Stjernswärd’s application of the Mantel-Haenszel statistical method” (Levitt et al. 1977). They suggested that there was no statistically significant evidence that survival was affected negatively by irradiation. I had not been invited to the American Cancer Society’s National Conference on radiation oncology at which these papers had been presented, but Nick Day joined me in coauthoring a further rebuttal (Stjernswärd and Day 1978). We pointed out that the authors had failed to refer to the earlier rebuttal, or to acknowledge that in every one of the trials included in my 1974 paper mortality was higher in the radiotherapy groups than in the control groups. Our use of the Mantel–Haenszel technique might have reduced the efficiency of the design of the analysis but would not have introduced bias. Given our findings, engaging in discussion of statistical niceties seemed to us to be displacement activity. I was invited to a breast cancer conference later the same year, organized under the aegis of the White House, the National Cancer Institute and the American Cancer Society, and I presented all the then available results from the relevant trials (Stjernswärd 1977).

In an attempt to explain the higher death rate in women treated with radiotherapy by a decrease in immuno-competent cells after irradiation, I had speculated that an increase of metastasis due to a decreased immuno-competence might explain the increased number of deaths observed in the irradiated groups. This speculation proved wrong, however. Later data showed that the increased mortality could be explained by irradiation of the heart (Höst et al. 1988; Haybittle et al. 1989; Giordano et al. 2005).”

The development of more sophisticated meta-analyses

In the 1980s, resources were secured to allow a substantial development of my early meta-analysis. These were increasingly based on individual patient data, which allowed more detailed analyses (albeit still using the Mantel-Haenszel method!). In the first of these, published in 1987 (Cuzick et al. 1987), all trials that had matured were analysed, and this confirmed a statistically significantly higher mortality rate after 10 years in women who had received radiotherapy compared with controls. Most of these patients had participated in trials similar to those I had analyzed in 1974. Orthovoltage irradiation had been used in these, and this damages the skin and the heart to a greater extent than subsequently adopted, more targeted, supervoltage irradiation. No statistically significant survival disadvantage was detected with these newer regimens. This was confirmed in a follow up analysis published in the 1990s. This reported that “the increased all-cause mortality rate in 10-year survivors previously reported is no longer [statistically] significant, although a numerical difference in favour of non-irradiated patients remains”. However, the authors also noted that “an excess of cardiac deaths was apparent in both early and more recent trials” (Cuzick et al 1994).

Analyses reported by the Early Breast Cancer Trialists’ Collaborative Group (2000) at the beginning of this century confirmed that local radiotherapy decreases local recurrences (8.8 per cent versus 27.2 per cent 10 years post treatment). Overall, they found a 20-year survival of 37.1 per cent after radiotherapy compared with 35.9 per cent survival among the un-irradiated control patients. There was little difference in early mortality, but analyses of later deaths showed that radiotherapy reduced mortality rates from breast cancer but increased mortality from other causes. The report estimated that, if radiotherapy regimens without long-term hazards could be found, they might produce an increase in 20-year survival, except for women at particularly low risk of local recurrence. The confirmation of an increased mortality associated with irradiation prompts one to wonder, however, how many women with a low risk of local recurrence have died because of postoperative irradiation. More than 70 per cent of the women in the analysis did not have any loco-regional recurrences.

Conclusion

A meta-analysis of five trials of radiotherapy for breast cancer published nearly half a century ago identified a previously unrecognised hazard of this therapy, namely increased mortality. This prompted more detailed systematic reviews and meta-analyses confirming the effect and elucidating its cause; improved radiotherapy regimens to control local recurrence of the disease; and more careful selection of the women most likely to be helped by this local beneficial effect of radiotherapy. These advances in knowledge and practice were reflected in a recent editorial: “it is a matter of some satisfaction that these early overviews have changed practice and we are now beginning to see the benefits of these improved protocols” (Cuzick 2005).

An important challenge is to make progress in identifying those women who can be expected to benefit from radiotherapy and those for whom it may only present a life-threatening hazard. The identification of new risk markers may improve this discrimination (Wirapati et al 2008; Nimeus-Malmström 2009; International Breast Cancer Study Group 2009), but there will be a continuing need for systematic reviews and meta-analyses of well designed studies addressing the possible differential effects of radiotherapy in newly identified subgroups.

A key ethical principle of good medical care is violated when patients are not told the truth about the effects of treatments, in terms that matter to them. Patients, clinicians and policymakers need ready access to valid systematic reviews of reliable evidence, with sufficient detail to support informed decision-making. Without these, patients will continue to suffer and die unnecessarily, and national resources for health care will often be wasted.

Thirty-four years after my article demonstrated harmful effects of the radiotherapy regimens in use at that time, the article is still being used for teaching at Harvard University (2008). Students are instructed to “Use the sign test to show that Stjernswärd was actually correct in his conclusion.” This is encouraging evidence that some scientific approaches – albeit based on ‘peasant sense’ (‘bondförnuft’) – seem to stand the test of time.

This James Lind Library commentary has been republished in the Journal of the Royal Society of Medicine 2010;103:297-300. Print PDF

References

Bonadonna G, Rossi A, Valagussa P, Banfi A, Veronesi U (1977). The CMF program for operable breast cancer with positive axillary nodes. Updated analysis on the disease-free interval, site of relapse and drug tolerance. Cancer 39(6 Suppl):2904-15.

Cuzick J, Stewart H, Peto R, Baum M, Fisher B, Host H, et al. (1987). Overview of randomized trials of postoperative adjuvant radiotherapy in breast cancer. Cancer Treat Rep 71:15–29.

Cuzick J, Stewart H, Rutqvist L, Houghton J, Edwards R, Redmond C, et al. (1994). Cause-specific mortality in long-term survivors of breast cancer who participated in trials of radiotherapy. J Clin Oncol 12:447–53.

Cuzick J (2005). Radiotherapy for breast cancer. Journal of the National Cancer Institute 97:406-407; doi:10.1093/jnci/dji086.

Early Breast Cancer Trialists’ Collaborative Group (2000). Favourable and unfavourable effects on long-term survival of radiotherapy for early breast cancer: an overview of the randomized trials. Lancet 355;1757–70.

Giordano SH, Kuo Y-F, Freeman JL, Buchholz TA, Hortobagyi GN, Goodwin JS (2005). Risk of cardiac death after adjuvant radiotherapy for breast cancer. J Natl Cancer Inst 97:419–24.

Harvard University (2008).

Haybittle JL, Brinkley D, Houghton J, A’Hearn RP, Baum M (1988). Postoperative radiotherapy and late mortality: evidence from the Cancer Research Campaign trial for early breast cancer. BMJ 298:1611-1614.

Høst H, Brennhovd IO, Loeb M (1986). Postoperative radiotherapy in breast cancer – long-term results from the Oslo study. Int J Radiat Oncol Biol Phys 12:727–732.

International Breast Cancer Study Group (2009). Goldhirsch A, Ingle JN, Gelber RD et al. Thresholds for therapies: highlights of the St Gallen International Expert Consensus on the Primary Therapy of Early Breast Cancer. Ann Oncol 20:1319-29.

Levitt S, McHugh R (1975). Early breast cancer and postoperative irradiation. Lancet 2:1258-1259.

Levitt S, McHugh R, Song C (1977). Radiotherapy in the postoperative treatment of operable cancer of the breast. Part 1. Critique of the clinical and biometric aspects of the trials. Part 2. A re-examination of Stjernswärd’s application of the Mantel-Haenszel method. Evaluation of the effect of the radiation on the immune response and suggestion for postoperative radiotherapy. Cancer 39:924-949.

Mantel N, Haenszel W (1959). Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst 22:719-48.

Nimeus-Malmström E (2009). Prognostic and treatment predictive factors for radio-and chemotherapy “resistance” in breast cancer patients- a step towards personalized medicine. Doctoral Dissertation, Department of Oncology, Clinical Science, Lunds University, Lund, Sweden.

Paterson R, Russell MJ (1959). Clinical trials in malignant disease. Part III. Journal of the Faculty of Radiologists 10:175-180.

Stjernswärd J (1974). Decreased survival related to irradiation postoperatively in early operable breast cancer. Lancet 2:1285-1286.

Stjernswärd J (1977). Adjuvant radiotherapy trials in breast cancer. Cancer 39: 2846-2867.

Stjernswärd J, Day N (1978). Rebuttal of two articles by Levitt et al. Cancer 1977. Cancer 41:381.

Stjernswärd J, Muenz LR, von Essen CF (1976). Postoperative radiotherapy and breast cancer. Lancet 307:749.

Wirapati P, Sotiriou C, Kunkel S, et al. (2008). Meta-analysis of gene expression profiles in breast cancer: toward a unified understanding of breast cancer subtyping and prognosis signatures. Breast Cancer Res 10:R65.