Posted by minervation on April 18, 2025
This podcast is based on the JLL Essay 2.9, Recognizing researcher/sponsor biases and fraud.
Transcript
Richard Lehman: My name is Richard Lehman, and I’m introducing the section on bias in the James Lind Library together with Raj Mehta, who is a young family practitioner in America, in Florida. I, on the other hand, I’m long retired, and I have an interest in primary care and evidence based medicine. And the history of medicine.
So together we’re going to discuss the problem of recognizing researcher and sponsor biases and fraud. That’s section 2. 9 of the essay section in the James Lind library. Raj, what are your observations on this chapter?
Raj Mehta: First of all, Richard, I’m delighted to be joining you and chatting about this. Historical Perspectives on Medical Bias
I had a very interesting time reading this essay, because I feel in my day to day practice I’m very much in the real world and things that are happening in the [00:01:00] current, and sometimes taking a step back and reading history reminds me that some of the new problems we’re dealing with are perhaps not so new, and it’s reassuring to know that we’ve been dealing with some of these things in the past, and that sometimes history does repeat itself.
Richard Lehman: Yes, I think in fact bias and fraud are the default positions for medicine over several millennia, and it’s only recently that we’ve managed to overcome the basic human need to sell cures that don’t work and make claims that are untrue. And we can see this very vividly in certain modern situations such as the arrival of COVID immediately brought people out claiming false efficacy for things like hydroxychloroquine or ivermectin, [00:02:00] and there is this saying that it takes 10 times as much effort to correct a bias than it does to generate one. Anyway, back to the essay, what were your thoughts on the actual text there?
Raj Mehta: Well, I thought it started off in a very clear way. I suppose the first question that comes to my mind is, why does bias exist? What’s going on here? Why do we worry about it?
And this chapter begins in 1764 with a gentleman, a physician named James, Dr. R. James, and he has this dissertation on fevers of inflammation and distempers, and he has these secret successful treatments for smallpox, yellow fever, slow fever, rheumatism, and so on, filled with testimonials from satisfied patients, and all this data about how it can reduce mortality and it’s a miraculous cure all, and I was like, this could be like a modern day advertisement I see [00:03:00] on a YouTube advertisement or something like that.
And it was very interesting. I mean, the writer of the essay just called him a snake oil salesman, which I think is a probably accurate description based on our modern knowledge of it. But even in the context of that time, it seems that there are concerns of people promising cures that didn’t really exist and whether they believed in them and they knew, or they knew they were lying.
I’m not so sure. And I guess that’s kind of where this begins, which is that why would individuals kind of make these false promises, what do they gain from it, where is this bias coming from, and how can we fight it, how can we find truth for ourselves as practitioners and for our patients?
Modern Examples of Bias and Fraud
Richard Lehman: Yeah, and it’s extraordinary how this recurs, as you say, I mean, snake oil is actually quite efficacious, I’m told, for certain types of eczema.
Whereas this cure that he was proposing is it was was actually terrible really. I mean he was proposing it for life [00:04:00] threatening illnesses and making epidemiological claims that were completely untrue.
I think if you want to go to the United States, which you probably do Oliver Wendell Holmes Sr. was a doctor. His son became the most famous American judge of all time. But Oliver Wendell Holmes was a wonderfully witty writer and he was always trying to, dismiss claims, particularly for homeopathy and osteopathy. And these claims are still still being made today. So there is this deep human need to look outside medicine and traditional methods and believe in the idea of a cure all.
And we have to accept that this is a human need and that it’s actually counter to the human brain to want to disprove things rather than promote them. We all have our favorite treatments and [00:05:00] these vary by doctor and they vary by country and so we are biased by nature, and it’s unnatural to try and discover things that disprove our fixed ideas.
So, we’re constantly working against the grain, and evidence based medicine is a constant fight that can never give up. And, uh, I think the current political situation in America proves that.
Raj Mehta: It’s such an interesting point because I think you have this great perspective. There’s so many oral transmissions of communications of belief and our bias by nature is to believe people and have trust and that we have to kind of create a degree of skepticism and criticism to counter our inherent biases that we have.
And I think this is what the history of medicine has shown, especially in the 1800s, you begin having the first attempts to counter this with [00:06:00] scientific approaches, where you have your first attempt at studies, where you compare different groups. You have your crossovers, you have the introduction of placebo for comparison, and you have the beginnings of clinical science developing, specifically as a form of skepticism, to counter these biases and these narrative or oral beliefs, and to try and bring in epidemiology, bring in data, and a scientific process to kind of base our judgments.
Richard Lehman: Yes, and I mean, it’s much easier to earn money from things that don’t work than it is from things that do work, oddly enough. The vitamin industry and the fitness industry, wellness industry, you could even say certain forms of counselling, and all the mineral supplements in the world. They really have no proven effect, and yet they probably account economically for almost as much as medicines that do [00:07:00] have an effect.
So I think that one of the biases that perhaps isn’t mentioned in the essay, but is very fundamental, is where do you pay attention, where do you look, and our attention is constantly diverted into disproving things that don’t work rather than looking positively at the things that do work and a very good example of this was this business in the covid epidemic where everybody was looking at ivermectin and treatments that should never have been considered in the first place.
So I think selection of topic biases is a big thing. And it’s also something that we see in conventional medicine too. As soon as people started coating stents, coronary stents, the attention of the cardiac community, the interventional [00:08:00] cardiologists swung entirely towards what coating you put on stents.
And there were actually no good trials that compared them with bare metal stents over a significant period of time and with real outcomes. So that’s one thing. And then we can come to those three biases that are mentioned in the essay. Shall we do that?
Raj Mehta: Yes, I think we’ve used a few terms here that’s worth defining for audience.
Defining Fraud and Bias in Research
Raj Mehta: The first is discussing fraud. Fraud is intention to deceive or misrepresent in research. It’s a deliberate act to manipulate or fabricate data results. That’s different from bias, which refers to systemic errors or deviations in research that can distort results and lead to inaccurate conclusions.
Richard Lehman: Yes, that’s very important, and because we try to be polite to our colleagues and to the world in general, we underestimate, I think, [00:09:00] the potential for deliberate fraud, and whenever it’s looked for, the figures are fairly horrific, and that’s in standard medical journal papers. Bias is almost universal because as I said at the beginning, we tend to be biased in favor of the things we believe in. And so we design trials that not hypothesis disproving, but trying to create a trial in such a way as to forward our favorite theory. And that’s design bias, which is one of the three biases that are specifically mentioned in this essay.
The other two are analysis bias and reporting bias with some overlap between the two. So do you want to move on from that?
Raj Mehta: Yes, I think this is an interesting time because you really follow history from the 1800s and 1900s [00:10:00] you go from dubious medical claims or medical practitioners to now in the early 20th century the rise of the pharmaceutical industry, the medical industry, and the new challenges in for-profit organizations and the concerns of bias and fraud, and their involvement.
Richard Lehman: Yes. And I’m afraid I think this has got a lot worse in the last two decades. Maybe I just think that because I reviewed the medical literature over that period, but I was quite horrified as oncology in particular developed treatments that were effective, particularly specific antibody treatments, but then confuse them with all sorts of treatments that are of dubious efficacy and may indeed make things worse for patients who are inevitably dying.
And this was a sort of design bias, but also an analysis bias. And I think one of the best [00:11:00] examples of analysis biases, when instead of looking at overall mortality, you look at regression or progression free survival. This progression-free survival idea sounds great and you can merge outcomes and make them sound much better than they actually are for patients.
And you can also analyze harmful effects in such a way as to minimize them so that those are possibly classified as analysis biases, but could also be characterized as reporting biases.
Raj Mehta: What’s interesting here is that these new concerns for bias are becoming highly specialized whereas in the 1800s there was people making claims and you could kind of use scientific inquiry. Now it’s very specialized. You need to be an expert in biostatistics and have a degree of understanding of clinical trial processes and really starting to scrutinize this. It makes it more and more challenging, you know, [00:12:00] for the average doc, much less the average patient or individual to deal with.
Challenges in the 21st Century
And I think that transitions to where we are now in the 21st century, where we don’t just now have the concerns from purveyors of claims or pharmaceutical or industry sponsors, but now we have social media and social media influencers, the claims they make, the biases they may have based on their own branding and their own personal goals, even out of good personal ideology and the challenges when you don’t have the expertise to perhaps discern certain things related to this.
Richard Lehman: I think that’s right, and I think it’s deliberate, and I think there’s a lot of gaming by industry in the way it presents things makes it really very, very difficult for the average reader, even a careful reader of the original research to detect bias, and it’s a terrible reflection on our scientific culture that we have this enormously well [00:13:00] funded, bias production line, as it were, whereas the detectors of bias are actually being actively defunded at the moment in the United States and elsewhere.
So we, we’re hoping that artificial intelligence might come to the rescue by providing very robust analyses of a kind that we can’t with our puny human brains, but that’s mere hope.
Conclusion and Future Outlook
Richard Lehman: So I think we can end on that note and go on to discuss this slightly depressing topic in various other aspects in further podcasts.
Raj Mehta: I will. And to end on a slightly optimistic note, in history when we’ve come across challenges and we’ve had societal conflict in our trust for individuals, organizations, and concerns with bias and fraud, I feel like we’ve had a robust immune response and had new advances in science and communication that allows us to have better scrutiny and make wise [00:14:00] decisions of the population.
And so as it’s happened in the 1800s and 1900s, I’m hoping it will continue the same in the 21st century.