Two Of Medicine's Top Editors Say You Can't Believe Much Of It. Cochrane Shows Why

You have heard the phrase used as encouragement, and you have heard it used to end a conversation. Trust the science.

Two of the most powerful editors in medicine have both said, in print, that you cannot take that instruction at face value. Neither of them is a crank. Both ran journals that decide what counts as medical knowledge.

And a Cochrane review has since measured the mechanism they were worried about.

What the editor of The Lancet published

In April 2015, Richard Horton, editor-in-chief of The Lancet, wrote a column reflecting on a symposium about reproducibility. He did not hedge:

The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue.

Richard Horton, editor-in-chief, The Lancet

That is the editor of one of the two most influential medical journals on earth, in his own journal.

Two things about that quote, because it gets used badly.

First, it is an opinion column, not a study. It is Horton’s judgment, not a measured figure, and anyone citing it as though somebody counted is overreaching. We are citing it because of who said it and where, not because it is data.

Second, and this is the part that usually gets cut: he was arguing for better science, not against science. Read what he names as the causes in the same passage. Studies with small sample sizes. Tiny effects. Invalid exploratory analyses. Flagrant conflicts of interest. An obsession with fashionable trends of dubious importance.

Every one of those is a fixable flaw in how a study was done. That is a repair list. People who quote the first sentence and stop are using an argument for rigour as an argument against rigour.

And the editor of the New England Journal

Marcia Angell spent two decades as an editor of the New England Journal of Medicine, finishing as its editor-in-chief. In 2009 she wrote:

It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.

Marcia Angell, former editor-in-chief, New England Journal of Medicine

Read the last clause again. Slowly and reluctantly. This is not somebody who came to the subject looking for a villain. It is somebody who spent twenty years at the top of the field and arrived somewhere she did not want to be.

Her essay was titled Drug Companies and Doctors: A Story of Corruption, and she named the mechanism: the industry that makes the drugs also funds most of the research that decides whether they work.

The part that is not opinion at all

Here is where this stops being two editors with a view and becomes a measured finding.

Cochrane, the most conservative evidence body in medicine, the organisation whose entire job is being unexcitable about claims, reviewed 75 papers comparing industry-sponsored drug and device studies against independently funded ones (Lundh et al., Cochrane Database of Systematic Reviews, 2017).

Industry-sponsored studies were:

• 27% more likely to report favourable efficacy results (RR 1.27, 95% CI 1.17 to 1.37)
• 34% more likely to reach favourable conclusions (RR 1.34, 95% CI 1.19 to 1.51)

And one finding that deserves its own line. In industry-sponsored studies, the conclusions agreed with the study’s own results less often than in independent ones (RR 0.83, 95% CI 0.70 to 0.98). The data said one thing and the summary said something friendlier.

The authors’ conclusion, in their words: sponsorship by the manufacturing company leads to more favourable results and conclusions, and the analyses “suggest the existence of an industry bias that cannot be explained by standard ‘Risk of bias’ assessments.”

That last clause is the whole ballgame. The standard tools that journals and reviewers use to catch bad studies do not catch this. It passes the checks. It is in the literature your doctor reads and the guidelines your care is built on, and the ordinary quality filters do not see it.

The most cited version of the same warning

A decade before Horton, John Ioannidis published a paper with a title that sounds like a headline and is in fact a careful statistical argument: Why Most Published Research Findings Are False (Ioannidis, PLoS Medicine, 2005).

It is not a rant. It is a model, and the useful part for you is that it lists precisely when a finding is less likely to be true:

• when the studies in a field are smaller
• when the effects are smaller
• when more relationships are tested with less preselection
• when there is greater flexibility in designs, definitions, outcomes and analysis
• when there is greater financial interest and prejudice
• when more teams are chasing statistical significance in the same field

Read that list again as a reader rather than a researcher. It is a checklist. Every item is something you can look for in the first paragraph of a news story about a study.

How bad does it get at the weakest tier

Two scientists set out to reproduce 53 landmark preclinical cancer studies, the kind of laboratory work that launches drug programmes (Begley and Ellis, Nature, 2012).

They confirmed the findings in about 11% of them. Six out of fifty-three.

Hold that number against how confidently laboratory results get reported. “Scientists discover compound that kills cancer cells” is almost always this tier. It is the least reliable rung on the ladder, and it is the rung that generates the most headlines, because a dish full of dying cancer cells makes a better story than a twelve-week trial where cholesterol moved a little.

This is why we flag lab work every single time we cite it, and why an article here about what green tea does to cancer cells in a dish says so in the headline.

So what do you actually do with this

Not despair, and not sneer. Grade.

Ask what kind of study it was. A pooled analysis of many human trials outranks one human trial. One human trial outranks a population survey. All of them outrank a dish of cells or a mouse.

Ask how many people, and for how long. Fourteen people for two weeks is a hint. Twenty-two trials pooled is an argument.

Ask how big the effect was, not just whether it was significant. “Statistically significant” can mean a change too small to feel. The number matters more than the word.

Ask who paid, and treat it as data. This is no longer a matter of taste. Cochrane put a number on it: a 27% higher chance of a favourable result, a 34% higher chance of a favourable conclusion, and conclusions that agree with the study’s own data less often. When a trial is funded by the company selling the thing, that is a measured tilt, not a slur. You are entitled to weigh it.

Ask whether anyone repeated it. One striking result is a lead. A result that survives other teams trying to knock it down is a finding.

Ask what the researchers themselves said about the limits. Almost every paper says plainly what it cannot show. That sentence rarely survives the trip into a headline.

Where we actually stand

Let us be plain, because the middle of this road is where nobody gets helped.

We do not think medicine is the enemy. Emergency surgery, antibiotics, insulin, a set bone: these are gifts, and anybody telling you otherwise is selling something worse than a drug.

We do think a trillion-dollar industry funding the studies that decide what gets prescribed is a conflict you are entitled to price in. That is not a conspiracy theory. It is a Cochrane review of 75 papers, and the people who wrote it said the standard quality checks do not catch it.

Most people are handed two options. Swallow it whole, or reject it whole. The first leaves you trusting a literature its own editors describe this way. The second leaves you with nothing but anecdote, which is exactly the terrain where the next person selling you something wants you standing.

There is a third position and it is the one worth having: read the evidence, and refuse to read it naively. Prefer whole foods that nobody holds a patent on, because there is no marketing budget bending that literature. Prefer interventions you can test on yourself for free. And when you do need the pharmacy, walk in knowing who funded the trial and what that does to the number.

That is why this site cites studies and grades them in the same breath. The claim and its source travel together, and the grade travels with them. When only weak evidence exists we say so. When a small trial says one thing and eight pooled trials say another, we go with the eight and tell you the cheerful result did not hold. We have run whole articles whose point was that the popular claim died: the cholesterol story on Brazil nuts, the reason usually given for why beets work.

Getting off is a legitimate goal, and it has been done in a trial

Here is where we part company with most health writing, which treats a prescription as a life sentence to be managed rather than a situation to be changed.

In the DiRECT trial, 306 people with type 2 diabetes were randomised in ordinary UK primary care practices. The intervention did not add a drug. Step one was withdrawing the antidiabetic and antihypertensive drugs, followed by a total diet replacement, stepped food reintroduction, and structured support (Lean et al., The Lancet, 2017).

At 12 months, 46% were in remission, off all antidiabetic medication, against 4% in the group receiving best-practice care. The odds ratio was 19.7. Remission tracked weight loss almost linearly: none of those who gained weight, and 86% of those who lost 15kg or more.

Two years on, 36% were still in remission, and only 40% of the intervention group were taking any antidiabetes medication against 84% of the controls. Serious adverse events in year two were fewer in the group that came off their drugs, nine against twenty-two (Lean et al., The Lancet Diabetes and Endocrinology, 2019).

Read that again. A Lancet trial, run in ordinary GP surgeries, deliberately took people off their medication, and a third of them were still off it two years later with fewer serious events than the people who stayed on.

That is our position, and it is not a fringe one. It is published in the most cited medical journal in the world. The goal is to need the pill less, and eventually not at all, by fixing what put you on it.

The difference between that and quitting on your own

DiRECT worked because of how it was done, and the how is not optional.

It was done with doctors, in primary care, with the withdrawal planned rather than improvised. It addressed the cause, rather than simply removing the treatment and hoping. It had structured support for the months afterwards, which is when most attempts fall apart. And it measured, so nobody was guessing whether it was working.

Be precise about who it studied, because precision is what separates us from the man on the internet telling everyone to flush their pills. DiRECT enrolled people with type 2 diabetes of less than six years, a BMI of 27 to 45, not on insulin. It says nothing about type 1 diabetes, where insulin is not optional and stopping it kills. It says nothing about anticonvulsants, transplant drugs, or lithium.

Knowing that difference is not timidity. It is the thing that makes the rest of the argument credible.

So: aim at getting off. Do it the way the trial did it. With a provider, with the underlying cause addressed, with a taper rather than a cliff, and knowing what rebound is before it arrives and convinces you that you were sicker than you are.

Where the design shows up

God gave you a mind that can weigh things. That is not a small gift, and outsourcing it entirely, to a slogan or to a headline, is a way of setting it down.

Discernment is not suspicion. It is care. The same attention you would give to a contract, a diagnosis, or a person selling you something belongs to a claim about your body.

“The simple believes everything, but the sensible one considers his steps.” Proverbs 14:15, NASB

The verse carries the conviction. The editors of The Lancet and Nature carry the warning, and they are not our enemies for having written it down.

What to do this week

Take one health claim you already believe and look up where it came from.

Not to debunk it. Most of them hold up fine. The exercise is to find out what tier it sits on: was it people or a dish, how many, for how long, and did anyone repeat it.

The first time you do this, one of two things happens. Either the claim gets stronger because you now know why it is true, or it quietly moves from “fact” to “interesting, unproven.” Both are upgrades on believing it because someone said it confidently.

Read this next: what happens when a prescription ends and the symptom comes back louder, which is one of the places this bias costs people most: why coming off a medication can feel like proof you still need it. For the grading in action, where three cheerful small trials met a pooled analysis of eight and lost: one Brazil nut a day closed a real nutrient gap, but not cholesterol. For the lab-tier flag in a headline, see what green tea does to cancer cells in a lab.

Sources

1. Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. “Industry sponsorship and research outcome.” Cochrane Database of Systematic Reviews, 2017;2(2):MR000033. https://pubmed.ncbi.nlm.nih.gov/28207928/
2. Horton R. “Offline: What is medicine’s 5 sigma?” The Lancet, 2015;385(9976):1380. https://doi.org/10.1016/S0140-6736(15)60696-1
3. Angell M. “Drug Companies and Doctors: A Story of Corruption.” The New York Review of Books, 15 January 2009. Quoted for the author’s position and testimony, not as research.
4. Ioannidis JPA. “Why Most Published Research Findings Are False.” PLoS Medicine, 2005;2(8):e124. https://pubmed.ncbi.nlm.nih.gov/16060722/
5. Lean MEJ, Leslie WS, Barnes AC, et al. “Primary care-led weight management for remission of type 2 diabetes (DiRECT): an open-label, cluster-randomised trial.” The Lancet, 2018;391(10120):541-551. https://pubmed.ncbi.nlm.nih.gov/29221645/
6. Lean MEJ, Leslie WS, Barnes AC, et al. “Durability of a primary care-led weight-management intervention for remission of type 2 diabetes: 2-year results of the DiRECT open-label, cluster-randomised trial.” The Lancet Diabetes and Endocrinology, 2019;7(5):344-355. https://pubmed.ncbi.nlm.nih.gov/30852132/
7. Begley CG, Ellis LM. “Drug development: Raise standards for preclinical cancer research.” Nature, 2012;483(7391):531-533. https://pubmed.ncbi.nlm.nih.gov/22460880/

Article metadata retrieved from PubMed. The Horton column is an opinion piece in The Lancet and is cited as such. Photograph from Pexels. Scripture quoted from the New American Standard Bible.

Educational content from True Health Solutions. We do not diagnose or prescribe, and nothing here is a substitute for care from someone who knows your history. The body heals. We write about what supports that design.