What Counts as Evidence?

Thoughts on Medical Credibility

One of the more revealing features of contemporary medical discussion is not that people disagree about evidence; although I think that disagreement is frequently productive. Rather it is that they usually apply different standards to identical kinds of evidence, and that this asymmetry almost always runs in the same direction.

This is not a problem of dishonesty. It is a problem of worldview.

Mention that a modest but reasonably consistent body of randomised trials suggests Saffron may confer benefits in low mood or anxiety, and a predictable set of objections materialises: the studies are small; most originate from Iran; there may be cultural bias; replication in Western settings is lacking, etc., etc.. These are legitimate concerns. Scientific production is embedded within social and economic life, and researchers working within a culture that has used a substance medicinally for centuries will bring historical assumptions to their work.

What I have come to notice is the selective application of this sociological awareness. There can be a dismissal of herbal trials tainted with ‘cultural bias’, but many drug trials [designed, funded, and analysed by pharmaceutical companies with revenues exceeding those of many nation-states] do not seem to raise the question of bias at all; at best there is a nodding acquaintance which is then averted. Cultural embeddedness is treated as contaminating results. Commercial embeddedness is treated as commendably neutral, or even as a marker of rigour.

This is not a defensible position. It is a double standard that has been mistaken for a standard.

Western institutional medicine has been granted a form of credibility by default that functions like cultural prestige. Studies from accredited Western universities are assumed rigorous before they are read. Studies from other Eastern traditions can often be assumed to be suspect before they are read. This reveals a set of assumptions in the background so naturalised they cease to register as assumptions at all.

The pharmaceutical literature has generated its own extensive record of systemic distortion, which have been highlighted by serious journals and researchers: selective outcome reporting, endpoint switching, publication bias in which negative trial results are suppressed, and the structural problem that commercially funded trials reliably produce more favourable results for the sponsor’s product than independently funded trials of the same intervention. These are documented phenomena, discussed in peer-reviewed literature and by former editors of major journals. None of this makes pharmaceutical medicine ineffective. But it means the very broad landscape of pharmaceutical research is not meaningfully cleaner than that of herbal research.

The phrase evidence-based medicine entered clinical discourse as a method to evaluate interventions through systematic appraisal rather than expert intuition alone.

A critic who says “saffron trials have methodological weaknesses” is making an argument that can be examined. A critic who says “saffron cannot work because it has not been validated through the route of institutional medicine” is making an assumption dressed as a proven claim.

Evidence is not a set of conclusions certified by powerful institutions. It is a practice of observations, of disciplined inquiry conducted by fallible people within imperfect systems. Recognising this does not undermine confidence in medicine. It is the condition for doing medicine well.

I listened to a lecture once where we were told that pure mathematics is the only discipline where you can prove something with absolute, logical finality; that physics, medicine, psychology, biology cannot prove things in the mathematical sense. Instead, they gather evidence, build models, test hypotheses, estimate probabilities, and falsify claims. The reason that they can’t deliver absolute certainty is because they rely on fallible things like measurements, instruments, interpretation, statistical interpretation and assumptions about the world.

Even the strongest medical evidence [a well‑designed RCT or a meta‑analysis] gives probabilistic confidence, not proof. [Even ignoring the funding question].

This is why scientific papers talk about their effect sizes, their confidence intervals, the p‑values and the likelihood ratios. It’s a sort of degrees of belief scale; but it is not logical inevitability.

In the end though, perhaps the most realistic evidence‑based position of all is acknowledging that no evidence arrives completely untouched by human interests, and learning to think clearly with objectivity is the best we can do.

Image attribution: Margaret Hamilton - restoration.jpg
[[File:Margaret Hamilton - restoration.jpg|Margaret_Hamilton_-_restoration]]