How Science Builds Its House: The Story of Cochrane, Evidence Databases, and What It Takes to Know Something

Dr. Aakash Kembhavi, MD (Ayu), PGDMLS, MS (Counselling & Psychotherapy)

“It is a capital mistake to theorize before one has data.” — Arthur Conan Doyle (Sherlock Holmes, A Scandal in Bohemia)

“We are all victims of our own experience, and the problem is that our experience is too narrow to be trusted.” — Archie Cochrane, Effectiveness and Efficiency, 1972

Prologue: The House Analogy

Imagine you are asked to build a house. You have excellent raw materials — seasoned timber, high-grade stone, centuries-old architectural wisdom. You have skilled craftsmen who have learned their trade through years of apprenticeship. You have a vision of what the finished structure should look like.

Now imagine that no one has ever drawn a blueprint. No one has tested whether the foundation will hold. No one has measured whether the walls are plumb. Every craftsman builds from memory and intuition, each convinced that his method is correct, none able to show the other why.

The house may still stand. Many such houses do, for a while. But when the rains come and one wall cracks, no one knows which decision caused it — the timber choice, the foundation depth, the mortar composition, or simply the weather. And so the same mistake will be made in the next house, and the one after that, because there is no record, no standard, no way of learning from failure at a systemic level.

This is the situation that biomedicine found itself in through most of the twentieth century. And this is, without exaggeration, the situation that Ayurvedic medicine finds itself in today.

The story of how modern medicine built its house — how it created the infrastructure of evidence, the architecture of systematic knowledge, the quality standards for clinical claims — is not just biomedical history. For the Ayurvedic community, it is a roadmap. Understanding how Cochrane was built, why it was needed, what problems it solved, and how it continues to evolve is not an exercise in admiring another system. It is a lesson in what every serious knowledge system must eventually undertake if it wishes to be trusted with human lives.

Part I: Before Cochrane — The Age of Eminence

To understand what Cochrane built, we must first understand what existed before it.

For most of medical history — and this applies as much to biomedicine as to Ayurveda — clinical authority was personal. The most experienced physician, the most senior practitioner, the most published academic set the standard of care. This is what epidemiologists call Eminence Based Medicine: not evidence of what works, but the authority of who says so.

The problems with this are not trivial. By the mid-twentieth century, biomedicine had accumulated decades of treatments that were accepted as standard care not because they had been rigorously tested but because influential physicians had used them and reported success. Tonsillectomies were performed on millions of children. Hormone replacement therapy was prescribed widely. Specific dietary regimens were recommended with confidence. Many of these practices, when eventually tested systematically, turned out to be ineffective, unnecessary, or actively harmful.

The fundamental cognitive vulnerability that eminence-based practice exploits is one of the most well-documented phenomena in human psychology: we are exceptionally good at finding patterns that confirm what we already believe.

“The human understanding when it has once adopted an opinion draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises, or else by some distinction sets aside and rejects.” — Francis Bacon, Novum Organum, 1620

Four hundred years before the randomized controlled trial, Bacon had already diagnosed the problem. The solution would take considerably longer to architect.

Part II: Archie Cochrane — The Man Who Refused to Pretend

Archibald Leman Cochrane was a Scottish physician and epidemiologist whose 1972 monograph Effectiveness and Efficiency: Random Reflections on Health Services became one of the most consequential texts in the history of medicine — not because it introduced new treatments, but because it asked a question that the medical establishment had conspicuously avoided:

How much of what we do actually works?

Cochrane had been a prisoner of war in World War II, serving as a medical officer in a German prison camp with almost no medicines and very limited resources. He observed, often with astonishment, that patients recovered from conditions that textbooks said required aggressive intervention. He began to question, systematically, what the actual therapeutic contribution of medicine was — as distinct from what the body did on its own.

His central argument in Effectiveness and Efficiency was stark: the medical profession was spending vast public resources on treatments whose effectiveness had never been rigorously demonstrated, and the reason this continued unchallenged was that no one had built a system for systematically collecting and evaluating the available evidence.

“It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials.” — Archie Cochrane, 1979

This single sentence — written three years before his death in 1988 — was the seed of the Cochrane Collaboration.

What made Cochrane’s thinking revolutionary?

Three things, each of which has direct relevance to Ayurveda:

First, he distinguished between efficacy (does it work under ideal conditions?) and effectiveness (does it work in real clinical practice?). This distinction matters enormously for Ayurveda, where treatments that appear to work in experienced practitioners’ hands may or may not translate to reproducible outcomes across diverse practitioners and patient populations.

Second, he insisted that the randomized controlled trial — the experiment in which patients are randomly assigned to treatment or control groups, eliminating selection bias — was the most reliable tool for establishing whether a treatment caused the observed outcome. Not the only tool, but the most reliable one for specific questions.

Third, and most importantly, he argued that individual trials were not enough. Individual trials produce individual findings. What medicine needed was a systematic synthesis of all available trials on a given question — a meta-analysis that could distinguish consistent signal from random noise.

This third insight was the architectural foundation of what would become Cochrane.

Part III: Building the Infrastructure — How the Cochrane Collaboration Was Born

Cochrane died in 1988, but his challenge lived on. In 1992, Iain Chalmers — a British epidemiologist who had worked with Cochrane — established the first Cochrane Centre in Oxford. The following year, the Cochrane Collaboration was formally launched in 1993, bringing together researchers from across the world under a single mission: to produce, maintain, and disseminate systematic reviews of the effects of healthcare interventions.

The founding of Cochrane was not a single dramatic event. It was a sustained, painstaking act of institutional construction. Understanding how it was built is as important as knowing what it built.

Step 1: Establishing a Common Language — The Protocol

Before Cochrane could review anything, it had to establish what a review was. The first decision was that every systematic review would begin with a publicly registered protocol — a pre-specified document that stated, before any data was examined:

  • What question was being asked
  • What types of studies would be included or excluded, and why
  • What outcomes would be measured
  • How data would be extracted and synthesized
  • How disagreements between reviewers would be resolved

The protocol is not bureaucratic formality. It is the single most important anti-bias mechanism in systematic review methodology. By specifying the analysis plan before seeing the data, it prevents the universal human tendency to frame the question in whatever way makes the favored conclusion most likely.

For the Ayurvedic community: this is the equivalent of writing your complete Nidana Panchaka assessment criteria before examining your first patient in a study — and committing publicly to those criteria before you know what you will find. It is profoundly uncomfortable. It is also profoundly honest.

Step 2: The Hierarchy of Evidence

Cochrane did not treat all evidence as equal. It developed — and has continued to refine — a hierarchy of evidence quality, recognizing that different study designs answer different questions with different reliability.

At the top of the hierarchy for questions of treatment effectiveness:

  1. Systematic reviews and meta-analyses of high-quality RCTs
  2. Individual RCTs with adequate randomization, allocation concealment, and blinding
  3. Controlled trials without randomization
  4. Cohort studies — following groups forward in time
  5. Case-control studies — looking backward from outcome to exposure
  6. Cross-sectional studies
  7. Case series and case reports
  8. Expert opinion and clinical experience

Notice that expert opinion and clinical experience — which constitute the primary evidence base of most Ayurvedic practice today — sit at the bottom of this hierarchy. This is not because experience is worthless. It is because experience, without the controls that eliminate systematic bias, cannot reliably distinguish what the treatment caused from what the patient’s body did independently.

Cochrane did not create this hierarchy to dismiss clinical wisdom. It created it to show clinicians where their knowledge was strong, where it was uncertain, and where it urgently needed testing.

Step 3: Standardized Data Extraction

Every systematic review involves extracting data from multiple individual studies. Cochrane developed standardized data extraction forms — templates that ensured every reviewer collected the same information in the same way, regardless of which research group they belonged to or which country they worked in.

This standardization was not trivial. Before it existed, different research groups analyzing the same studies reached different conclusions — not because of dishonesty, but because they were measuring different things, defining outcomes differently, and handling missing data differently. Standardization made comparison possible.

Step 4: The GRADE System — Grading the Quality of Evidence

As systematic review methodology matured, it became clear that simply pooling studies was not sufficient. The quality of the underlying evidence varied enormously. Some RCTs were well-designed with rigorous blinding; others were small, poorly conducted, and prone to bias. A systematic review that treated all RCTs as equal would produce a misleading pooled estimate.

In 2004, a working group that included Cochrane researchers developed the GRADE system — Grading of Recommendations, Assessment, Development and Evaluation. GRADE evaluates evidence quality across five domains:

  • Risk of bias — was the study well-designed and conducted?
  • Inconsistency — do different studies point in the same direction?
  • Indirectness — do the study populations and interventions match the clinical question?
  • Imprecision — are the sample sizes large enough to draw reliable conclusions?
  • Publication bias — is there reason to believe negative results were not published?

Based on these domains, evidence is rated as High, Moderate, Low, or Very Low quality. Clinical recommendations derived from that evidence are rated as Strong or Conditional.

GRADE is now used not only by Cochrane but by the World Health Organization, national health technology assessment bodies, and most major clinical guideline developers worldwide. It is the global standard for translating research evidence into clinical recommendations.

Step 5: Continuous Updating

Unlike a textbook, a Cochrane review is a living document. As new trials are published, reviews are updated. If new evidence changes the conclusion, the conclusion changes. This is the defining feature that separates a systematic review from a narrative review written by an expert: it is not loyal to any prior conclusion. It follows the evidence wherever it leads.

“In science it often happens that scientists say, ‘You know that’s a really good argument; my position is mistaken,’ and then they actually change their minds and you never hear that old view from them again.” — Carl Sagan

This willingness to change conclusions in response to new evidence is not a weakness. It is the most important quality a knowledge system can have.

Part IV: The Evidence Ecosystem — Beyond Cochrane

Cochrane is the most rigorous and influential component of what is now a large, interconnected ecosystem of evidence databases. Understanding this ecosystem is essential for any researcher or practitioner who wants to engage seriously with the evidence literature.

4.1 PubMed / MEDLINE

What it is: The primary bibliographic database of biomedical literature, maintained by the United States National Library of Medicine. Currently indexes over 35 million citations from more than 5,200 journals worldwide.

What it does: PubMed does not evaluate evidence quality. It indexes published research. A paper in PubMed may be a high-quality RCT or a poorly conducted case series — the database does not distinguish. What it provides is access to the primary literature.

Relevance to Ayurveda: PubMed does index Ayurvedic research, including publications from journals like the Journal of Ethnopharmacology, Ancient Science of Life, and Ayu. The presence of Ayurvedic research in PubMed is not evidence of quality; it is evidence of publication. Quality must be assessed independently.

4.2 Embase

What it is: A biomedical and pharmacological database produced by Elsevier, with particular strength in European literature and drug-related research.

What it does: Provides broader geographic coverage than PubMed, particularly useful for systematic reviews that aim to capture all available evidence globally.

4.3 CINAHL — Cumulative Index to Nursing and Allied Health Literature

What it is: The primary database for nursing and allied health research.

Relevance to Ayurveda: CINAHL indexes research on complementary and alternative medicine, including some Ayurvedic studies. For Ayurvedic practices that overlap with nursing and rehabilitation — Panchakarma for rehabilitation, dietary counselling, yoga — CINAHL is a relevant search database.

4.4 The Cochrane Central Register of Controlled Trials (CENTRAL)

What it is: A curated database of controlled trials identified through systematic searching of PubMed, Embase, and other sources — including hand-searching of journals not indexed electronically. CENTRAL is the most comprehensive database of RCTs in existence.

Why it matters: Many controlled trials — particularly older ones, and those published in languages other than English — are not adequately indexed in PubMed. CENTRAL’s systematic hand-searching captures these. For Ayurvedic researchers conducting systematic reviews, CENTRAL is an indispensable resource.

4.5 WHO International Clinical Trials Registry Platform (ICTRP)

What it is: A global registry of clinical trials, integrating data from national registries including India’s Clinical Trials Registry — India (CTRI).

Why it matters: Trial registration before commencement is now mandatory in most jurisdictions, including India. It prevents outcome switching — the practice of changing the primary outcome of a trial after seeing the results. For Ayurvedic researchers, registering trials in CTRI (which feeds into WHO ICTRP) is both a regulatory requirement and an ethical one.

4.6 PROSPERO — International Prospective Register of Systematic Reviews

What it is: A database maintained by the Centre for Reviews and Dissemination at the University of York, in which systematic review protocols are registered before the review begins.

Why it matters for Ayurveda: PROSPERO registration is the systematic review equivalent of trial registration. It establishes, on public record, what question was being asked and how it would be answered — before the reviewer has seen the results. Any Ayurvedic systematic review that is not PROSPERO-registered is open to the suspicion of post-hoc question framing. Registration is free and takes less than an hour.

4.7 DHARA — Digital Helpline for Ayurveda Research Articles

What it is: India’s own bibliographic database for Ayurvedic research literature, developed by the Central Council for Research in Ayurvedic Sciences (CCRAS).

Why it matters: DHARA indexes Ayurvedic journals that are not captured in PubMed or Embase. For systematic reviews of Ayurvedic interventions, DHARA search is not optional — it is essential for capturing the full body of relevant literature.

4.8 IndMED and NLM India

What it is: IndMED is an Indian biomedical literature database covering Indian journals. NLM India is a collaborative initiative to strengthen biomedical informatics infrastructure in India.

Relevance: These databases ensure that research published in Indian journals — which constitute a significant proportion of Ayurvedic clinical research — is captured in systematic searches.

Part V: The Methodological Architecture — What It Actually Takes to Know Something

The databases and institutions described above are the infrastructure. But the more fundamental question is epistemological: what does it actually take, in terms of process and standards, for a knowledge system to make a credible claim that a treatment works?

The answer involves not one step but an interlocking sequence. Think of it as a ladder, where each rung must be secure before the next can be climbed.

Rung 1: The Clinical Observation — Hypothesis Generation

Every evidence journey begins with clinical observation. A practitioner notices that patients with a particular presentation respond to a particular intervention. This observation is the raw material of hypothesis generation. It is valuable. It is also, by itself, nothing more than a hypothesis.

The mistake — made routinely in Ayurvedic practice and documented extensively in biomedical history — is to treat the clinical observation as the endpoint rather than the starting point. The observation says: something interesting may be happening here. Science says: let us find out whether it is actually happening, and why.

Rung 2: The Case Report and Case Series — Documentation and Pattern Recognition

Structured case documentation — recording patient demographics, presenting complaints, diagnostic findings, intervention details, and outcomes in a standardized format — converts clinical experience into data. A case series of twenty patients treated with a specific Ayurvedic formulation for a defined condition, with pre- and post-treatment outcome measures, is far more informative than twenty years of undocumented clinical experience.

Case reports and series are at the lower end of the evidence hierarchy not because they are useless but because they cannot control for confounding. Without a comparison group, we cannot know whether the patients improved because of the treatment, despite the treatment, or regardless of the treatment.

Rung 3: Observational Studies — Finding Associations

Cohort studies (following treated and untreated groups forward in time) and case-control studies (comparing those with and without the outcome, looking backward at exposure) allow us to identify associations between interventions and outcomes in real-world settings. They are essential for generating hypotheses about causation and for studying conditions where RCTs are not feasible.

For Ayurveda, well-designed observational studies of clinical practice — using standardized diagnostic criteria, validated outcome measures, and appropriate statistical analysis — represent a realistic near-term evidence-building strategy. They cannot prove causation, but they can establish whether associations worth testing further actually exist.

Rung 4: The Pilot RCT — Feasibility and Signal Detection

Before committing to a large, expensive RCT, the standard practice is to conduct a pilot trial — a small-scale study designed not to prove efficacy but to test whether the trial design is feasible. Can patients be recruited? Can the intervention be standardized? Can the outcome measures be collected reliably? Does the treatment show any preliminary signal of benefit?

A well-conducted pilot RCT in Ayurveda is not a trivial achievement. It requires ethical clearance, trial registration, standardized intervention protocols, validated outcome measures, and a statistician involved from the design stage — not added at the end to salvage the analysis.

Rung 5: The Confirmatory RCT — Establishing Efficacy

The confirmatory RCT is the experiment designed to test whether a treatment works. Its essential features:

  • Randomization: patients are assigned to treatment or control by chance, eliminating selection bias
  • Allocation concealment: the randomization sequence is hidden from those enrolling patients, preventing manipulation
  • Blinding: where possible, patients, treating practitioners, and outcome assessors do not know which treatment was received
  • Pre-specified outcomes: primary and secondary outcomes are defined in the protocol before the trial begins
  • Adequate sample size: calculated a priori based on the expected effect size and acceptable error rates
  • Independent data monitoring: a data safety monitoring board reviews interim results and can stop the trial early if necessary

Each of these features exists not as bureaucratic hurdle but as a specific defense against a specific bias that has been documented to distort clinical findings.

For Ayurveda, blinding is often cited as a design challenge — how do you blind patients to whether they received Panchakarma? This is a real methodological challenge, but it is not unique to Ayurveda. Surgery cannot be blinded either. The methodological literature on active placebos, matched sensory experiences, and practitioner-blinding offers partial solutions. The challenge demands creative methodological thinking, not abandonment of the standard.

Rung 6: The Systematic Review — Synthesizing the Evidence

Once multiple RCTs of an intervention exist, the systematic review synthesizes them. It asks: across all well-conducted trials of this intervention for this condition, what does the totality of evidence show?

The systematic review is more reliable than any individual trial because individual trials are subject to random variation. A treatment that is genuinely effective may appear ineffective in one poorly-powered trial. A treatment that is genuinely ineffective may appear beneficial in one trial with a chance imbalance in baseline characteristics. The systematic review reduces the influence of such random noise.

When the systematic review includes a quantitative synthesis of results — a meta-analysis — it produces a pooled estimate of effect size with a confidence interval, giving clinicians the most reliable available answer to the question: how large is the effect of this treatment, and how certain are we?

Rung 7: The Clinical Practice Guideline — Translating Evidence into Practice

The endpoint of the evidence ladder is the clinical practice guideline — a structured set of recommendations for clinical practice, graded according to the quality of the underlying evidence (using GRADE or an equivalent system), developed by a panel that includes both researchers and clinicians, and updated as new evidence emerges.

A clinical practice guideline is the formal mechanism by which a knowledge system says: “This is what we know, this is how confident we are, and this is what we recommend on that basis.” It is the difference between a practitioner deciding what to do based on what they were taught twenty years ago and a practitioner making decisions grounded in the best currently available evidence.

Ayurveda has clinical texts. It does not yet have evidence-graded clinical practice guidelines. This is the gap.

Part VI: The Cost of Building — What It Actually Takes

It would be dishonest to present the evidence-building enterprise without acknowledging its costs. Building the infrastructure of scientific knowledge is slow, expensive, institutionally demanding, and intellectually humbling.

Time

A single well-conducted RCT, from protocol development to publication, typically takes three to seven years. A systematic review takes one to three years for a well-resourced team. The Cochrane Collaboration took decades to build to its current scale. Evidence accumulates slowly because reality does not yield its secrets quickly.

Resources

Funding for clinical research is a persistent structural problem in Ayurveda. CCRAS and the Ministry of AYUSH allocate some research funding, but the amounts are modest relative to the scale of the task. Academic institutions conducting Ayurvedic research frequently lack biostatisticians, methodologists, clinical trial coordinators, and data management infrastructure. These are not luxuries. They are prerequisites for generating credible evidence.

Intellectual Humility

Perhaps the most demanding cost is psychological. Building an evidence base requires accepting, in advance, that the results may not confirm what you believe. A well-conducted RCT of a beloved Ayurvedic formulation may show no benefit over placebo. A systematic review of a widely used procedure may reveal that the evidence for it is very low quality. This is not failure. This is information — information that allows the system to correct itself and improve.

“The scientific method is a way of trying not to fool yourself. And you are the easiest person to fool.” — Feynman

The willingness to be wrong — to design studies that could prove you wrong and then run them anyway — is the defining intellectual virtue of science. It is also the virtue that most closed knowledge systems find most difficult to cultivate.

Part VII: What This Means for Ayurveda — A Constructive Agenda

The preceding account is not a counsel of despair. It is a map. The Cochrane Collaboration did not emerge fully formed; it was built incrementally, by committed individuals who understood both the problem and the stakes. Ayurveda’s evidence-building enterprise can follow the same trajectory.

What can be done now, with existing resources:

Structured outcome documentation — Every Ayurvedic practitioner who treats patients can begin recording outcomes in a standardized format. Pre-treatment and post-treatment scores on validated instruments (SF-36, disease-specific patient-reported outcome measures, or even locally validated Ayurvedic outcome scales) transform clinical experience into data. This costs nothing except discipline.

Inter-rater reliability studies — Small-scale studies testing whether two or three trained practitioners reach the same diagnosis for the same patient, using the same criteria, are feasible in any Ayurvedic college. They do not require external funding. They require only a commitment to honesty.

CTRI registration as default — Every clinical study conducted in an Ayurvedic institution, however preliminary, should be registered in the Clinical Trials Registry India. This is now a regulatory requirement in India for any research involving human participants. Compliance should be 100%.

PROSPERO registration for systematic reviews — Every PG dissertation that involves a systematic review component should be PROSPERO-registered. This takes less than an hour and costs nothing.

Methodologist partnerships — Ayurvedic research institutions should formally partner with biostatistics departments, public health schools, and clinical epidemiology units. The methodological expertise needed to design credible Ayurvedic trials exists in India. What is missing is the institutional will to build these partnerships.

A national Ayurvedic evidence registry — A centralized, publicly accessible registry of all Ayurvedic clinical trials, systematic reviews, and observational studies — building on DHARA’s existing infrastructure — would make the existing evidence base visible, identify gaps, and prevent duplication of effort.

Epilogue: The House Revisited

We return to where we began — the house without blueprints.

The Cochrane Collaboration did not build medicine’s house. It built the tools for checking whether the house was sound — the plumb line, the spirit level, the load-bearing calculations. It took decades, the contributions of thousands of researchers across dozens of countries, sustained institutional commitment, and a willingness to report results that were inconvenient, disappointing, or practice-changing.

It was built because Archie Cochrane, prisoner of war, sat in a German camp with inadequate medicines and the sharp intelligence to ask: what is actually helping these people? Not what tradition said should help. Not what authority prescribed. Not what intuition suggested. What was actually, demonstrably, reproducibly helping.

That question — simple, honest, and genuinely difficult to answer — is the beginning of every evidence-based knowledge system that has ever been built. It is the question that Ayurveda, with its remarkable clinical heritage, its sophisticated diagnostic framework, and its three millennia of accumulated observation, must now have the courage to ask of itself.

Not because the questioner is hostile. Not because the tradition is not valuable. But because the patients who will receive Ayurvedic care in the next fifty years deserve to know that what they are receiving has been tested by the same standard of honesty that Archie Cochrane demanded of medicine in 1972.

The house is not yet built. But the tools exist. The blueprint is available. What remains is the will to build.

Share your thoughts in the comments below.