One of Cochrane’s founders has been dismissed from the organisation’s board. His dismissal is symptomatic of deeper problems in evidence-based medicine.
Photo: Einar Nilsen
In the last 25 years, Cochrane has grown from being a small, voluntary initiative to becoming the world’s largest independent producer of systematic reviews of healthcare interventions. It is to Cochrane that we turn, doctors, patients, next of kin and decision makers alike, when we seek answers to clinical questions. Do vitamin A supplements help treat cystic fibrosis? Do omega-3 supplements prevent myocardial infarction? The best and most up-to-date reviews of the available evidence base for thousands of such questions are to be found in Cochrane.
Professor Peter Gøtzsche is one of the organisation’s founders and the director of the Nordic Cochrane Centre. He is an uncompromising critic of ‘big pharma’, but also of Cochrane itself when he finds it necessary. For some, he is the defender of the medical profession’s true integrity; for others, an enfant terrible.
For a long period there has been tension between Gøtzsche and Cochrane’s management. On 17 September 2018, the conflict came to a head, and Gøtzsche was dismissed from the board by the narrowest possible margin, whereupon four board members resigned in protest. According to Gøtzsche himself, his dismissal forms part of a hidden agenda to distance Cochrane from its original purpose and principles in favour of a more commercial business model (1). According to the management’s statement it was, on the contrary, Gøtzsche’s worrisome ‘behaviour’ over ‘many years’ that necessitated his dismissal (2).
Beneath the outward drama lie deeper, more structural problems which revolve around more than who sits on Cochrane’s board. Criticism levelled by both Gøtzsche and others in recent years has concerned the degree to which we can trust the results from large-scale clinical studies (3). This kind of fundamental criticism is nothing new (4). Some people believe that we are undergoing a crisis in evidence-based medicine (5).
The influence of the pharmaceutical industry is an acknowledged aspect of the problem. The industry partly sets the research agenda by defining new diseases and the need to treat these; it partly influences the research process by defining components such as inclusion criteria, endpoints and length of sponsored studies; and it partly influences the scientific literature through selective publication. For example, a review of industry-sponsored outcome studies of antidepressants found that 37 out of 38 positive studies were published, but only 14 out of 36 negative ones (6).
To this may be added the problem presented by the overwhelming volume of evidence. This is partly a resource problem. Of the estimated USD 250 billion spent each year on biomedical research, it is calculated that approximately 85 % is devoted to studies that have little or no value (7). The problem lies in determining the studies to which this applies and in preventing too many of them from insinuating their way into the evidence base for meta-analyses and systematic reviews. The complexity of large-scale clinical studies is increasing all the time. A simple ten-page article may have up to 8 000 pages of supporting material (8). What must be plucked from this for the finished article may result in a high degree of (unintended) selection bias. Where several research communities share the same world view (as they often do), the same selection bias can transplant itself to many single-case studies and result in significant bias in later meta-analyses.
At the same time, the greatest benefits of pharmaceutical interventions have to a large degree already been realised. Evidence-based medicine is thus becoming increasingly the science of marginal benefits (5). Studies whose potential benefit is marginal can easily lead to their statistical significance being unaccompanied by clinical significance. One example of this is the new US guidelines for hypertension, which when applied to the general population mean that 63 % of all persons over the age of 45 need treatment (9).
Several initiatives in recent years represent attempts to cure these and other structural problems. Mandatory registration of studies is one example, although the effect on publication bias, for instance, has been less than was hoped for (8). The upcoming requirements of the International Committee of Medical Journal Editors for data sharing are another recent example. However, even more important are the requirements we should set for all of us, irrespective of where we are in the chain from clinical research question, via research to publication and clinical use of the results. The scientific method’s primary hallmark is still the principle of constantly and critically challenging accepted truths – and rejecting them when they are no longer valid. This requires that truth is more important than consensus. Let us hope that this is not what has failed in the recent events at Cochrane.
