Marie-Caroline Schulte

# Evidence-Based Medicine – A Paradigm Ready To Be Challenged?

How Scientific Evidence Shapes Our Understanding And Use Of Medicine

Evidence-Based Medicine – A Paradigm Ready To Be Challenged?

Marie-Caroline Schulte

# Evidence-Based Medicine – A Paradigm Ready To Be Challenged?

How Scientifc Evidence Shapes Our Understanding And Use Of Medicine

Marie-Caroline Schulte Hamburg, Germany

Dissertation zur Erlangung des Grades der Doktorin der Philosophie (Dr. phil.) an der Fakultät der Geisteswissenschaften der Universität Hamburg im Promotionsfach Philosophie, vorgelegt von Marie-Caroline Schulte, Hamburg, 2017.

ISBN 978-3-476-05702-0 ISBN 978-3-476-05703-7 (eBook) https://doi.org/10.1007/978-3-476-05703-7

© The Editor(s) (if applicable) and The Author(s) 2020. This book is an open access publication. **Open Access** This book is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specifc statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affliations.

This J.B. Metzler imprint is published by the registered company Springer-Verlag GmbH, DE part of Springer Nature.

The registered company address is: Heidelberger Platz 3, 14197 Berlin, Germany

# **Content**

# **List of figures and tables**

respiratory distress syndrome ...................................................... 102

# **1 Introduction**

"Despite one popular caricature of 'the philosopher' as being somehow 'deep', the ones I know (coming mostly from what is known as the 'analytical' school) make it a point of honour never to have anything profound or clever to say on any matter whatsoever. In fact, they consider it the hallmark of a good philosopher that one is always prepared to ask the sort of naive question that others are too scared to ask, for fear of appearing ignorant."1 (Michael Loughlin)

#### **1.1 EBM — a unifying force in medicine?**

Evidence-based medicine (EBM) was and is all the rage in medicine and the 'new way' to practice and teach medicine in our century. The term 'evidence-based medicine was coined in 1992 by scientists at the McMaster University, Hamilton, Ontario.2 Today it is *the* established method in medicine, at least in what is called 'allopathic' or 'conventional' medicine and the term is by now part of the very fabric of medicine and of medical knowledge. EBM encompasses research, practice and the teaching of medicine, and many articles in peer review journals, and even entire journals, have dealt with the how and why and where of EBM since its appearance on the medical scene. However, it is still not quite clear what exactly EBM is, what exactly it encompasses and why it should be so much superior than other movements in medicine. The definition of EBM offered in the original paper by the 'Evidence-based medicine working group' is "the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research."3 Sackett and colleagues have since refined the definition: "evidence-based medicine is a systematic approach to clinical problem solving which allows the integration of the best available research evidence with clinical

#### © The Author(s) 2020

M.-C. Schulte, *Evidence-Based Medicine – A Paradigm Ready To Be Challenged?*, https://doi.org/10.1007/978-3-476-05703-7\_1

1 Michael Loughlin. (2009). "The basis of medical knowledge: judgement, objectivity and the history of ideas." in Journal of Evaluation in Clinical Practice. 15(6): 935.

2 Gordon Guyatt, et.al. for the Evidence-Based Medicine Working Group. (1992). "Evidence Based Medicine: A New Approach to Teaching the Practice of Medicine." in the Journal of the American Medical Association (JAMA), 268 (17): 2420 - 2425.

3 David Sackett, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. (1996). "Evidence based medicine: what it is and what it isn't." in the British Medical Journal (BMJ), 312: 71-2.

expertise and patient values."4 However, and although both of these definitions are still frequently cited, EBM has changed over time. Probably inevitably so because of its own growth factor, but also because EBM and associated terms can be, and are used, to further the cause of pharmaceutical companies and interest groups alike without giving EBM more credence or credibility.5

For many reasons EBM has still 'won' the claim to be the 'best' form of medicine of today against some competition. However, when evaluating EBM historically, it is not quite clear against what type or form of competition EBM had to actually compete. It seems more reasonable to argue that *some* sort of change in medicine was simply inevitable. The technical and inventional progress in medicine over the decades preceding the advent of EBM, including the first randomised controlled trials (RCTs), clinical trials which compare treatments with each other in two groups of patients and which were already performed before the method became part of a medical standard, was so profound that a rethinking of medical research and medical practice was the only possible consequence. Therefore it can be argued that EBM appeared on the scene at the right time and with the right instruments to pick up the different pieces of medicine and to attempt to unite them into one coherent system.

EBM was and is embraced by many physicians, clinicians and medical researchers, especially after it transpired that EBM can eliminate faulty medical reasoning that was prevalent over a long time. There are multiple examples available for this phenomenon. Many treatments that were widely accepted in the medical community up until the 1990's were discontinued after EBM had put them to the test and considered them to do more harm than good.6 Prominent examples for this are the hormone replacement therapy (HRT) for menopausal women and the thalidomide scandal, causing birth defects, both of which will be discussed thoroughly in the coming chapters.

It is however important to clarify right from the beginning that there are treatments which will never be tested with the EBM methodology, since it would make no medical sense to do so. Treatments in this category are, for example, the removal of the appendix in the case of acute appendicitis, the Heimlich manoeuvre to unblock a blocked airway in a chocking patient or heart fibrillation to restart a

4 David Sackett, Strauss SE, Richardson WS,et al. (2000). *Evidence-based medicine: how to practice and teach EBM.* London: Churchill-Livingstone.

5 Trisha Greenhalgh, Jeremy Howick and Neal Maskrey for the Evidence Based Medicine Renaissance Group. (2014). "Evidence based medicine: a movement in crisis." in BMJ: 1-7.

6 Imogen Evans, Hazel Thornton, Ian Chalmers, and Paul Glasziou. (2011). *Testing Treatments: Better Research for Better Healthcare.* 2nd. Edition. London: Pinter and Martin: 3.

stopped heart.7 These treatments, and many more, are proven to be the only effective treatment for the underlying disease. Cause and effect are directly observable and well established. Often however, pure observation is not enough to successfully establish a causal connection. Researchers and clinicians have, over the course of medical history, dealt with spurious correlations which looked plausible in the beginning, but where not so in the end. A wonderful example from the Middle Ages about such a spurious correlation that actually worked to a certain degree and therefore was widely accepted, are the masks of the plague doctors.  The masks contained herbs because the assumption was that the plague was passed on by bad odours and therefore transmitted through the air. Patients suffering from the disease often stank badly because of their open and infected wounds.8 In actuality, the plague is passed on through droplet infection. Since the infection was still airborne but based on bodily fluids, the masks and overall get-up helped, because they prevented the plague doctors of inhaling the infected droplets. The reasoning was spurious, the preventive 'treatment' nonetheless useful. If that is the case, it would be arguable that as long as the ends are achieved, the means do not matter. Unfortunately there are a lot of examples in medicine where spurious correlations led to very harmful treatments. One example out of the more recent medical history is about hormone replacement therapy (HRT) for menopausal women. HRT was considered to take away the unpleasant side effects of menopause and to even prevent certain types of cancer. And it seemed to work well. However, after conducting randomised-controlled-trials (RCTs), trials that compared HRT to either other treatments or no treatment, it transpired that HRT did not prevent these types of cancer, nor did it prevent possible heart attacks. On the contrary, HRT has considerable side-effects, ranging from headaches to cancer that were for some time swept under the carpet.9 The reason why it was so "successful" was that it was most often prescribed in more affluent areas where many women were already in better shape, lived a healthy lifestyle and where overall more aware of their health and therefore better equipped to quickly deal with upcoming medical problems.10 EBM and its rigorous methods prevented more women from receiving HRT and therefore prevented the treatment from doing more harm.

As well as EBM seems to function for all these examples, there are still many aspects of it that are not entirely explained, defined or understood, even by staunch

7 Jeremy Howick.(2011). *The Philosophy of Evidence-Based Medicine.* Oxford: Wiley Blackwell, British Medical Journal (BMJ) Books. Introduction: xiii.

8 Jacob L. Kool and Robert A. Weinstein. (2005). "Risk of Person-to-Person Transmission of Pneumonic Plague." in Clinical Infectious Diseases, 40(8):1166–1172.

9 Imogen Evans, Hazel Thornton, Ian Chalmers, and Paul Glasziou. (2011): 16.

10 Ben Goldacre. (2008). *Bad Science*. London: Fourth Estate: 108.

supporters of EBM. As with all 'new' systems in an established field, EBM was and is not without its critics and their often very valuable criticisms and contributions.11 EBM in general is enthusiastic about criticism, because medicine, like all sciences, advances through criticism and through asking the right questions. As will become obvious throughout the dissertation, some of these criticisms are aimed at improving EBM but some are just phrased to discredit the program altogether. This dissertation will defend EBM, albeit all its shortcomings, as the currently best way to conduct medicine. And it will offer a way forward to produce, understand, and use evidence in medical research, and, even more importantly, in medical practice.

#### **1.2 EBM — one term used for different areas of medicine**

EBM is by now sort of an umbrella term for multiple areas in medicine. Related terms are also in use, and most often their aim is to either be more specific or to 'solve' an apparent problem of EBM. Examples of such terms are: EBP for evidence-based practice12 (I don't like using EBP because it also stands for evidencebased policy and even though both are context specific, there is a real danger of confusing them.) EBP can also refer to social work or education and therefore always has to be specified. EBHC which stands for evidence-based health care, EBN for evidence-based nursing and the number of abbreviations containing 'evidence' seem to increase daily. And the abbreviations seem to go in and out of fashion and therefore continue to be confusing. HTA stands for 'health technology assessment', and CER stands for 'comparative effectiveness research', these are two more terms that are interchangeably used with EBM but which do mean slightly different things and are themselves in need of clarification.13 A fairly new term that appears by now in conjunction with, or as a contrast to EBM, is PCHC standing for 'person centred health care'.14 PCHC is used in arguments both as an add-on to EBM and as a new way of looking at medicine. Therefore it is yet again not a term that can be used as such but is context-dependent as to its actual use.

11 Robert Smith. (2014). "Medical research—still a scandal." in the bmjopinion. http://blogs.bmj. com/bmj/2014/01/31/richard-smith-medical-research-still-a-scandal/. Last accessed on January 23rd, 2020.

12 K. Ann McKibbon. (1998). "Evidence-based practice" in Bulletin of the Medical Library Association 86(3):396-401.

13 Bryan R. Luce, et.al. (2010). "EBM, HTA and CER: Clearing the Confusion" in The Milbank Quaterly. Vol. 88.

14 European Society for Person Centred Health Care. www.pchealthcare.org.uk. Last accessed on January 23rd, 2020

Another example of a presumed 'alternative' is narrative-based medicine.15 It claims that the patient's narrative, his or her story, is needed to make an informed decision about a medical treatment. However, in every patient assessment the personal history of the patient is taken and used as part of the treatment process. But narratives can never replace evidence. They are necessarily subjective and have to be understood by the physician in the wider context of the patients diagnosis. One improvement that a focus on narratives can bring to the diagnostic side of EBM might be that more time is spend on the wishes and values of the patient, already making the patient feel more comfortable in the medical setting.

All these terms and their underlying approaches and assumptions however are still somewhat part of EBM, because none of them would work without recourse to a solid medical evidence base. In every proposed scenario, from 'personcentred health care' to 'narrative based medicine,' the clinician nonetheless needs a solid foundation of medical evidence to make a treatment decision, and EBM seems to still be the only solution providing this solid foundation. The clinician can still decide to not use the 'best' available evidential treatment either because it does not fit the individual patients needs or it is not available at the point in time. However, this decision is only possible when the treatment options, based on medical evidence, are known. Even if the option is to forgo treatment, the associated benefits and risks are based on solid evidential grounds and can therefore be assessed and calculated, at least to a certain degree. There will always be an element of surprise in medicine, like spontaneous healing or remission, but even this element of surprise will to a certain degree be part of the calculation and decision on treatment, which by its very nature is based on mathematical possibilities.

For clarity's sake and because many of these terms are referring to EBM, are using evidence as their underlying base or are simply too confusing to use, the term EBM will be used throughout the dissertation. I will do so, however, with the important caveat that the term 'EBM' itself is in need of clarification. The most important distinction that should be made explicit when using the term 'EBM' is between evidence-based medical *research* and evidence-based medical *practice*, since these two areas differ widely from each other today and hence should be discussed separately, especially where ethical and methodological questions are concerned. The division of the term 'EBM' into 'evidence-based medical research' and 'evidence-based medical practice' will also be one of the key arguments of the dissertation, since it is that division which will make medical practice more person-centred. And making EBM more person-centred again will save it from much of the criticism levelled against it.

15 Trisha Greenhalgh. (1999). "Narrative based medicine: Narrative based medicine in an evidence based world." in BMJ (318).

#### **1.3 The necessary division of EBM into 'evidence-based research' and 'evidence-based practice'**

Medical research starts at the molecular level and aims to make inferences from there to the population level and tries to answer questions regarding all patients with a certain disease, illness or handicap. Medical practice, in contrast, deals with the individual patient and has to ask which treatment is the right one for this special patient, with the patient's individual circumstances, preferences and values taken into consideration.16 The two areas of medicine, research and practice, are closely related and dependent on each other. But they are not the same and should be discussed in a separate manner and with a slightly different focus with regard to their ethical and methodological advantages and problems.

Although it would be tempting to use the abbreviations EBP, for 'evidencebased practice', and EBR, for 'evidence-based research', again it seems counterintuitive to the usual use of the term 'medicine'. 'Medicine' has always encompassed both, practice and research and for the longest time it seems as if the two were so fluidly going hand in hand that a clear distinction between them appeared to be none-sensical. However, today with EBM in full effect and with the incorporation of all the medical advances of the last decades, research and practice are, sort of by necessity, two separate entities in medicine, often with different personnel involved, or even employing entirely different companies to perform research. Pharmaceutical companies are conducting their own research into new drugs and have changed how the science of medicine works considerably by focusing entirely on research and the production of drugs. Physicians are still the ones dispensing the drugs and researchers in a clinical setting are coming up with new drugs and treatments, but nevertheless pharmaceutical companies are a driving force in the production and marketing of drugs.17

Since medical trials, the evidential basis for EBM, and the hallmark of evidence-based research, are fairly complex to perform and need a special set up, it can be a methodological advantage to remove them from the regular running of a clinic or hospital.18 Big university hospitals often can and do both, research and regular care, but they tend to be the exemption to the rule. And even if research and clinical care are happening in the same building, they are distinct from each

16 Julian Reiss and Ankeny, Rachel A., (2016). "Philosophy of Medicine" in The Stanford Encyclopedia of Philosophy (Summer 2016 Edition), Edward N. Zalta (ed.), https://plato.stanford. edu/archives/sum2016/entries/medicine. Last accessed on January 23rd, 2020.

17 Peter Konrad. (2005). "The Shifting Engines of Medicalization." in Journal of Health and Social Behavior. 46(1): 3-14.

18 Ben Goldacre. (2012). *Bad Pharma: How medicine is broken and how we can fix it.* London: Fourth Estate: 225. Ben Goldacre lobbies for large randomised trials which can happen in everyday medical care by integrating existing patient data. That would be a solution where care and research are combined.

other, because those patients who participate in a trial are not regular patients anymore, but change into participants who are treated differently. Participants agree to be part of a trial, knowing that the medication they receive is part of a trial and its safety not yet fully established. Some participants could be receiving a treatment that might not have any clear benefits and might even be harmful, or receiving a placebo that at least does not harm but also does not directly benefit the participant, above and beyond the placebo effect. Clinicians on their part change into researchers. And researchers are by their very nature interested in the test results on the population level, and not so much in the individual patient. They are blinded to the actual treatment of their participants so that all sorts of biases can be avoided.19

There cannot be a 'one-size-fits-all' approach in medical practice, nor really in clinical research, and hence the term EBM encompassing both can be misleading, but, as said above, it is the most useful term and it can easily be made context specific as to which part of the overall EBM methodology is actually meant. One major critique that is levelled at EBM is that it is too population centred and has lost sight of the individual patient. Later chapters will explicitly deal with this criticism in various ways and they will provide a solution that, for now in simple terms, comes back to the distinction between evidence-based research and evidence-based practice.

#### **1.4 EBM as a new paradigm in medicine — or is it?**

The authors of the original 1992 JAMA paper who have coined the term EBM made some interesting claims about the philosophical foundation of EBM. They called it a new paradigm in medicine and based this on Thomas Kuhn's definition of a scientific paradigm shift or revolution.20 According to Kuhn, a scientific revolution is a shift in scientific methodology and its accompanying worldview, i.e. the old paradigm, triggered by a revolutionary process that leads to a new methodology and worldview, i.e. the new paradigm. This process or shift starts if and when the old paradigm cannot successfully explain and incorporate new problems anymore because by doing so, it would inevitably collapse. In order to move on in science, a new paradigm has to appear that can successfully incorporate and explain these anomalies. This 'new' paradigm will work until it too will be unable

19 The actual set up of regular RCTs and their related problems and biases will be explained in detail in the next chapter. The reason why a short description here is necessary, is to set the stage for all discussions later in the text, since the division of EBM in research and practice will be the main topic and focal point of the entire dissertation.

20 Thomas S. Kuhn. (1996) *The Structure of Scientific Revolutions.* Third Edition. Chicago and London: The University of Chicago Press.

to explain upcoming anomalies. The old and the new paradigm are, according to Kuhn, "incommensurable" because, or so he claims, scientists working in different paradigms do not even speak the same scientific language anymore and therefore cannot communicate meaningfully with each other. A valid criticism to this definition of a paradigm in science is that if the language would really be that 'incommensurable' then Kuhn, and everyone else who tries, should not even be able to explain different paradigms in a coherent and comparative manner.21

Kuhn claims that "Since new paradigms are born from old ones, they ordinarily incorporate much of the vocabulary and apparatus, both conceptual and manipulative, that the traditional paradigm had previously employed. But they seldom employ these borrowed elements in quite the traditional way."22 That means for Kuhn that they do not share a common measure.23 The terms the scientists are using do not have to same meaning from one paradigm to the next. Kuhn centres his incommensurability theory on specific examples like the shift from the Newtonian to the Einsteinian understanding of space and time. Admittedly, the shift here was a major scientific one and left some physicists, who were still believing in the Newtonian world view, stranded. The progress in medicine however does not seem to be comparable in magnitude to the Newton-Einstein example, or the shift from the Aristotelian to the Copernican understanding of the solar system.24 Progress in medicine happened and still happens gradually and often slowly and not in seismic shifts which would make the old non-comparable to the new.

Trisha Greenhalgh, one of the proponents of EBM, claims, based on Kuhn, that the paradigm shift can also be triggered by a young generation of scientists who are not accustomed to the established paradigm and are starting to question its premises. Since these questions are not appreciated by the senior scientists, the young group branches out and establishes a new paradigm.25 However, even if a younger generation is branching out and establishing a new view on the medical

21 Scott R Sehon and Donald E Stanley. (2003). "A philosophical analysis of the evidence-based medicine debate" in BioMedCentral Health Services Research 3(14).

22 Thomas Kuhn. (1996): 149.

23 Alexander Bird. (2013). "Thomas Kuhn", The Stanford Encyclopedia of Philosophy. Ed. Edward N. Zalta, https://plato.stanford.edu/archives/fall2013/entries/thomas-kuhn/. Last accessed on January 23rd, 2020.

24 Kuhn uses the example of Einsteinian and Newtonian physics and although for the most part that shift in the sciences really did happen, the Newtonian system is still used for example to explain gravity. In the realm of the microcosmos, Newtonian physics still holds. (Brian Greene. (2003). *The Elegant Universe: Superstrings, Hidden Dimensions, and the Quest for the Ultimate Theory.*  New York: Vintage Books.)

25 Trisha Greenhalgh. (2012). "Why do we always end up here? Evidence-based medicine's conceptual cul-de-sacs and some off-road alternative routes." in Journal of Primary Health Care; 4:92-7.

problems, that is not a paradigm shift in the Kuhnian sense, since the overall medical language is still the same. The groups can communicate in a meaningful way, even if new terms are incorporated, like EBM itself, which has emerged as a new term. There is a lot of criticism levelled at Kuhn for his theory of scientific change. The problem with Kuhn's theory, especially for medicine is the 'incommensurability theory'. EBM in the early 90's brought something new to the medical world, a new approach to look at all the available evidence, to produce new evidence and to assess the available evidence. However, that 'something new' was not a radical change or an anomaly in an established practice that needed to be included in a new and different way. The 'new' approach was to acknowledge that not all of medical practice is and was actually based on the most robust evidence and that treatments are and were commonly used which could be outdated and might even be dangerous. The proponents of EBM also realised that medicine was not unified. Prescription and treatment habits did not only differ between countries, but very often between surgeries in the same country and sometimes even within a single surgery, if multiple clinicians were employed and were part of the decision making process.26 The prescription and treatment habits were often based on previously learned approaches which were not questioned over time. One goal of EBM was to make these differences disappear and, in consequence, to provide all patients with the same, and if possible excellent, level of care.

Therefore it is arguable if EBM is and was far less a paradigm in the Kuhnian sense, but was far more an inevitable measure at the time, and to a degree even today, to incorporate the rapid medical scientific progress between the 1950's and the 1990's into everyday medical practice and teaching of medicine. Medicine was shifting because of the progress and the accompanying changes, and a unifying movement, like EBM, made it possible to shift it in such a way that medical research as well as patient-centred care became equal parts of the fabric of medicine by asking the right methodological questions but without devaluing everything that medicine had achieved so far.

EBM still works by using the available information technology of our time, i.e. the internet and all the medical search engines, online journals and medical publishing corporations that are coming with it. The authors of the original paper had already predicted that the amount of available information would be growing exponentially in the future and hoped that there would be a workable solution for the problem. Today one of the biggest challenges of EBM and all other approaches

26 Gordon H. Guyatt. et.al. for the Evidence-Based Working Group. (2000). "Users' Guide to the Medical Literature: XXV. Evidence-Based Medicine: Principles for Applying the Users' Guides to Patient Care." in JAMA 284(10): 1290-1296.

in medicine is exactly the problem of too much information coupled with publication bias and industry funding which can distort the evidence base, since still not all information is available to everyone.27

EBM questioned the available treatments and medications at the time and demanded that many of them were put under rigorous testing for safety and efficacy. As seen in the first paragraph, not all treatments were put under test, because their causality to the underlying disease was well established. This alone could be enough to argue against any incommensurability based on language. The language of medical research and medical practice changes necessarily over time and that change is predictable and accepted. However, it changes fluidly and slowly, not in an instant, and practitioners can always converse meaningfully with each other.

The era of heroic medicine were treatments like blood-letting were used had already ended well before the advent of EBM.28 Therefore it is save to say that medicine before EBM was also based on research. But it was lacking in an overall, globally usable structure and because of the ever increasing methods of communicating research results globally, it needed a type of foundation on which it was possible to utilise the available information wherever it was needed. Most vaccines were already established for example and the language regarding immunology did not change abruptly from one day to the next, and neither did the authors of the JAMA paper seem to expect that. They used the term Kuhnian revolution or paradigm shift, it seems not really in a philosophical sense, but as a convenient catch phrase to demonstrate how big the shift that they were attempting actually was, and apparently how important it was to give the new movement a solid base, not only a scientific one but also a philosophical and an ideological one. Sehon and Stanley for example understand the term 'paradigm' as used by the original authors as a metaphor, however a poorly chosen one, which, as we have seen, does not work according to the Kuhnian definition of 'paradigm' and 'paradigm shift.'29

#### *1.4.1 The possible 'ideological' base of EBM:*

The scientific and the philosophical base of EBM will be the two main foci of the dissertation. However, it is interesting to at least briefly discuss the ideological

27 Ben Goldacre. (2012): 27. Goldacre and others show how difficult it can be to get hold of complete trial protocols and clinical study reports. Many companies claim confidentiality issues and withhold the information. As Goldacre and others rightfully claim, patient data could be anonymised, but so far the companies are not legally bound to disclose all available information.

28 P. Stavrakis. (1997). "Heroic medicine, bloodletting, and the sad fate of George Washington." in Maryland Medical Journal. 46(10): 539-540.

29 Scott R Sehon and Donald E Stanley. (2003).

base of EBM. The ideological base is the leaving behind, or at least the challenging, of authority. Because EBM is based on research facts, authority in the form of senior clinicians telling everyone else, including the patients, what to do and thereby acting paternalistically, will be undermined and vanish in due time, or so the authors predicted. The original JAMA paper claimed that a junior physician was in a better position to judge a patient, after using the methods of EBM rather than after asking and relying on his or her superior. The authors were aware that that approach could undermine hospital hierarchies and could threaten the value of experience. In fact, although the authors specifically state that this outcome is not intended by them,30 the undermining of hierarchies and the lack of use of clinical experience is one of the main criticism towards EBM. In a little side note, it is interesting to observer that, at least in Germany, patients are willing to pay extra to see the head physician and to attain special treatment, although it is acknowledged that the head physician of a department might be the one member spending the least amount of time on the ward.

Following the original thinking of EBM, the junior doctor would be precisely as well equipped to make the right diagnosis and be as successful as the senior one. What the authors undermine with this assumption is the fact that it is also a part of clinical experience to be able to interpret the results from a trial, from diagnostic tests or from a literary search about the symptoms or the diagnosis. They claim the opposite, but the idea that a junior physician can utilise the available evidence in the same way in which a senior physician would be able to do, already implies the unimportance of clinical expertise. However, clinical expertise and experience is needed to ask the right questions and to realise that a 'minor' symptom mentioned by the patient can be the actual indicator for a major disease. Even if the right diagnosis is found, clinical experience is needed to decide which treatment to choose in this particular instance. In order for the patient to be in the centre of the medical practice, clinical experience must be understood as vital and should be understood in todays view of EBM as being outside the evidence hierarchy altogether, informing and being part of the decision making process a) about which research to conduct, and b) about which treatment to administer to the patient.

Research results, notwithstanding their robustness, are only half of what is needed to successfully treat an ailing patient. Other forms of evidence in the form of expertise by the physician might be more important for this patient and it might even be the case that not treating a patient could be an option. Some patients, especially towards the end of their lives, prefer to only receive palliative care and one would not treat such a patient with antibiotics against for a example, an inflamed lung, whereas a young patient presenting with the same symptoms but otherwise healthy would definitely be treated with antibiotics. The basic research, i.e.

30 Gordon Guyatt, et.al. (1992): 2423.

how to successfully treat an inflamed lung, is the same for both cases. The individual patient's history and situation however decides the actual treatment.31

The authors are careful to point out that colleagues can learn from colleagues, and that the actual practice of medicine cannot be taught exclusively through books and articles but must also be taught through the greater expertise of these clinicians who have a lot of experience in their field. It seems as if the authors themselves would have been slightly uneasy in devaluing expert opinion of clinicians completely.

One main theme of EBM is the literary search, i.e. finding and assessing the important medical literature for a given medical case. At McMaster Hospital, where most of the authors of the original JAMA paper worked, each floor had a search computer with a simple Medline32 search tool. All through the following chapters, it will become obvious that the simple search approach that was heralded in 1992 is just not feasible anymore in 2017, and for multiple reasons. The most compelling of these would be the sheer volume of literature in any given medical field and the number of search engines that provide access to medical articles and peer reviewed journals. The same article can be accessible via multiple links, some of them with open access, some only accessible through research institutions.

EBM today is often accused of being "Cookbook medicine"33, however, in the original paper, the proponents of EBM claim the exact opposite, saying outright that cookbook medicine has its appeal, because it is quick and easy and EBM is not, but that EBM in the long run will provide by far the better base for medicine because it is more rigorous and better designed to help the actual patient.34

#### **1.5 Conclusion**

The advent of evidence-based medicine should not be understood as a paradigm shift in the Kuhnian sense. The different fields and areas of medicine still have a meaningful way to communicate with each other and research results and medications that were used before EBM were not simply abandoned but scrutinised and either tested, kept or abandoned, according to their safety, efficacy, and effectiveness. But even if EBM is understood as 'just' inevitable change because of the advances in science and medicine itself, some of the criticisms levelled at EBM

31 Gordon H. Guyatt. et.al. for the Evidence-Based Working Group. (2000): 1291.

32 MEDLINE/PubMed Resources Guide. www.nlm.nih.gov. Last accessed on January 24th, 2020. Medline has merged since the 90s and naturally has expanded and is one of many search engines for medical articles.

33 RB. Darlenski, et. al. (2010) "Evidence-based medicine: Facts and controversies." in Clinics in Dermatology, Elsevier:554.

34 Gordon Guyatt, et.al. (1992): 2423.

are still very valid and need to be addressed. Not only for EBM to progress further, but also to make it more patient-friendly and to produce valuable research which is important to future patients and not just profitable.

EBM can be used as a unifying force in medicine to bring different areas of medicine closer together by providing a solid base for the use of medicine and for shared decision making between physician and patient in order for both sides to profit from it. The solid base needs to be the available evidence which informs not only the patients and physicians but also shapes new research since evidence can never be complete.

Therefore, instead of proposing all sorts of alternatives to EBM, it should be clear that the solution to many of EBMs problems have to come from within EBM. And one such solution is to divide EBM into 'evidence-based practice' and 'evidence-based research.' In the former, the patient with his or her values, concepts, wishes and goals should be the centre of all considerations. The physician has the task to individualise all the available evidence and to use it in ways appropriate to every particular patient. In 'evidence-based research' patients and physicians have to be aware that their roles are changing and that they both are participating in the generating of new evidence. Individual needs and wishes do not play the active part anymore that they play in evidence-based practice.

Research evidence needs to be robust, quantifiable and reproducible. EBM uses a hierarchy to rank evidence according to these criteria, but as will become apparent, this hierarchy is only as useful as are the results of its individual ranks and the highest ranking evidence might not be the best evidence for the individual patient. However, it is necessary to take a close look at how the most high ranking evidence in evidence-based research is achieved.

This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. **Open Access**

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder..

# **2 The methodology of evidence-based research.**

#### **2.1 RCTs and their importance for the methodology of EBM**

EBM is heavily based on quantifiable research and its results with the goal to achieve qualitative health care on the population level and on the level of the individual patient. The step from the population level to the individual in health care is a fairly complicated one and will be analysed in the following chapter, because it is also the most important one to free EBM from the criticism that it is not person-centred enough. In order to do so, it is necessary to look at how quantitative results are achieved and how the actual methodology of generating 'good' and 'usable' evidence functions. The methodology of evidence-based research is based on a hierarchical view of the quality of evidence. The actual evidence-hierarchy will be explained in detail in the chapter about evidence and epistemology. This chapter will focus on randomised controlled trials (RCTs) as those medical firstin-man trials which play a major role in the generating of evidence. RCT's are medical experiments that are designed to test safety, risk, efficacy and effectiveness, normally in this order, of novel treatments like drugs, surgery, and even acupuncture and physiotherapy.

The results of RCTs inform most medical and health policy decisions, since they are often considered to be the best option to arrive at robust and usable evidence. The following chapter will closely look at RCTs and will assess if they really are as fail-safe in producing 'good' evidence as they are made out to be.

On the very top of the overall hierarchy of evidence, including steps to *assess* evidence, are systematic reviews, or meta-analyses, because they pool and assess the available data, generated from RCT's and often from well-conducted cohort and observational studies, and make it comparable. The best-known institution for systematic reviews and meta-analyses is the Cochrane Collaboration and the Cochrane Library.35 Archie Cochrane could be called one of the founding fathers of EBM and epidemiology. He argued that it was vital, especially with sparse resources, that only those treatments are used which have shown beyond a doubt to be effective. He greatly favoured RCTs as a means to arrive at this goal. 36 Cochrane reviews are instrumental in making evidence-based practice manageable

35 Cochrane Collaboration. http://www.cochrane.org. Last accessed on January 23rd, 2020.

36 Cochrane, trusted evidence, informed decisions, better health. http://www.cochrane.org/aboutus/our-name. Last accessed on January 23rd, 2020.

for the evidence user. The literature about medical evidence is so vast that it is vital to have succinct reviews which pool the most robust evidence and are able to give short and precise statements about the pros and cons of the treatments in question. Meta-analyses and systematic reviews are not without faults however, since they can only be as good as the initial data provided. The Cochrane Collaboration, and like-minded organisations, strive diligently to regularly assess new data about topics of already conducted reviews in order to keep the evidence as up to date, and as manageable, as possible. Lesser known organisations are the Joanna Briggs Institute based in Australia, the "Centre for review and dissemination" and the EPPI Centre.37 Individual scientists and authors can also conduct systematic reviews and have them published. Yet again, even the amount of meta-analyses and systematic reviews about each single diagnosis is staggering and not manageable in its entirety for any physician or clinician. However, because of their special training, most clinicians can assess the relevance of the research pretty quickly and focus on the most important parts for their individual patients. Increasingly, these skills are already taught in medical schools.38

In this chapter the focus lies on the methodological and ethical problems of RCT's, since they are still viewed as the most important tool in the arsenal of conducting meaningful medical research and are therefore the cornerstones of EBM. For the longest time it was argued that the results from RCTs were the only admissible evidence in EBM.39 In the last decade however, with growing criticism towards EBM and its methods, it has been acknowledged that other forms of medical evidence can play an equally valuable role, especially when and where the individual patient is concerned.40 41 But not all evidence is automatically 'good' evidence and even if it is considered to be 'good' the question remains: for whom?

#### *2.1.1 Scientific goals of RCTs*

The overall scientific goal of RCTs is to establish the absolute risk reduction, ARR, or inversely, the number-needed-to-treat, NNT, of patients with the novel treatment compared to standard or placebo. These numbers stand for the efficacy of the novel treatment.

37 Library Guides: Subjects, Services and Resources. http://libguides.rgu.ac.uk/c.php?g=536793& p=4389919. Last accessed on January 23rd, 2020.

38 Jorgen Nordenstrom. (2007). Evidence-Based Medicine in Sherlock Holmes' Footsteps. Oxford: Blackwell Publishing.

39 Gordon Guyatt, et.al. (1992): 2420.

40 Jo Rycroft-Malone, Kate Seers, Angie Titchen et.al. (2004). "What counts as evidence in evidence-based practice?" in Journal of Advanced Nursing. 47(1): 81-90.

41 Trisha Greenhalgh, et.al. for the evidence-based renaissance movement. (2014).

I am using the example for NNT and ARR of the Centre for Evidence-Based Medicine Oxford here due to its conciseness: "The absolute risk reduction or absolute effect is the amount by which your therapy reduces the risk of the bad outcome. For example, if your drug reduces the risk of a bad outcome from 50 per cent to 30 per cent, the ARR is:

ARR = CER – EER = 0.5 – 0.3 = 0.2 (20 per cent)."42

CER stands for Control Event Rate and EER for Experimental Event Rate.

The NNT is the inverse to the ARR and is always rounded up to the nearest number since whole patients are treated, not fractures.

NNT=1/ARR

For the example above that means that:

NNT = 1/ARR = 1/0.2 = 5

The NNT stands for the number of patients that need to be treated in order to improve one patient. Therefore the ideal NNT is 1, as in one patient. The higher the NNT is, the less patients benefit from the novel treatment. However, even a treatment with a high NNT can be useful in particular circumstances, especially when no other treatment for this particular patient is available. It is very context dependent if a treatment with a high NNT can still be deemed acceptable. The experience of the clinician can guide the judgement call if a treatment with a high NNT might still be useful in a particular case. In trials about disease prevention, the NNT is often allowed to be higher than in trials testing singular treatments or drugs.43

Additionally to the efficacy of the novel treatments, RCTs are also, and even more importantly, conducted to assess the safety of a novel treatment. This is the special focus of early phase RCTs. If the safety cannot be established, then no further RCTs will be performed. Efficacy does not trump safety!'Safety' however should not be confused with 'overall risk' of a treatment. The risk/benefit factor needs to be established in a trial. If the actual benefit of a treatment outweighs its risk/s, then further RCTs should be conducted to assess how far they actually do so and if it is still reasonable to use a treatment even though it contains a certain amount of risk.

42 CEBM: Centre for Evidence-Based Medicine, Oxford. "Definition of ARR and NNT." http:// www.cebm.net/2014/03/number-needed-to-treat-nnt/. Last accessed on January 23rd, 2020.

43 Katja Suter, Matthias Briel and Judith Günther. (2015). "Number needed to treat (NNT) and Number needed to harm (NNH): weitere Abkömmlinge der Vier-Felder-Tafel." in Medizinische Monatszeitschrift für Pharmazeuten 38(3): 103-106.

RCTs come in multiple forms and with various research protocols attached to them, always depending on what it is that is to be tested. Trials testing non-drug treatments are slightly different to those testing drugs, and those testing preventing measures like vaccines are yet again slightly different. However, there are some aspects that they all have in common and these will be the focus of the discussion. RCTs are so-called 'first-in-man trials and are conducted in different phases and with a different amount of participants necessary, exactly to assess overall safety first and subsequently, efficacy, risk and effectiveness.

RCT's are set up in a way in order to arrive at "ideal" results, i.e. results that are reliable and robust. Since it is good scientific practice never to rely on the results of single trials, most treatments are tested in multiple clinics, in multiple RCTs, sometimes with differing, sometimes with the same research protocols. The results of one RCT need to be reproducible, and preferably more than once.44 Reproducibility of results leads to scientific acceptance, and to the results to be possibly externally valid and therefore of relevance to the actual target population. However, a multitude of trials and trial data, often running over hundreds of pages, is not usable as such for the actual evidence user.45 That is why systematic reviews and meta-analyses are so important, because they assess the available data, dismiss badly conducted trials and make the results easier to 'digest.' Therefore, they are a vital part of EBM, because they make evidence-based practice possible in the first place.

If RCTs are set up correctly, they are considered to be internally valid, which means that they yield correct and rigorous results about the treatment for the actual trial population under test and have used the correct methods to do so. They are therefore comparable to an experiment in a laboratory setting, insofar as that they adhere to the same form of rigorousness. However, that internal validity does not yet make RCTs also viable for the actual target population needing that specific treatment for a that specific ailment, let alone for the individual patient. On the contrary, a very high internal validity can lead to a quite low external validity.46 If and how this specific problem can be solved will be discussed later.

The quality of the results of RCT's is based on the validity of the trials and the validity of the RCT's is based, more often than not, on successful randomisation and on successful blinding. Randomisation and blinding are the two main methodological features of RCTs that set them apart from trials like cohort studies and observational studies, both of which will be evaluated as part of the hierarchy

44 J. Shao and SC. Chow. (2002). "Reproducibility probability in clinical trials." in Statistics in Medicine. 21(12):1727-42.

45 Jorgen Nordenstrom. (2007):

46 Nancy Cartwright. (2007). "Are RCTs the Gold Standard?" in BioSocieties. 2 (1) (Special Issue: The Construction and Governance of Randomised Controlled Trials). 11-20.

discussion, and they are the two features which need to be maintained throughout the trial to really render robust results.

#### *2.1.2 Publication of all trial data*

In order to successfully assess RCTs and to conduct meta-analyses, the results of all RCTs need to be published in a coherent and complete form, which unfortunately they are not, at least in many cases.47 To establish reporting standards, the CONSORT Group, a consolidated group of two research groups, both aiming at a standardisation of trial reporting, merged in 1996 and published the first CON-SORT Statement in 2007 and a revised form in 2010.48 The CONSORT statement and tool consists of a flow chart and checklist for authors of RCTs to control if all necessary information is written up and all data is included. Many academic journals prefer that the trial write-up is done according to CONSORT or even stipulate the necessity for authors to adhere to CONSORT before publishing any research results. CONSORT however does not judge the methodology of RCTs. It only aims at good reporting standards to prevent reporting bias. The value of the methodology of RCTs however lies in their set-up and in the quality of said set-up which starts with randomisation.

#### *2.1.3 Introduction of the basic methods of RCTs*

As the name already indicates, the most prominent and important feature of RCTs is 'randomisation.' Randomisation means that trial participants are divided into two different, but equal, groups. There are different methods of randomisation available and these will be discussed throughout the chapter. In its most basic, but also most usual form one group receives the treatment under trial and the other group receives either the standard treatment or a placebo. Placebos can be either inactive or active and both pose their own problems, ethical and methodological, as do placebo-controlled trials as a whole. These 'special' trials will be discussed at length later on.

The main question here is if randomisation really has the methodological virtue to make the groups equal and what role confounding factors play for answering that question. Blinding is the second most important step in the set-up of RCTs and means that at least two or more groups involved in the trial do not know in

47 Ben Goldacre. (2012): 81.

48 The Consort Statement. http://www.consort-statement.org/ Last accessed on January 23rd, 2020.

which groups the participants are randomised into.49 Blinding is supposed to control for confounders such as selection - and allocation bias, i.e. the researcher consciously or unconsciously putting participants into either the control or the treatment arm based on where they will do better. The other case which is to be avoided is that the participant knows which treatment he will receive in order to minimise expectations toward the treatment. The difficult part in any blinded trial is to maintain the blinding. At least one party can be blinded but the goal is to blind all parties involved in a trial. In trials were treatments such as pills or injections are under test, blinding of a large group of people involved is possible, even to the extent that the outcome assessors, those who receive the raw data and evaluate it, are blinded. In trials concerning surgical procedures it is much more complicated to maintain the blinding, since the surgeon at least needs to know if he is supposed to perform a real or a sham surgery. The same is true for all trials testing treatments where at least the dispensing physician or health practitioner needs to know which treatment he or she is administering. Acupuncture would be such an example where it is impossible to blind the acupuncturist.50

#### **2.2 The different phases of RCTs**

Before a drug or treatment is tested on human beings, it is developed and rigorously tested in the laboratory. Often, in the later stages of the development, these tests and trials involve laboratory animals, such as rats or monkeys for example. The necessary requirement is most often that the DNA has a special similarity to that of humans, or that some other feature is close enough to humans to make the results of animal studies usable in later human studies. Animal studies however suffer the same problems that RCTs in humans do. They can be flawed through observation or publication bias and through the lack of external validity. It is acknowledged that they are and were necessary to further the understanding of the mechanisms of disease but are less well equipped to reliably inform about the effectiveness and safety of treatments.51

In these animal trials, randomisation and blinding is not absolutely necessary and often hard to perform because the sample sizes can be too small. It is however important to mention that in theory, and even sometimes in practice, RCTs can be performed in animal studies where the groups are sufficiently big enough and can

49 Jeremy Howick. (2011): 63.

50 Edzard Ernst and Simon Singh. (2008). *Trick or Treatment? The undeniable facts about alternative medicine. London: Transworld Books:* 67.

51 H. Bart van der Worp, David Howells, et.al. (2010). "Can Animal Models of Disease Reliably Inform Human Studies?" in PLoS Med 7(3): 1.

be meaningfully randomised and the observer could even be blinded as to the intervention, just recording the observations without attaching any results.52 Since 2010 the ARRIVE guidelines, as an equivalent to the CONSORT guidelines, have been adopted to make animal research and the publication of animal research more transparent and to make it liable to ethical considerations.53

However, when there is talk of RCTs in the medical setting, it is necessary to assert that it is almost always based on trials involving humans. So when we talk about RCTs here, it is about phase 0 to IV trials, those following the laboratory stage of the research, involving human beings. The most common RCTs are performed in phases I to III, phases 0 and IV are rarely performed. However, especially phase IV trials, those which are conducted after the market approval of the treatment, are of vital importance for the safety assessment of a drug, because they can be conducted over a lengthy period of time, in many patients of the actual target population, and side effects can be more easily detected.54

#### *2.2.1 Phases of RCTs according to Benedetti:*55

Phase 0 trial: These trials are fairly rare, mostly conducted in cancer research and are only involving a very small number of participants who are usually suffering from the disease. The aim of phase 0 trials is to establish safety and the potential of the drug to reach the target area depending on the dosage. Therefore it is essential that the participants are suffering from the disease in question. The novel treatment is rarely tested against a placebo.

Phase I trials: These are for most drugs and treatments the actual first-in-man trials. They are conducted, usually, on a very small number of healthy participants. The main questions are 'safety of dosage', possible side effects, and the bodies reaction to the drug. The novel treatment is rarely tested against placebo.

Phase II trials: These trials try to answer the same questions as in a phase I trial, but are conducted with a larger number of participants, often patients with the actual disease or illness in question. Since a base-safety of the drug is established after successful phase I trials, efficacy of the drug plays a bigger role than safety.

52 Beverly Muhlhausler and Frank Bloomfield, et.al. (2013). "Whole Animal Experiments Should Be More Like Human Randomized Controlled Trials." in PLoS Biology, 11(2).: 1.

53 Beverly Muhlhausler and Frank Bloomfield, et.al. (2013).

54 Fabrizio Benedetti. (2014). *Placebo Effects.* 2nd Ed. Oxford: University of Oxford Press, Kindle Version: Chapter 1.2.1 Placebos are the tenet of the randomised, double-blind, placebo-controlled trial design.

55 Fabrizio Benedetti. (2014): Chapter 1.2.1 Placebos are the tenet of the randomised, double-blind, placebo-controlled trial design.

Phase III trials: These are the last trials before a possible licensing and market approval of the drug. They are preferably conducted with a huge number of participants and the novel treatment should be tested against the standard treatment, and only against placebo if no standard is available. Safety, efficacy and effectiveness of the novel treatment are again the main questions for the researchers. Side effects are observed over some length of time.

Phase IV trials: These trials are conducted after the licensing and market approval of the novel treatment. They are most often observational studies which gather information of the long-term risk and safety of the drug as it hits its actual target population and therefore play a part in establishing external validity. Unfortunately phase IV trials are fairly rare, even though they could be conducted, even as RCTs, quite easily, if GPs and clinicians were allowed to gather the appropriate data in their daily practice.56

The trials as they are described above are idealised versions of RCTs. The number of participants per trial, especially if it is a small number, contributes in a big way to the problems of RCTs. Phase II and III trials require, to be performed 'correctly,' a fairly large number of participants. The more participants the better, since the stratification of possible confounders, factors that influence the trial results, is better guaranteed and even rare side-effects are picked up more easily, *if* they are picked up at all in the given time. The obvious problem is the recruitment of this larger number of participants for many different reasons which will be discussed throughout the chapter. Sometimes the disease is simply so rare that there are just not as many patients available as a trial would actually need. In these cases provisions are done for smaller trials and the smaller number of participants is not interpreted as a flaw of the trial.57 In most cases however there are just not enough volunteers. Either because patients are not made aware that a trial is conducted for which they would be eligible or they are not 'ideal' enough, which means they are too ill or have too many co-morbidities. Or they simply do not want to participate in a trial, fearing that they receive the lesser treatment or lesser care.58 However, in many trials the care is even better in the research setting than it would be in the regular setting, since the participants are under close supervision and are controlled much more regularly than a patient in a normal GP practice would be. That is especially true, and alarmingly so, in trials that are conducted in the developing

56 Ben Goldacre. (2012): 225-241.

57 Niklas Juth (2014). "For the Sake of Justice: Should We Prioritize Rare Diseases?" in Health Care Annals.

58 Yvonne Brandberg, Hemming Johansson, Mia Bergenmar. (2016). "Patients' knowledge and perceived understanding —Associations with consenting to participate in cancer clinical trials." in Contemporary Clinical Trials Communications 2: 6-11.

world. Trials are less expensive to conduct there and participants are presumably easier to find, because the level of overall care is so bad that research settings provide the one possibility for many patients to even have some form of treatment.59

#### **2.3 Confounding factors and their influence on trial results**

Confounding factors are those factors, like age, gender, overall health, weight, pregnancy, existing illnesses, chronic diseases, etc., but also many forms of bias, that can severely change the outcome of a trail. The attempt to control for confounding factors is what makes randomisation and blinding so important for medical trials. Confounding factors are divided into known and unknown confounders and both types can differ significantly from trial to trial, depending on the treatment or drug under test.60 Therefore, there is no exhaustive list of possible confounders and every investigator has to conclude from previous research which confounders are relevant for the trial and subsequently need to be controlled for. This before-hand control however can only work for known confounders. Unknown confounders make control mechanisms, such as randomisation and blinding even more necessary.

Howick explains confounding factors by labelling them with three properties. First: "the factor potentially affects the outcome." Second. "the factor is unequally distributed between experimental and control group" and third: "the factor is unrelated to the experimental intervention."61 The most important feature of confounding factors is that "each confounding factor provides a potential alternative explanation for the results of a clinical trial."62 This chapter will therefore aim to explain why the control for possible confounders is on the one hand important for the overall validity of the test results and therefore desirable and on the other hand not always possible and not always necessary, even if the above explanation of confounding factors is correct.

Sometimes, mostly in phase IV trials, does it become apparent that confounding factors were responsible for the overall outcome of previous trials. In some cases the deviation between the results of a drug or treatment before market approval and afterwards is so big that the drug will be, and needs to be, taken off the

59 Sonia Shah. (2008). *Am Menschen Getestet!  München: Redline Wirtschaft.

60 Jeremy Howick. (2011): 34.

61 Jeremy Howick. (2011): 77. Howick modifies this third property with regard to side effects. In the moment I will let it stand as is and will come to the point about side effects later. "the confounding factor must be unrelated to the positive characteristic effects of the experimental treatment on the target disorder (as opposed to side effects)."

62 Jeremy Howick. (2011): 36.

market. A well-known example of such a case, albeit without a phase IV trial was running, is the thalidomide scandal. Thalidomide was a drug manufactured in the 1950s and 1960s in Germany and given to many expecting mothers to treat nausea and fatigue.63 After the drug was on the market for quite some time it was discovered that it caused malformation of the limbs, called phocomelia, in newborns. Thalidomide was subsequently taken off the market, but only after a huge scandal ensued in Europe, the United States and Canada.64 It is now in use again for certain types of cancer and for rare complications in leprosy, but pregnant women are specifically warned not to use the drug.65 The Thalidomide scandal has influenced, and changed, how drugs and treatments are tested today and how the marketing approval of new drugs is governed to prevent similar 'mistakes' in the future.

Medical scandals show how necessary it is to test the safety of a new treatment or drug, long before it reaches the open market, and to stay diligent even after it is approved. And in order to securely arrive at a conclusion about safety, all the other factors that could influence the outcome must be eliminated in the best possible way. That makes it so important to control for all confounders, known and unknown, as good as possible. As will become obvious, randomisation and blinding are far from perfect to always and reliably control for unknown confounders and many scientists seem to agree about their imperfectness without presenting valid alternatives. Therefore it is important to study the shortcomings of these control mechanisms and to try and improve them, instead of vilifying RCT's in general, as some seem to do.66

So in order for RCT's to be internally valid and to arrive at "ideal" results, they have to control for confounders and use randomisation and blinding to do so.67 The two methods go hand in hand with each other but need to be examined somewhat separately because they each have their inherent problems and strength.

63 Imogen Evans, Hazel Thornton et.al. (2011): 4.

64 Jack Botting. (2002). "The History of Thalidomide." in Drug News & Perspectives. 15(9): 604.

65 Teru Hideshima, Dharminder Chauhan et. al. (2000). "Thalidomide and its analogs overcome drug resistance of human multiple myeloma cells to conventional therapy." in Blood, Vol. 96:2943-2950.

66 John Worrall. (2010). "Do we need some large, simple randomized trials in medicine?" in M. Suarez, M. Dorato and M. Redei (eds). EPSA Philosophical Issues in the Sciences. Dordrecht: Springer.

67 Jeremy Howick and some other philosophers of science would prefer to call "blinding" masking, because blinding is a derogatory term towards visually impaired patients. Since most articles and books about RCT's still use the term "blinding", that is what I will use, without any derogatory intent.

#### *2.3.1 Randomisation*

Randomisation will be explained first, since it is understood as playing the most important role in the control for possible confounders.68 Randomisation in its most simple form means that patients who are willing to participate in a trial are selected into two groups, the treatment group that receives the novel drug or treatment and the control group that receives the standard treatment or a placebo. The selection process is randomised, most often according to a numerical pre-specified code that is not known to the researcher or the participant. Therefore, neither know in which treatment arm the participant is randomised into. Eligible participants are most often randomised via an independent agency which has produced the randomisation codes. The researcher receives a number for the patient that is matched either for control or for treatment. The random numbers are generated via a computer program and coded before they are given out. Although it sounds very complicated, the method is very close to throwing a dice or flipping a coin.69 The main reason that computer generated numbers are used is that there is less of a possibility to manipulate the process. The method of computer based randomisation can be compared to a lottery. Most often in practice, after the computer-generated lottery has run, envelopes are prepared which "contain randomly generated instructions about which group to assign the next patient."70 Here blinding and randomisation go hand in hand. The researcher receives the envelope but since he does not have sufficient information to decipher the code, he is blinded to the intervention that the participant is randomised into.

There are a number of softwares available on the internet which are fairly easy to use and can be used for multiple types of randomisation. One of those often used by medical statisticians is http://www.graphpad.com/quickcalcs/index.cfm. Graphpad even goes beyond that and offers different statistical methods to arrive at and calculate results.

In EBM both single and cluster randomisation can be used.71 In evidencebased policy (EBP), cluster randomisation is most often the method of choice, since policies are never implemented on an individual level. Cluster randomisation means that entire groups of people are randomised as groups, not divided into individuals. Single randomisation simply means randomisation at the individual patient level, with the methods described above. In EBM cluster randomisation could

68 John Worrall. (2004). "Why there's no cause to randomize." Technical Report in Causality: Metaphysics and Methods. CPNSS: 3.

69 K.P. Suresh. (2011). "An overview of randomization techniques: An unbiased assessment of outcome in clinical research." in Journal for Human Reproductive Sciences. 4(1): 9.

70 Jeremy Howick. (2011): 43.

71 BetterEvaluation: Sharing information to improve evaluation. http://betterevaluation.org. Last accessed on January 23rd, 2020.

for example mean that the participating clinics would be randomised, and the patients would be treated with either the novel or the standard treatment, depending in which clinic they would be treated at. Another form of cluster randomisation is also called block randomisation and is sometimes used when it is important to have equal sample-sizes. Block-randomisation however is not very good in eliminating confounding factors.72 Cluster randomisation can achieve that, if enough patients in each clinic participate in the trial.

To eliminate known confounders before the trial, especially when the number of participants is high, stratified randomisation can be used. It starts with a type of block randomisation in which participants are assigned to different blocks depending on the confounder. "After all subjects have been identified and assigned into blocks, simple randomization is performed within each block to assign subjects to one of the groups."73 This method however only works when all participants are selected before the trial. If they are selected on a continuous basis while the trial is running, then regular single randomisation should be used.74 Cases in which stratification on the individual level is asked for are normally involving certain aspects of a specific disease. An example here can be any cancer that can appear with or without metastases. A novel treatment can yield positive results in both cases, but the participants should be randomised according to the presence or absence of metastases, so that the control and the treatment arm contain a fair number of both types of cancer patients. Therefore both 'types' of cancer are used as strata and the participants within the strata are then randomised into control and treatment arms.75 However, an approach like this is obviously only workable with very few strata, otherwise the subgroups would be too small. Machin claims that "For continuous prognostic variables such as age, stratification can only be carried out when these variables are divided into categories. [blocks] … Although age (or some other continuous variable) may be prognostic for outcome, it is usually preferable not to stratify for this but to record the information for each patient and take account of this in a retrospective sense at the analysis stage."76 However 'controlling' for confounders during the analysis stage, when all the data is gathered, can be very complicated and can lead to false results. Therefore it would be better to use some pre-stratification and thorough randomisation in order to control for known confounders right from the beginning of the trial.

72 K.P. Suresh. (2011): 10.

73 K.P. Suresh. (2011): 10.

74 BetterEvaluation Pg. 2.

75 David Machin and Peter M. Fayers. (2010). *Randomized Controlled Trials, Design, Practice and Reporting*. Oxford: Wiley-Blackwell: 101.

76 David Machin and Peter M. Fayers. (2010): 102.

In systematic reviews, all types of randomisation can be compared, but it should be specified which type of randomisation was used in order to better evaluate the outcome. An example where specification was necessary is a meta-analysis of trials in low- and high-income countries about antenatal care for pregnant women. The women in the high-income countries were randomised individually and the women in the low-income countries were randomised in clusters. The trials were set-up to evaluate how much care was necessary for women with a low-risk pregnancy and if the number of antenatal visits could be decreased.77 One main reason between the differences in randomisation in this example is the difference between high- and low-income countries. In high-income countries, 'contamination' is less likely. 'Contamination' can occur when participants communicate with each other and find out in which arm they belong and what they potentially miss out on. Contamination can lead to participants leaving trials and seeking for a higher level of care elsewhere. Even though pregnant women speak with each other and might compare their levels of care in high-income countries, the overall level of care is high and being in the test arm of a trial, especially in this case, does not pose a greater risk to either mother or child, as was shown in the sub-group

analyses of high-income countries in which no more perinatal death occurred than in standard care.78 In low-income countries the risk of contamination was deemed higher. Cluster randomisation of the clinics was used to prevent this, since all patients in one clinic are either treated with the novel or the standard treatment. However, possible hidden 'individual' confounders are not controlled for in cluster randomisation. Therefore, it is impossible to directly compare trials with individual and with cluster randomisation in most cases, simply because possible confounders cannot be equally controlled for. One obvious confounder in the example is already the difference in income and overall care given. Perinatal death were higher in the test-arm in low income countries, because conditions which would have led to the admission into neonatal intensive care were not recognised at all or too late. One outcome of the trials and the subsequent meta-analysis was that all women were less satisfied with fewer prenatal visits, regardless of low- or highincome countries.

Even single randomisation can be problematic, if it is not done correctly, since the different methods of single randomisation, if they are not done as described above, can be easily manipulated. One very questionable method is the Zelen method of randomisation, because participants are only asked for their consent to participate in a trial after the randomisation has already happened.79 The

77 T. Dowswell and G. Carroli. (2010). "Alternative versus standard packages of antenatal care for low-risk pregnancy." in Cochrane Database of Systematic Reviews, Issue 10. Art.

78 T. Dowswell and G. Carroli. (2010).

79 Marvin Zelen. (1979). "A new design for randomized clinical trials." In New England Journal of Medicine 300: 1242-1245.

method can lead to obvious selection bias and to early drop-out rates if and when the participants are questioning the trial and opt out to use the standard treatment. A choice denied to those patients in the control arm of the trial, because there were not informed at all about their trial participation. When Zelen wrote his randomisation proposal, such an approach would still have been possible. Today however, participants have to consent to the use of their data. Therefore they need to be informed about all the possible choices and the Zelen method is ethically and methodologically problematic. The latter is the case because neither the participant nor the researcher are blinded as to the intervention which can lead to selection and observation bias. The former because participants in the control arm are left in the dark as to the treatment possibilities.

"Randomised plays the winner" is another form that was and still is used to randomise participants on an ongoing basis.80 It functions in a way that if one treatment is more effective than the other, then more participants will receive the "effective" treatment and this can result in false-negative or false-positive overall results of the trial, because the patients are not divided equally anymore between the arms and confounders and other deviations in the participant population can severely change the results.

Howick points out the problem of "pseudo-randomisation" which occurs when a randomisation procedure is too easy to decipher. An example would be if every other patient, who would be eligible for a trial, would be put in the control arm. If this type of randomisation would be used, blinding of the different parties could be easily subverted.81 Howick makes the argument that knowledge about the allocation does not need to undermine the validity of the trial, if the randomisation is not tampered with.82 Further down the argument, he however admits that "allocation bias and self-selection bias can become worrisome again."83 Howick is convinced that proper blinding is simply not possible and should therefore not have as much methodological value as Worrall and others are allowing it. However, blinding seems to be the only method to successfully rule out self-selection and allocation bias. I will come back to this problem later in the chapter.

80 Elbourne, D. Field, D. Mugford, M. (2002). "Extracorporeal membrane oxygenation for severe respiratory failure in newborn infants (Review)". The Cochrane Collaboration. Wiley.

81 John Worrall. (2003). "What Evidence in Evidence-Based Medicine." in Causality: Metaphysics and Methods. CPNSS: 14.

82 Jeremy Howick. (2011): 44.

83 Jeremy Howick. (2011): 44.

#### *2.3.2 Confounding factors during trials:*

If, despite all efforts to the contrary, confounders do appear during the trial which are influencing the results, the trial needs to be changed in some way.

An obvious solution would be re-randomisation or stratification, taking the new known confounder into account. Re-randomisation however is hardly ever discussed since the option of choice is to match the results post trial or to abandon the trial altogether. Since re-randomisation sounds like the most common-sense approach, I will briefly discuss the reasons why it is not a viable solution to the problem. In re-randomisation it could happen that the previously gathered data cannot be used anymore. The whole trial would have to be organised again, preferably with new participants and a new protocol. If the same participants would be re-randomised, a wash-out period would be needed, since almost all drugs stay in the body for a certain amount of time. Additionally, the treatment or drug would have already shown benefits, that for example in a placebo-controlled trial only appeared in the treatment arm. Blinding would be nearly impossible to maintain. To add new participants to the 'old' participants would be equally complicated, even if a wash-out period is maintained, because the reactions to the new treatment might be different and again blinding would not be maintained. The participant base is therefore too contaminated to successfully extrapolate useful research results.

Another problem of re-randomising is that it is time-consuming and therefore costly. RCTs however are extremely costly to begin with and the number of participants, regardless of how many there are, is usually too small anyway. Research hospitals and clinics have provided facilities and staff for a certain amount of time to run a trial and in the case of re-randomisation all that would be needed again, again at substantial costs to the research facility. Since in many RCTs it is unsure if the trial leads to positive results, and since there is always a risk that it does not, re-randomisation is simply too expensive to consider, even if it would be scientifically the best way to go about the problem of controlling mid-trial for unknown confounders. In actual clinical practice the least time- and money consuming solution to the problem is to factor the now-known confounders in post-trial. The method most often used to account for confounders after a trial is the 'analysis of covariance method' also called ANCOVA.84. However, ANCOVA only works reliably if the groups are homogenous. The method however does not work in every case. Sometimes it can only demonstrate the problem. And the same is true for

84 Medcalc. Easy to use statistical software. Software provider for statistical methods in medical research. https://www.medcalc.org/manual/analysis-of-covariance.php. Last accessed on January 23rd, 2020.

almost all statistical models in use today.85 Therefore it is important that randomisation works in the first place without having to result to statistical means after the trial.

It is a simple mathematical truth however that very large and diverse trials are the ones which control the best for most possible confounders, since there are simply enough participants to equal out the groups.86 In small trials, and many trials are small, which means too small for randomisation to do its entire magic, confounders are, almost necessarily, unequally distributed, even if some sort of stratification has taken place. This is also the main argument John Worrall uses against randomisation. However, as Jeremy Howick rightly points out, the supporters of EBM and randomisation do not make that strong a claim that randomisation is the only control. Randomisation is an important tool in the set-up of trials, but not the only one to maintain internal validity. And many critics, including Worrall, do not provide a sufficient alternative to randomisation, save then saying that confounders can be controlled for statistically once a trial is finished.87

#### *2.3.3 Blinding — its problems and its virtues*

Blinding is used to control for many forms of bias, many of which can severely alter the results of every trial. Different possible biases are:

85 K. P. Suresh. (2011): 8-11.

86 Jeremy Howick. (2011): 50. Howick and many others have already pointed that out. It is almost impossible to create a trial that is sufficiently large enough to control for all confounders. That is why in the set-up for most trials, in-trial check-ups are mandatory to see if the distribution is still equal.

87 John Worrall. (2003):

Many authors have defined more biases, but these four are the most important ones because they are the ones that have to the most influence on the actual trial results.

Randomisation without blinding can only control for the initial selection-bias if and when the researcher is prevented through randomisation to decide to which treatment arm the participant is allocated.88 All further forms of selection-bias and especially the selection-bias on part of the participant are only avoided if at least the researcher and the participant are blinded. "Six groups involved in a trial are sometimes blinded, namely participants, caregivers, data collectors, outcome evaluators, statisticians, and manuscript authors."89 However it is often not defined or explained in research protocols and post-trial write-ups which of the six possible groups was actually blinded.90 All groups can be filled in by different people or one person can fill in almost all of the roles. In the write-up of an RCT it should be made clear not only which groups were blinded, but also how many different people or groups there were to blind, and if the blinding could be maintained for the duration of the trial.

Howick for example groups together the researchers and those who are dispensing the treatment into "caregivers" but separately names "data collectors" and "outcome evaluators." It seems to be an arbitrary decision and one that every author and researcher can make for themselves. The lack of a reporting standard and the lack of clear definitions of terms can make the results of RCTs less robust and therefore vulnerable to manipulation.

The most often used method of blinding is double-blinding. This means that at least participants and researcher are blinded. If Howick's terminology is assumed, researcher means caregiver. 'Caregiver' can include the acting physician and possibly the nurse who might dispense the treatment. In order to successfully maintain the blinding however it should be at least triple-blinding, including the participants, those who dispense the medication and the researcher. The latter should be blinded regardless if he or she is having an active part in the trial, or is more on the sidelines, organising the trial and assessing the results.

After a blinded randomisation, blinding has to be maintained during the trial. The randomisation codes should not be revealed and all possible treatments, be it the new, the standard, or placebo has to be sufficiently alike to not be distinguishable from each other. There are different methods available to achieve this, depending on the type of the treatment. Oral treatments, as in pills or liquids, can be manufactured in a way that the control treatment looks, smells and tastes exactly like the treatment under test. This is true for both placebo-controlled trials and

88 John Worall. (2004): 2.

89 Jeremy Howick. (2011): 65.

90 Isabelle Boutron and David Sackett, et.al. (2006). "Methods of Blinding in Reports of Randomized Controlled Trials Assessing Pharmacologic Treatments: A Systematic Review." PLoS Medicine, Volume3, Issue 10: 1923.

active controlled trials, where the novel treatment is tested against the accepted standard. Taste and smell can be masked by strong flavours such as peppermint or simply sugar. Food colouring works as well, or the use of gelatine capsules which all look the same from the outside and have the same texture.91 There are many methods available to make the different treatments, standard, novel and placebo to look and taste alike and the whole scope of possibilities is usually used to successfully blind all those who are involved in the trial. Treatments which have to be administered intravenously or intramuscular or any other way rather than oral, can also be manipulated in such a way as to mimic the treatment under test.

It can be hard to maintain the blinding, especially in cases where the intervention under test yields dramatic results, either positive or negative, very early on. For some philosophers of science dealing with EBM such as John Worrall or Jeremy Howick, the fact that the blinding can be hard to maintain and easy to subvert is a real problem. However, I believe that especially in those cases were there are dramatic results, the occurring unblinding can be easily dealt with. And sometimes it is even necessary to quickly and effectively treat the participants, especially when the effects are dramatic and not in a positive, but in a negative way, as for example in the TGN1412 trial of 2006.92 In the TGN1412 phase I trial, eight healthy male participants were randomised to receive either TGN1412 or a placebo. Six received the active treatment and deteriorated very quickly due to a cytokine storm, a condition in which the entire immune system and consequently the organs are shutting down very quickly.93 Accordingly it was immediately obvious which two participants had received the placebo. All six participants who had received the active treatment were subsequently treated in intensive care and survived, but they did so with lasting repercussions to their health and well-being. The most dangerous 'flaw' that happened during the trial was that the initial dose was too high and that the intervals in which the drug was administered were too short. The waiting period between the single administrations of the drug in each individual participant and the intervals between administering the drug to the next participants should have been significantly longer.94

However, it is important to realise that especially phase I trials have the potential to be dangerous for the participants and are therefore very tightly controlled. They are only be approved if a maximum of safety can be guaranteed. The unblinding in the example occurred quickly and without the possibility of avoiding it, much to the safety and security of the participants who could be treated fairly quickly.

91 Isabelle Boutron and David Sackett. (2006): 1935.

92 Michael Goodyear. (2006). "Learning from the TGN1412 trial." in BMJ: 1-2.

93 E. William St. Clair. (2008). "The calm after the cytokine storm: lessons from the TGN1412 trial" in Journal of Clinical Investigation. 118(4): 1344–1347.

94 E. William St. Clair. (2008): 1344-1347.

A further problem which Howick mentions in connection to blinding is the Philip's paradox which claims that many dramatically successful interventions are not supported by best evidence, if one follows the claim that best evidence can only be attained through successfully blinded and randomised RCTs.95 The usual examples for such interventions are the Heimlich manoeuvre to unblock a closed airway of a chocking person or the removal of the appendix in a patient with acute appendicitis. However, it seems as if the Philip's paradox is not a real problem for medical research. Nobody would propose a non-surgical option for acute appendicitis and similar surgeries. It sometimes seems as if EBM proponents claim that research always starts at the bottom and has to work its way up. Instead, 76 - 96% of treatments used today are actually proven to be effective and can be used as comparison, or base, for novel treatments.96 And since it is less ethically questionable to compare a novel treatment against the standard treatment, instead of against a placebo, it is actually of great value that the available 'standard' treatments are that effective.

#### *2.3.4 Blinding in surgical trials*

Surgical trials are a special area of medical testing, because some important tools that make RCTs so internally valid are difficult to maintain during a surgery. Blinding would be one of these tools. Among all the groups which can possibly be blinded, at least the performing surgeon needs to know which surgery he or she is performing.97 There are two surgical trial options available however in which at least the patients and most other groups can be blinded. Either the novel surgery is compared against the standard form of surgery, or it is compared against a form of 'placebo' surgery. The latter is a so-called 'sham' surgery. 'Sham' surgeries mean that the patient is prepped as for a real surgery. A small incision is made and subsequently looks like a real wound, just that it is not as deep and the resulting scar might be less dramatic. Since the patient still has to receive anaesthesia, there is a certain risk of complications involved without any benefit at all, especially when 'general anaesthesia' is used. And even small incisions can lead to scars and sensitive tissue around the surgical area and hence, in the worst case, to longterm problems. So there is, even when sham surgeries are used, no entirely safe way to perform surgical trials. These trials are often performed with the overall goal to make a surgery as minimally invasive as possible or to reduce the time and amount

95 Jeremy Howick. (2011): 64.

96 Jeremy Howick. (2009). "Questioning the Methodological Superiority of 'Placebo' over 'Active' Controlled Trials." in The American Journal of Bioethics, 9(9): 33-48.

97 Paul Karanicolas, Forough Farrokhyar, and Mohit Bhandari. (2010). "Blinding: Who, what, when, why, how?" in Canadian Journal for Surgery, 53(5): 345-348.

of anaesthesia. Anaesthesia itself is very risky and can even be life-threatening. Recently researchers are focusing more on the connections between anaesthesia and amnesia and even dementia.98 Anaesthesia for major surgery can, and often does, lead to to some type of temporary memory loss. Most patients experience that memory loss but do not suffer any further consequences because it is such a short-term phenomenon. In rare cases when the patient wakes up during surgery, the intrinsic memories of waking up, irrelevant if the patient actively remembers or not, can lead to post traumatic stress disorder and therefore to longterm harm for the patient.99 Current research also investigates the risk of dementia caused by anaesthesia for older, not necessarily only elderly, patients. So far there are not enough research results to back-up the hypothesis of a connection, but that does not mean that there is no risk.100

The use of 'sham' surgeries is also regulated by the Declaration of Helsinki, since 'sham' surgeries are equal to the use of placebo, albeit having a higher risk. The Declaration of Helsinki states that the use of placebo is allowable "where for compelling and scientifically sound methodological reasons the use of any intervention less effective than the best proven one, the use of placebo, or no intervention is necessary to determine the efficacy or safety of an intervention and the patients who receive any intervention less effective than the best proven one, placebo, or no intervention *will not be subjected to additional risks of serious or irreversible harm as a result of not receiving the best proven intervention. Extreme care must be taken to avoid abuse of this option."*101 (My emphasis). Following the Declaration of Helsinki, it is advisable to compare new surgical methods against the standard. That the blinding here cannot be fully maintained has to be accepted.

#### **2.4 Placebo and the placebo response**

Placebo means in Latin "I will please". Placebos are pills or injections that look and smell like a treatment with an active ingredient but are actually made either from sucrose or lactose or come in the form of a saline solution as an injection.

98 V. Fodale and L.B. Santamaria, et.al. (2010). "Anaesthetics and postoperative cognitive dysfunction: a pathological mechanism mimicking Alzheimer's disease." in Anaesthesia, 65: 388– 395.

99 Walter Glannon. (2014). "Anaesthesia, amnesia and harm." in Journal of Medical Ethics. 0:1-7.

100 Alzheimer's Research UK. "General anaesthesia linked to increased dementia risk." Published online May 31st, 2013. http://www.alzheimersresearchuk.org/general-anaesthesia-linked-to-increased-dementia-risk/. Last accessed October 15th, 2017.

101 World Medical Association, The. Declaration of Helsinki. Adopted by the 18th WMA General Assembly, Helsinki, Finland, June 1964. Lastly editorially revised by the 64th WMA General Assembly, Fortaleza, Brazil, October 2013. Article 33, Use of Placebo.

Placebo pills can be manufactured and manipulated in many ways to mimic an actual treatment, but usually do not contain any active ingredients. Placebos can however trigger a special effect in the patient, despite their lack of active ingredients. "The placebo effect arises out of the patient's confidence in the treatment (of the physician)."102 It can be triggered by the physician, if he or she can convince the patient that the offered treatment will be succesful. These instances of the use of the placebo-effect are linked to the phenomenon of 'classical conditioning' and refer back to the physiologist Ivan Pavlov who discovered that his laboratory dogs would salivate if he, or one of his assistants, would enter the room to feed the dogs. After a while they would salivate even if one or the other would arrive without food. The dogs were conditioned to react in that way, regardless of the presence or absence of food. In the same way a human being can be conditioned to believe in an effect, even if the there is no actual effect to be measured, as in the use of placebo.103

The American anaesthetist Henry Beecher used saline injections on soldiers during World War I when morphine was running low. He discovered that they worked, especially when he told the soldiers that they received a powerful pain medication.104 Again, the placebo-response here refers back to some type of conditioning which can be very strong. In the instance of the soldiers receiving the injection, the use of placebo was ethically correct because it had, due to the conditioning, a positive effect on the patient and there was no active treatment available. However, successful placebo effects can lure the patient into a false sense of security, since only the symptoms are addressed, not the underlying disease.

Testing active treatments against placebo factors the placebo effect for both into the results. Every active medication contains both, the active effect and the placebo effect.105 However the placebo effect is not present in every patient and it is certainly not the same for every patient.106 Therefore it would be a logical fallacy to attribute a 100% occurrence to the placebo effect. Still, it needs to be taken into account. When a new active ingredient is tested against the standard treatment, the placebo effect is also factored in, because it is potentially present in both cases.

Since the placebo effect is unspecific and can vary from patient to patient, it is difficult to filter it out for correct assessment. Some participants do not react at all to the placebo, some might react overly strong. And there is always the possi-

102 Edzard Ernst and Simon Singh. (2008)*:*57.

103 Ivan P. Pavlov. (2003). *Conditioned Reflexes.* New York: Dover Publications. [Book is a compilation of his writings over time.]

104 Henry K. Beecher. (1955). "The Powerful Placebo." in JAMA: 1602-1606.

105 Edzard Ernst and Simon Singh. (2008): 65.

106 Natalie Grams. (2015). *Homöopathie neu gedacht. Was Patienten wirklich hilft.* Berlin, Heidelberg: Springer Verlag: 133.

bility that the actual placebo effect is confused with other naturally occurring effects of the disease. Regression to the mean or fluctuation of symptoms can be mistaken for, and reported as, a placebo-effect, without the possibility of clarifying which is which.107

#### *2.4.1 Passive versus active placebo*

To make matters even more complex, placebos can come in two different forms, active placebo and inactive placebo. Inactive or pure placebos are those that mimic the active treatment in looks, smell and taste but contain no active ingredients. Active placebos are supposed to mimic all of the above and additionally contain active ingredients to bring only the side-effects about.108 If the side-effects are known or suspected beforehand and can be included in the placebo, then in the case of positive side-effects the inclusion might be at best beneficial and at worst without any consequences at all, since not all possible side-effects have to appear in every patient. The case is different with regard to negative side-effects. In some treatments, the negative side-effects are well known and accepted because the overall outcome of the treatment is beneficial and the negative side-effects are just temporary. Chemotherapy is an example in which harmful side-effects are taken for granted, but are accepted because the benefits outweigh the harms done by the treatment. The same is true for every surgery, in which bodily harm is inflicted for the purpose of healing the patient.

To produce a placebo with harmful side-effects, but without any positive treatment effects, can potentially pose a real risk to the health of the participant that is not outweighed by some positive outcome. These placebos are also called nocebos and they can either be manufactured or the nocebo-effect appears in the patient, although an inactive or an active placebo has been given.109 "Nocebo" again is latin for 'I will harm.' Since risks should be minimised in trial settings, active placebos are highly problematic and their use is ethically challengeable. In cases where an active placebo would be needed, because otherwise the blinding is impossible to maintain due to the lack of side-effects in the control arm, it would be advisable to opt for the standard treatment as a possible control instead. Only if there is no standard treatment available should an active placebo be used.

107 Damien G Finniss, Ted J Kaptchuk, Franklin Miller, Fabrizio Benedetti. (2010). "Biological, clinical, and ethical advances of placebo effects." in The Lancet; 357: 686-95

108 Isabelle Boutron and David Sackett. (2006): 1938.

109 Herbert Benson. (1997). "The Nocebo Effect: History and Physiology." in Preventive Medicine, 26 (5): 612-615.

However, even inactive placebos are not without problems. They might suffer from a misnomer inasmuch as that there is no such thing as an inactive pill. Placebos contain something and they trigger a response, and even if it is just the bodies response to a small dose of sugar. Can something that does something ever be called inactive?110

Some proponents of placebo-controlled trials are adamant that the side-effects should be prevalent in the placebo, since it would be impossible to keep the blinding if the treatment arm noted side-effects and the control arm does not.111 Again, due to the placebo effect however, participants in the placebo arm might report side-effects. It is not as clear-cut a phenomenon as some of the placebo-trial proponents like it to be. The knowledge and subsequent sharing of any side-effects can lead to contamination and can therefore invalidate the results of the trial. The question remains if the risk of contaminating the results is really higher than the risks inherent to the actual side-effects of an active placebo? The reason why I estimate the risk of the active placebo higher is that it is a direct risk to the participant and one which cannot be avoided if active placebos are used. The risk of contamination can be avoided by separating the participants of the two treatment arms whenever it is possible. In other cases, when participants are bound to meet in the clinic or the doctor's office, what I would call patient to patient contamination, could be avoided by putting a clause in the initial agreement that forbids to talk about any symptoms of the medication. Allegedly that approach does not help against possible selection bias on the part of the researcher when the obvious absence of known side-effects jeopardises or even violates the blinding. However, a trial can still yield successful results, even if the blinding is terminated prematurely, as in fact happens in many trials.112

#### **2.5 Stopping clauses**

The 'ideal' running of a trial would entail that it runs until set endpoints are reached. The timing is often pre-calculated, since trials are expensive and cannot run indefinitely, due to lack of funds and willing participants. As we have seen before, trials are run to either test for safety or efficacy or both. Every trial however can be stopped prematurely, if and when necessary, as was for example the TGN1412 trial. In order to do so, many trials have stopping clauses implemented in their protocol. Stopping clauses define when a trial can be stopped for either safety or efficacy, before it actually has run its pre-ordained course. In order to

110 Jeremy Howick. (2011): 72.

111 Jeremy Howick. (2009): 37. And Howard Brody and Franklin Miller. (2002). "What makes placebo-controlled trials unethical?" in American Journal of Bioethics. 2(2): 3-9.

112 Jeremy Howick. (2011):

successfully use stopping clauses however, terms like 'harm', 'safety' and 'efficacy' must be scrutinised closer.

Safety and efficacy are both natural endpoints for trials; as are mortality and morbidity. The TGN1412 trial is one example where the trial was stopped prematurely because of harm to the participants. Harm is divided into toxicity and death. In the literature the terms 'harm', 'safety' and 'toxicity' are often used interchangeably. "…morbidity and/or mortality outcomes may reflect both risk and benefit.113 As counterintuitive as this sounds, there are cases where obviously contradicting endpoints where established and one was deemed to be the actual endpoint, while the other was supportive, or surrogate, endpoint. This distinction means that negative results can be measured in interim analyses and do not lead to the trial being stopped, because they are considered to be a normal part of the trial. But one which is only limited and still supports an overall positive outcome. The idea behind this utilitarian sounding reasoning is that it cannot be quite clear if a treatment is harmful or successful until all the data is accrued. Surrogate endpoints should never be used as a reason to stop a trial prematurely.114

There are four specific reasons that are mentioned in most of the literature for stopping a trial early:115

Following Bassler's et al. reasoning here, it would be best if interim analysis are done by an independent committee. These committees are data monitoring committees (DMC) or data safety and monitoring boards (DSMB). They should be independent from those organising the trial and are therefore not blinded. If a committee is part of a trial set up, it can be blinded, because it would be sufficient to have it see the raw data. That is another reason why these committees are used more and more frequently. They can act as outside observers who do not have any interest in the outcome of the trial and who can therefore more easily determine if there is a problem in the setup or the conducting of the trial. "Nowadays, some

113 Yanli Zhao, Patricia Grambsch and James D. Neaton. (2007). "A decision rule for sequential monitoring of clinical trials with a primary and supportive outcome." in Clinical Trials, (4): 140-153.

114 Jan Sprenger and Jacob Stegenga. (2017). "Three Arguments for Absolute Outcome Measures." in PSA2016: The 25th Biennial Meeting of the Philosophy of Science Association.

115 Dirk Bassler, Victor M. Montori et. al. (2008). "Early stopping of randomised clinical trials for overt efficacy is problematic." in Journal of Clinical Epidemiology 61: 241-246.

funding, ethics, and regulatory bodies consider an independent DMC essential for major RCT's. For instance, the Food and Drug Administration has published a draft guidance for clinical trials sponsors on the establishment and operation of clinical trials DMCs."116, 117 The important note on the FDA guidance is that it is non-binding. That means that other guidelines can be used, if and when the FDA ones do not fit to the actual setup of a trial. On the one hand, this approach gives room for multiple ways of setting up of trials and of establishing stopping clauses and DMC's, on the other hand, non-binding guidelines mean that they do not have to be used at all, and that trials can be set up without these safeguards.

If a trial is set up with stopping rules, there are initially two ways in which those can be used. One way is an 'or' stopping rule, which means that the trial is stopped "for either a safety *or* efficacy outcome."118 The other way is an 'and' stopping rule, meaning that the trial is stopped for safety *and* efficacy, rendering the results potentially more robust.119 In phase 0 and I trials, 'safety' is the main concern of the trial and therefore the natural endpoint. In phase II, III and IV trials, when a base-safety is established, 'efficacy' is of higher concern and can be the more natural endpoint of a trial, without neglecting safety. If real safety concerns do appear in later trials, those are definitive indicators to potentially stop the trial prematurely.

There are multiple ways, and multiple reasons, to terminate a trial before it would actually end. Stopping rules should be decided before the start of a trial and specified in the trail protocol.

Judged by the amount of literature about stopping rules in the case of benefit (a lot),120 compared to the case of harm (very little)121 it would seem as if the decision to stop a trial in case of harm or toxicity is fairly easy. However, if looked at it closely, even in the case of harm, it is not an easy decision to stop a trial. A multitude of concerns can play a role here. Ethically it sounds obvious to stop a trial that is dangerous for the participants involved. However, some 'harms', like some side-effects might be only temporary. "The decision to stop early for harm is potentially more complex than benefit or futility because it may involve a trade-

116 Yanli Zhao, et.al. (2007): 142.

117 Food and Drug Administration (FDA) guidelines: "Guidelines for clinical trial sponsors." https://www.fda.gov/downloads/regulatoryinformation/guidances/ucm127073.pdf. The guidelines are fairly specific but nonbinding. It is assumed that when a different method is more appropriate for the test on hand then that method will be used. Last accessed on January 23rd, 2020.

118 Yanli Zhao, et. al. (2007): 140.

119 Yanli Zhao, et.al. (2007): 141.

120 Unrefined PubMed search with the search terms "stopping rules due to benefit" yielded 67 results.https://www.ncbi.nlm.nih.gov/pubmed/?term=Stopping+rules+due+to+benefit. Last accessed on January 23rd, 2020.

121 Unrefined PubMed search with the search terms "stopping rules due to harm" yielded 8 results. https://www.ncbi.nlm.nih.gov/pubmed/?term=Stopping+rules+due+to+harm. Last accessed on January 23rd, 2020.

off between potential — but as yet undemonstrated — benefits and apparent (but possibly spurious) adverse effects."122 Mill et.al. argue further on how important these DMCs in the possible case of stopping a trial for harm are, because they might be able to distinguish real harm from side-effects and can react quickly to avoid greater harm for the participants.

In the case of stopping a trial for benefit, the question can only be if the benefit is a real benefit and not a spurious event at that particular point in the trial which makes the treatment look to be beneficial, but hides harms that would have come to light later on, for example through 'regression to the mean' of the disease in question. "The social value of the research is severely compromised when overly sanguine estimates of treatment effect result in misleading risk - benefit ratios, misguided practice recommendations, and suboptimal clinical practice…."123

Stopping clauses are important, because they can act as a safe-guard to prevent harm for the participant and in very rare cases they can speed up a drugapproval process, if, and only if, benefit is established beyond a doubt. The latter case can be important if and when no standard treatment is available and the treatment under test is the only one to prevent harm for the patients in the actual target population. This is most often the case in drug-trials concerning orphan drugs.124 Orphan drugs are those drugs that are treating a very small patient population with a very rare disease. Often orphan drugs are poorly researched because they bring little to no profit. However, if they are researched, most patients are quite desperate for the trial results because no other treatments are available. If early on a benefit is detected it might be possible to stop a trial early and launch a phase IV trial after market approval. Another solution however would be to enter in a compassionate use program, so that patients outside the trial can receive the treatment and the actual trial can run its pre-considered course, without the recourse to any stopping clauses.125

#### **2.6 External validity**

As we have seen, when RCTs are performed correctly, they are internally valid and therefore their results should be robust and presentable. Questions however remain: presentable for what, and for whom? Can the treatment be used in the

122 Edward Mills, Cooper Curtis, et.al. (2006). "Randomized Trials Stopped Early for Harm in HIV/AIDS: A Systematic Survey." in HIV Clinical Trials 7(1): 24-33.

123 Dirk Bassler and Victor M. Montori. (2008): 244.

124 Aaron S. Kesselheim, Jessica A. Myers and Jerry Avorn. (2011). "Characteristics of Clinical Trials to Support Approval of Orphan vs Nonorphan Drugs for Cancer." in JAMA, 305 (22): 2320-2326.

125 Hanna I. Hyry and Jeremy Manuel et.al. (2015). "Compassionate use of orphan drugs." in Orphanet Journal of Rare Diseases. 10 (100).

target population as it is used in the treatment population? Is the treatment even feasible in the target population? How can the problem of external validity or extrapolation, as it is also called, be solved?

The results of a RCT with a positive outcome show us that the treatment can be used for the population in which the treatment has been tested. However, treatments are not developed for some small trial population, they are supposed to work in the wider population who suffers from this specific disease. But are the treatment and the target population sufficiently alike so that the treatment could be successfully used in the actual target population? How can this necessary 'alikeness' be established? The problem of transporting research results from trial conditions to the actual target population is called the problem of external validity or of extrapolation. Some authors do use the term generalisability, but for the sake of clarity, I will stick with 'external validity', as the opposite to 'internal validity.' Howick prefers to use the term extrapolation, because sometimes it can be necessary to extrapolate results within a trial to make it applicable to a participating subgroup. As an example Howick uses a trial in which a beneficial result was only pertaining to a very small subpopulation, within the larger trial population, with a very specific condition.126 Even though the results have been extrapolated to a specific target population, the trial over all is not at that point deemed to be externally valid.

It is often said that the higher the internal validity of a trial, the bigger the problem of external validity. So the focus here is on assessing what contributes to the difficulty in extrapolating results in the set-up of, and recruitment for, RCTs and how trials results can be made usable for evidence-based practice.

#### *2.6.1 External validity and trial design*

The problem of external validity already becomes prevalent in the set-up of a trial. The claim: 'the more internally valid a trial is, the less externally valid the results seem to be' is already precluding that the 'ideal' set-up of a trial might not be the 'best' set-up.127

The randomisation process in RCTs is used to prevent selection bias and to render control-and treatment arms as alike as possible, in order to achieve a common base line. Participants are chosen in such a way as to accomplish comparability. A certain similarity is therefore helpful. Most trial designs include a run-in

126 Jeremy Howick, et.al. (2013). "Problems with using mechanisms to solve the problem of extrapolation." in Theoretical Medicine of Bioethics; 34: 277.

127 Nancy Cartwright. (2007). "Are RCTs the Gold Standard?" in BioSocieties. (2): 11-20.

period as a so-called 'enrichment strategy'.128 One type of run-in period is when already included participants have to submit to a wash-out period before data is gathered. That means that they have to stop taking other drugs or stop certain habits, like smoking, so that no traces are left in the body, before they can participate in the trial. A different form of wash-out period is when participants, also before data gathering, are given placebos and those participants that respond to the placebo are not included in the actual trial.129 Reaction to placebo however can again be very varied and might be hard to detect, since any change in the patient receiving the placebo can have reasons unrelated to the non-working treatment. It is questionable therefore if a run-in period based on the placebo criteria is helpful.

Another type of run-in period is the randomised withdrawal design. Here participants are excluded from the trial when they are either non-compliant or if they show toxic effects because of the treatment under test. All types of enrichment strategies are compromising the external validity of RCTs. But they are still fairly prevalent, because they guarantee a more favourable result overall, and the more favourable the result for a new treatment, the earlier it will be approved and released into the market, thereby making profit. Hence, any form of enrichment strategy of a trial should be avoided if the trial results are supposed to be external valid overall and valid for the individual patient as well.

When all possible participants are included in the trial, with or without a runin period, then the researchers have achieved nearly perfect laboratory conditions. But they have also narrowed the playing field considerably. The ideal participants for a trial seem to be male, in their mid-thirties to mid forties and without any coor multi-morbidities.130 And even if gender and age is more mixed, the lack of coand multi-morbidities plays a significant role, because they can severely influence the outcome of any trial. Either because necessary medication that cannot be flushed out, interacts with the novel treatment or the disease under trial is overshadowed by these other diseases. Female patients are asked less often to participate in a trial, even if the treatment in question is treating a disease that is not dependent on gender.131

Obviously most trial participants in real life are far removed from the above ideal. The chief reason for this is that almost no one, regardless of gender, is free

128 Robert Temple. (2012). "Enrichment Strategies." in Guidelines of the U.S. Food and Drug Administration. https://www.fda.gov/ucm/groups/fdagov-public/@fdagov-afda-orgs/documents/document/ucm 303485.pdf. Last accessed on September 15th, 2017.

129 Ariel Pablos-Méndez, R. Graham Barr, and Steven Shea. (1998). "Run-in Periods in Randomized Trials Implications for the Application of Results in Clinical Practice." in JAMA 279(3): 222-225.

130 Ben Goldacre. (2012): 159.

131 Wendy Rogers. (2004). "Evidence-Based Medicine and Women: Do the Principles and Practice of EBM further Women's health?" in Bioethics. (81)1: 61.

of co - or multi-morbidities. Simple examples for this can be asthma, skin or joint problems, etc. Those minor ailments which the patient does not even perceive as morbidities but which nonetheless can alter the outcome of a trial can and are used to exclude possible participants from trials. Hence, the wish to either exclude as many of these predicaments or to distribute them equally to both groups is very understandable in theory, but hard to maintain in practice. Equal distribution is a valid goal in trial designs, but the exclusion of possible participants based on their co-or multi-morbidities is not advisable, since the results that would be garnered by such an idealised trial would not be generalisable to the target population. Often age does play a role. Children and the elderly are often excluded, again explained as a precaution. "Up to 90% of potentially eligible participants are sometimes excluded from trials according to often poorly reported and even haphazard criteria. For example, the most effective antidepressants in adults have doubtful effects in children."132

#### *2.6.2 External validity and recruitment*

Since the ideal participant, with the qualities outlined above, seems to be nonexistent, the goal the recruiter has to aim for is 'eligible'. The patient has to be eligible to become a participant. Therefore, recruitment of a group of eligible patients is of the utmost importance for a trial. But who is eligible? First of all, the patient/participant has to have the illness in question to even be considered for a trial from phase II onward. Phase I is usually conducted with healthy volunteers.133 Secondly, the patient has to be at the right place at the right time. Some trials are advertised openly, but recruitment often happens on a walk-in basis, meaning that the clinic conducting the trial asks those patients who are walking in with the disease in question, to participate. Naturally, that excludes many eligible patients who are treated in a different clinic or by a GP who is not informed about the trial. Open advertising does not reach that many possible participants because as always one has to be attuned to look out for it. If one is not aware about the possibilities of a trial and is not made aware by their health-care provider, the clinics miss out on many eligible volunteers.

132 Jeremy Howick, et. al. (2013): 277.

133 There are, as always, exceptions to this rule. Some phase I trials are conducted in participants with the illness in question, because the treatment is so special that it should not or could not be trialled in healthy volunteers. A currently running trial at the UKE in Hamburg comes to mind in which children with NCL2, a severe form of childhood dementia, are receiving a very experimental treatment that would not be ethical to give to healthy volunteers. https://clinicaltrials.gov/ct2/resu lts?cond=NCL2&term=&cntry=&state=&city=&dist=. Last accessed on January 23rd, 2020.

Even if the patient is eligible and is asked to participate, he or she might have good reasons to refuse. Many patients are afraid of being treated as human guinea pigs and rather opt for the standard treatment. Or they are afraid of just receiving the placebo. However, if they agree, it can be for many different reasons which subsequently can influence the actual outcome of the trial.

There are different types of patients which are participating in trials, but almost none do so for a greater societal good. Either there is a financial component involved, or there is a real sheer need on the part of the patient, because the disease is so rare or so little studied, that even the chance of receiving some treatment, 50/50 in placebo controlled trials, is better than receiving none. A financial component can be that the patients are paid, as is very common in phase I trials where healthy participants are recruited. Again the TGN1412 trial comes to mind. And even in later trials, some sort of compensation is often offered.134 Another case can be that patients within a trial receive care and medication for free and do not have to pay for it or have to claim payment from the insurers. Often trial medication can be very expensive. Additionally, in a trial setting, clinicians are a lot more thorough and the overall care is often better and the patients are more closely monitored. So, for many patients, being recruited to participate in a trial can be a win/win situation.