Evidence-based practice is based on a combination of the best current research evidence, clinical expertise and the patients’ own preferences. In evidence-based practice, tools are used to find trustworthy answers to relevant clinical questions, and this requires skills in knowledge management. To achieve changes in clinical practice, knowledge management should be accompanied by quality improvement.
Doctors need to provide patients with a correct diagnosis and safe and effective treatment. Evidence-based clinical practice means integrating the best current research evidence with clinical, experience-based expertise and the patient’s values and preferences (1 , 2 ). Evidence-based practice is increasingly recognised as a required part of clinical practice, but is still not very common (3 – 5 ). This could help explain the gap between what doctors are doing in practice and what they ought to do based on scientific documentation (6 ).
At the same time, clinicians are struggling to find good answers to clinical questions. In an American study, young doctors in a medical department formulated a new clinical question for every other patient (7 ). Answers were found to only one-third of the questions, most often through unsystematic, random sources such as colleagues and textbooks, and this concurs with the results from a study of Norwegian doctors (4 ). Medical textbooks have not undergone quality assurance with regard to the collection, critical appraisal and updating of the evidence base. Advice from senior colleagues is invaluable in clinical practice but should also encompass updated and research-based knowledge.
Doctors need to possess basic skills in identifying, interpreting and applying research evidence in their own practice (8 ). Groups responsible for developing tools for decision support, such as clinical practice guidelines, local protocols and care plans – as well as clinicians who are using these tools – are to a varying extent aware of new standards and requirements for such tools (9 ).
Evidence-based practice can be regarded as a toolbox that enables us to find reliable and applicable answers to clinical questions. New methods and tools make the quest for trustworthy answers easier than before. In this article we will use the question described in the introductory vignette (enteral nutrition in the case of acute pancreatitis) to describe a stepwise process – from the recognition of a need for information through the formulation of a specific question and search and critical appraisal of research evidence to integration of this evidence through clinical experience and the patient’s preferences (Figure 1). Evidence-based practice in the health services is applicable at the individual as well as the system level (10 ).
Figure 1: Example of a clinical question as a starting point for a stepwise process from question to answer
Formulation of questions
The first step is the formulation of specific questions. In formulating the question, it is necessary to recognise that we have various types of core questions in health care, and formulate these questions in a precise manner. Clinicians tend to ask questions concerning diagnosis, prognosis and treatment, but they may also have questions related to prevalence, causes of disease or the experiences, values and preferences of patients.
Different types of core questions require studies with a suitable design. The question of enteral nutrition in the case of acute pancreatitis is related to treatment, and should be answered by studies with a randomised, controlled design. A correctly performed randomisation procedure ensures that prognostically important factors that may affect the outcome are equally distributed between the patients who receive enteral nutrition and those who receive total parenteral nutrition. Other systematic errors may be caused by insufficient blinding and large attrition of patients. For questions related to the patients’ experiences, values and preferences, a qualitative study design would be more appropriate.
Precisely formulated clinical questions highlight the problem and pave the way for the literature search. A precise question should comprise information about the patient group we are studying, the interventions we would consider and the outcomes that are relevant. The questions should be formulated in the so-called PICO format, i.e. with a precise description of the patient or the population (P), treatment alternatives, i.e. interventions (I) and comparators (C), and clinical end points/outcomes (O) (11 ). To ensure that the treatment provides more benefit than harm, we should emphasise those end points/outcomes that are important for the patient, and keep in mind that treatments may also have side effects.
A well-formulated PICO question for the problem facing this patient could be as follows: What is the effect of enteral nutrition on mortality, duration of hospitalisation, multiorgan failure, surgical treatment, sepsis and pain when compared to total parenteral nutrition for patients with acute severe pancreatitis?
Searching for research evidence
Traditional teaching of evidence-based practice typically emphasises systematic literature searches and critical appraisal of the identified studies. As this process is time-consuming to perform and difficult to master, clinicians need to have unrestrained access to clinical practice guidelines that provide recommendations for diagnosis and treatment.
Figure 2 shows the so-called knowledge pyramid (the 6S model), which has been developed to help doctors and other health care providers quickly identify research evidence about diagnosis, prognosis and treatment (12 ). The model ranks information sources in a hierarchy, where the uppermost ones, such as evidence-based textbooks and guidelines, contain advice and recommendations based on pre-appraised systematic reviews, while sources further down in the hierarchy, such as systematic reviews and individual studies, require more knowledge about research methodology, including critical appraisal of the results.
Figure 2: The Knowledge Pyramid, a hierarchy of information resources for clinical questions
Searching the literature has been simplified through web-based services such as The Health Library and Pyramid Searches. The Health Library (www.helsebiblioteket.no 13 ). The front page displays the Pyramid Search, which quickly provides answers at various levels of the knowledge pyramid with the aid of one or two search terms from the PICO question. Moreover, through the search engine McMaster PLUS, the pyramid search will help you find pre-appraised systematic reviews and individual studies that are deemed to be valid and interesting for clinical practice. The selected studies are easily available through e-mail alerts, which can be received after registering with the Health Library. Such e-mail alerts provide a good opportunity to keep updated in the field without being inundated by a flood of new studies.
e-Figure 3 shows the results from a pyramid search for the question concerning enteral nutrition in the case of acute pancreatitis. The surgeon entered the search term «acute pancreatitis» in the search field, and within two minutes she had found a relevant recommendation in the evidence-based textbook UpToDate . Here, a GRADE 1B recommendation for use of a nasojejunal tube was given, i.e. a strong recommendation (indicated by the figure 1) with a moderate quality of the evidence (indicated by the letter B).
Figure 3: Pyramid search for the question on acute pancreatitis, with a recommendation from UpToDate (via www.helsebiblioteket.no)
The surgeon is surprised. Even though she is somewhat uncertain of what is meant by GRADE 1B, her interpretation is that the patient should be provided with a nasojejunal tube, not with total parenteral nutrition. This contradicts prevailing practice in the department. She therefore wants to know more about the basis of this recommendation, and with one click in UpToDate she finds a text section that refers to a systematic review with documented positive effects of enteral nutrition on the outcomes that are important to the patient (14 ).
Critical appraisal
Doctors should be able to critically assess guidelines and research results before clinical practice is changed. The purpose of a critical appraisal is to make sure whether one can trust, understand and apply guidelines or studies in practice (15 ). The online training course www.kunnskapsbasertpraksis.no
The GRADE system (Grading of Recommendations Assessment, Development and Evaluation) (e-Box 1) is an alternative to traditional critical appraisal with the aid of checklists (16 ). This is a method for the systematic and transparent appraisal of the quality of research-based knowledge and for the development of professional guidelines. The GRADE system has been developed by organisations that participate in the development of guidelines and systematic reviews, including the Cochrane Collaboration. The Norwegian Directorate of Health recommends the use of GRADE for development of professional guidelines (17 ).
BOX 1
GRADE (Grading of Recommendations Assessment, Development and Evaluation)
System for quality assessment of research evidence and development of professional guidelines (www.gradeworkinggroup.org/society/
GRADE defines quality of evidence as confidence in the results (effect estimates) generated by the studies (16 ). The five following factors¹ may reduce the quality of the documentation from high to low:
Risk of bias: Was the randomisation concealed, who was blinded, was there major attrition?
Consistency: Are the results consistent across individual studies in the meta-analysis?
Applicability: Can we transfer the results to our clinical practice?
Precision: How precise are the effect estimates as expressed by the confidence intervals?
Publication bias: May key studies have remained unpublished?
In the development of recommendations in clinical practice guidelines, GRADE emphasises a systematic process based on an integrated assessment of four factors when moving from evidence to either strong or weak recommendations:
The balance between desirable and undesirable consequences of the intervention studied, expressed in absolute effect estimates.
Quality of the evidence: How much confidence do we place in the effect estimates?
The patient’s values and preferences: What would patients have chosen if they had been well informed about desirable and undesirable consequences of the treatment alternatives?
Costs: Does this treatment represent a sensible use of healthcare resources?
The strength of the recommendation reflects the following:
A strong recommendation indicates that the desirable consequences clearly outweigh the undesirable consequences, and is based on the assumption that virtually all well-informed patients most likely would have opted for this treatment.
A weak recommendation indicates a finer balance between advantages and disadvantages, and is based on the assumption that most well-informed patients would have opted for this treatment.
¹ Applies to studies with a randomised, controlled design. Observation studies start at low quality, but may end up with high quality on the basis of three factors that increase our confidence in the effect estimates.
Evidence-based textbooks and guidelines are intended to provide balanced recommendations by integrating research evidence and clinical expertise, patient preferences and other contextual factors, for example availability of resources (17 ). New standards and criteria for guidelines require, for example, a systematic review of the evidence and a balanced assessment of advantages and disadvantages of various treatment alternatives (18 , 19 ). The development of guidelines is a demanding task. This can be illustrated by a study of a random sample of international guidelines, less than half of which complied with established criteria for trustworthiness (9 ). The situation is no better with regard to Norwegian guidelines and local protocols.
Well-executed systematic reviews that summarise existing research evidence through meta-analyses provide the best knowledge base for the recommendations found in guidelines and should be used as a basis for clinical decisions in the absence of trustworthy guidelines. Criteria for critical appraisal of guidelines include clearly formulated questions, explicit inclusion criteria, systematic literature searches, critical appraisal of individual studies and summaries of results with the aid of adequate methods, such as meta-analyses. Effect estimates from meta-analyses are shown as a square diamond and provide the relative effect estimate for treatment (e-Figure 4).
Figure 4: Meta-analysis of enteral nutrition (EN) versus total parenteral nutrition (TPN) in the case of acute pancreatitis. From Al-Omran and collaborators (14 )
Individual studies are placed at the bottom of the knowledge pyramid, since they are not collated with other studies and since they also may include sources of error or other factors that limit their applicability in clinical practice (20 ). Nevertheless, individual studies may occasionally – after a systematic review of the documentation and a critical appraisal – constitute the best knowledge base for clinical practice (21 ).
The strong recommendation made by UpToDate for providing enteral nutrition through a nasojejunal tube to our patient Eva who suffers from pancreatitis is based on a well-executed, systematic Cochrane review and meta-analysis. The status of the Cochrane review as the best current research evidence for the question on enteral nutrition in the case of pancreatitis is confirmed by its position high up in the knowledge pyramid, at the level below evidence-based textbooks and guidelines (e-Figure 3).
e-Figure 4 shows a forest plot from the Cochrane meta-analysis for the outcome of mortality. Enteral nutrition in the case of pancreatitis results in a relative risk reduction of 50 % and an absolute risk reduction of 8 % (number needed to treat, NNT = 12) when compared to total parenteral nutrition. The surgeon interprets this as an impressive effect of enteral nutrition, in terms of relative as well as absolute effect.
Figure 5: GRADE Summary of findings table for the question on enteral nutrition in the case of acute pancreatitis. From Al-Omran and collaborators (14 )
The absolute effects of a treatment are often less impressive than the relative ones. A prime example is provided by mammography screening, whose introduction reduced the relative risk of death from breast cancer by 10 %. The corresponding relative risk reduction, however, amounts to less than 0.5 % (22 ). As important as being able to interpret relative and absolute effects is the ability to balance advantages and disadvantages by identifying effects on other outcomes that are important for the patient.
The surgeon decides to read the Cochrane review, and quickly retrieves the results of the meta-analysis summarised in a «GRADE Summary of Findings table». e-Figure 5 shows an excerpt from this table, in which the authors use the GRADE system to summarise relative and absolute effects of treatment and the quality of the evidence across the outcomes that are important for the patient.
Application and evaluation of new knowledge in practice
The final step in evidence-based practice is to apply new knowledge and evaluate this change of practice. This requires implementation of systematic efforts for quality improvement (23 ).
The surgeon who discovered that enteral nutrition was superior to total parenteral nutrition in the case of acute pancreatitis finds support in a strong recommendation from UpToDate and claims that practice ought to be changed, in light of the dramatic gains provided. Her colleagues agree, but remain uncertain about how to proceed.
Systematic quality improvement is only to a minor extent integrated into clinical work and specialist training of doctors. However, good training methods, tools and systems are available (23 ). Medical students at the University of Oslo learn about evidence-based practice and systematic quality improvement through the course «Knowledge management, leadership and quality improvement», referred to as the KLoK course (24 ). The Ministry of Health and Care Services and the Norwegian Directorate of Health have proposed to introduce mandatory training in evidence-based practice and also quality improvement in specialist training (23 ).
A group of medical students in Oslo have shown how a local hospital can implement enteral nutrition for patients with acute pancreatitis (25 ). The students undertook this assignment during their studies and in contact with the department of surgery, which now has good opportunities to implement this intervention in practice.
Training in knowledge management
Substantial training is required to develop sufficient skills in knowledge management (8 ). The medical curriculum at the University of Oslo now includes such training (24 ), for example through clinically integrated learning with the aid of work files (25 ). In an article in this issue of the Journal of the Norwegian Medical Association we describe how medical students in Oslo learn knowledge management through clinically integrated learning with the aid of such work files (26 ). Teaching of knowledge management is also on its way into the medical curriculum in Tromsø and Trondheim, and has been proposed as a key teaching module in post-graduate studies in Norway. Furthermore, training courses in knowledge management for hospital doctors have been developed, as well as a Nordic workshop for all professional groups in health care. In the meantime, the online training course www.kunnskapsbasertpraksis.no
