Doctors are not supposed to guess. Written exams and other systems for the assessment of medical students should reflect the way doctors work.
Photo: Einar Nilsen
May and June means exam time, and that also applies to medical students. Exams are meant to ensure that the students have the requisite knowledge and skills, and that the universities produce competent doctors. There are strong indications that students prioritise the knowledge and skills that they know may be included in the exam, particularly when a grading scale is used. It has also been shown that testing can enhance motivation, concentration and effort (1 ). So, whether you like it or not: exams are important.
For those who studied medicine many years ago, there were essentially two types of exams: written and oral. Written exams generally consisted of a few questions that required the candidate to give an account of a central topic related to the relevant discipline, providing a fairly long and detailed answer – not unlike an essay. This type of written exam is now seldom used due to a limited and random choice of subjects (low validity) and time-consuming and subjective grading (low reliability). At the University of Oslo and other universities, written exams are now organised as so-called digital exams, where the students use a PC (2 ). These exams require considerable preparation on the part of teachers and faculty administration, but have many practical and pedagogical advantages.
Digital exams may consist of many types of questions, mainly various types of multiple choice questions (MCQ) and free-text questions requiring short answers (3 , 4 ). Multiple choice questions entail that for each question, the candidate must choose one of 4–5 given response alternatives (in some cases 2–3 of 6–10 response alternatives). Since such questions can be answered rather quickly, the exam may contain a high number of questions and thereby cover more subjects in the relevant discipline – in other words, they give higher validity. Studies show that multiple choice questions can distinguish relatively well between strong, average and weak candidates (4 ).
Multiple choice questions are, however, more difficult to formulate than many people are aware of. Ideally, these questions should test the candidates’ understanding and ability to apply knowledge, but we often end up with questions that instead test superficial knowledge and ability to reproduce knowledge. Another weakness is the significant chance that a candidate answers correctly through pure guesswork. In essence, the chance of answering correctly by guessing is 25 %, and as much as 33 % and 50 % if one and two of four response alternatives, respectively, are formulated in such a way that even weak candidates realise that they are incorrect. Some students answer correctly only after recognising the correct answer from among the response alternatives, described in the academic literature as the ‘cueing effect’ (4 ).
The most worrying aspect of multiple choice questions, however, is that the students prepare for, and become accustomed to, answering questions by choosing between given response alternatives rather than formulating an answer on their own. Proper medical work is based on the ability to observe, reason and justify (5 ). Doctors must not guess – clinical choices are not a guessing game. No patient comes to a doctor’s surgery with 4–5 relevant diagnoses written on a piece of paper that the doctor must choose between. Exams with well-formulated multiple choice questions can certainly test knowledge and adequately differentiate between students, but they have an effect on students’ way of thinking and ability to deal with difficult issues and challenges. Then validity and relevance may not be so high after all.
Free-text questions in digital exams should be answered with answers that are short and to the point – quite different from the long, essay-type answers in written exams of former times. The answers are scored by one or more teachers based on a pre-defined scoring template. The advantage of this type of question is that the candidates must answer without the help of given alternatives, in the same way that doctors must act in their clinical work. These questions also demand precise formulation of the questions. Free-text answers can be difficult to score, partly because it is impossible to predict all the answers. A consistent scoring practice can be secured by a good scoring template and by the teachers scoring pilot answers before the exam and having the opportunity to confer with each other after the exam. Free-text questions take somewhat longer to answer, so that the number of questions in one exam will be lower than if the exam contains only multiple choice questions.
The discussion regarding multiple choice questions and free-text questions is often presented as an irreconcilable battle between two camps, MCQers versus essayists . This is a mistaken, outdated and not very constructive notion. Both multiple choice questions and free-text questions with short answers have their strengths and weaknesses. They complement each other (4 ). Written exams with long, essay-type answers are no longer practicable.
It is positive that the interest in and attention to exams, exam types and grading in medical education seems to be increasing, both in Norway and elsewhere (6 , 7 ). Thorough preparations, high credibility among students and teachers, and the ability to complete the exam with a reasonable amount of time and resources are just as important as the form of the exams (3 , 4 ). Digital, computer-based exams are demanding, both for faculty and students. That is how it should be, for doctors’ work is also demanding.
To all those about to sit their exams: Good luck!
