On 12 March 2013, Jonas Gahr Støre, then Minister of Health and Care Services, declared that patients with metastatic malignant melanoma would receive the drug ipilimumab, provided that they otherwise fell within the approved indication. This was a complete about-turn from the decision made by the Directorate of Health shortly before, and Støre was criticised for this move. In my opinion, this was one of the most visionary decisions he ever made during his career as a government minister.
In issue 21/2012 of the Journal of the Norwegian Medical Association, published on 12 November, there was a small announcement from the Norwegian Medicines Agency, declaring that ipilimumab (Yervoy) is not cost-effective for treating advanced melanoma, since the treatment costs NOK 1 million per quality-adjusted year of life (1 ).
This modest announcement was not proportionate to the importance of the principle involved in this matter: this was the first time that the Norwegian Medicines Agency had undertaken an assessment of the cost-effectiveness of drugs for use in hospitals. The decision stirred a debate and gained support from the Director of Health (2 ), but was revoked by the Minister of Health the very next day (3 ). His justification was evasive and not appreciably based on principle, but the decision was the only correct one and perhaps also the only one that was politically possible.
The Norwegian Medicines Agency had calculated their way to a conclusion that was so far removed from common sense that the Minister of Health had to intervene, even when this meant that he had to be disloyal to his own civil servants. How could this happen?
The simple facts
The effect of ipilimumab was investigated in a controlled study, in which 676 patients with advanced melanoma were randomised into groups receiving ipilimumab or the cancer vaccine gp100 (defined by the study as standard treatment), or a combination of these.
After one year, 45 % of the patients who had received ipilimumab-based treatment were still alive, against 25 % of those who had not. After two years, the figures were 23 % against 14 %. The survival curve (Figure 1) gives the impression that ipilimumab accounts for 20 % long-term survivors, but the numbers are small and the estimates uncertain, and any predictors of which patients may profit from the treatment have yet to be identified (4 ). Adverse effects are mainly autoimmune diseases that can be serious: colitis, hepatitis, hypophysitis and dermatitis (5 ).
Figure 1 Survival curve for treatment with ipilimumab or ipilimumab in combination with the cancer vaccine gp100 versus only gp100. Reprinted from Hodi et al. (4 ) with permission from the New England Journal of Medicine
The financial side of the issue, to which politicians also need to relate, is that the drug costs close to NOK 820 000 per patient. In Norway, approximately 150 patients will be candidates for this treatment each year, although some will be ineligible for various reasons. Depending on the number of patients who will in practice be eligible, the total annual expenses have been estimated to NOK 53 – 80 million (6, p. 37).
Politicians as well as people in general are fully able to relate to such data, which could have formed a good basis for a political decision.
The complicated facts
Difficulties arise when one wishes to assess the effect on so-called Quality-Adjusted Living Years (QALY). This implies that an extra year of survival will not necessarily be counted as one year, but must be recalculated on the basis of an assumption about the quality of the patient’s life during that year. For this purpose we need a decimal fraction, a figure in the range 0.0 – 1.0, as an indicator of the patient’s quality of life with which to multiply the gain in life expectancy.
If you live for one year longer than you would otherwise have done and maintain full quality of life (quality of life 1.0), you have gained one QALY. If you live for two years longer than you would otherwise have done, but maintain only half your quality of life (quality of life 0.5), you have also gained one QALY. In other words, we need to find an indicator of quality of life which can be quantified as such a decimal fraction.
The approval study for ipilimumab (4 ) identified the patients’ quality of life with the aid of the questionnaire EORTC QLQ-C30, one of the most common instruments for measuring quality of life in oncological research. It contains a total of 30 questions on issues such as symptoms (e.g. «During the past week, have you had pain?»), function (e.g. «Do you have any trouble taking a long walk?» and emotional conditions (e.g. «During the past week, did you feel tense?»). Two of the questions have seven response categories, the remaining have four. Theoretically, there are approximately 3.5 · 1018 different ways to respond to the questionnaire, and the way it is completed says nothing about how the respondent values his or her current condition.
From this, how should one produce a decimal fraction ranging from 0 to 1 that can tell us something about the value of life? At this point, the calculations are becoming so complicated that confusion may ensue.
The manufacturer’s calculation
The Directorate of Health requested the manufacturer of ipilimumab, Bristol-Myers Squibb, to submit a pharmacoeconomic analysis. The company chose a method in which EORTC QLQ-C30 scores are recalculated to another instrument for measuring quality of life, EORTC-8D. Using sophisticated mathematical methods and based on scores from 655 American patients with myelomatosis, an algorithm has been developed to allocate any combination of scores on EORTC QLQ-C3 to a score on the simpler instrument EORTC-8D, which has only eight dimensions, each with four or five levels (Table 1) (7 ). Even this results in a total of 81 920 possible response combinations. Further simplification is required if this instrument is to be used for calculation of QALY.
Table 1 The dimensions of the quality-of-life instrument EORTC-8D with examples of categories (7).
Dimension
Number of response alternatives
Best and worst category
During the past week …
Physical functioning
5
… you had no trouble taking a long walk
… you had very much trouble taking a short walk outside of the house
Role functioning
4
… you were not limited in pursuing your hobbies or other leisure time activities
… you were limited very much in pursuing your hobbies or other leisure time activities
Pain
4
… pain did not interfere with your daily activities
… pain interfered very much with your daily activities
Emotional functioning
4
… you did not feel depressed
… you felt depressed very much
Social functioning
4
… your physical condition or medical treatment did not interfere with your social activities
… your physical condition or medical treatment interfered very much with your social activities
Fatigue and sleep disturbance
4
… you were not tired
… you were tired very much
Nausea
4
… you did not feel nauseated
… you felt nauseated very much
Constipation and diarrhoea
4
… you were not constipated and did not have diarrhoea
… you were constipated and/or had diarrhoea very much
Eighty-five of these combinations were therefore selected and submitted to a panel of 350 persons randomly drawn from the population of Northern England (response rate 40.3). Each of them was presented with eight of the 85 selected combination, and for each combination they were asked whether they would prefer to live for ten years with the health condition in question and then die, or to live for x years in good health and then die. The number x was changed until the respondent started expressing doubts about the choice.
For example, if the respondent was in doubt when x was set to 9, the interpretation was that he or she gave the health condition described (the given combination of the eight statements in Table 1) a value of 0.9. Since each of the 85 selected combinations was assessed by more than one respondent, an average value could be calculated for each combination.
However, 85 combinations (health conditions) account for only an infinitesimal proportion of the 81 920 possible combinations. Regression analysis was therefore used to estimate the weight that should be apportioned to each of the scoring alternatives of all the eight dimensions in EORTC-8D. In this way, a value could be assigned to all of the 81 290 health conditions (7 ). By retrieving values for EORTC QLQ-C30 from the original study (4 ), recalculating them to EORTC-8D and estimating their utility value as described here (7 ), Bristol-Myers Squibb estimated the cost per QALY gained to NOK 771 000 within a time horizon of 15 years (6, p. 22).
The Norwegian Medicines Agency’s calculation
The Norwegian Medicines Agency assessed the pharmacoeconomic analysis presented by Bristol-Myers Squibb (6 ). They based their review on two flawed premises.
First, they claimed that the recalculation from EORTC QLQ-C30 to EORTC-8D was based on an algorithm that «as far as the Norwegian Medicines Agency is aware» remained unpublished (6, p. 28), and thus had to be deemed uncertain. However, the algorithm had been published at the time (7 ), a fact that could easily have been ascertained, for example by a search in PubMed.
Second, the Norwegian Medicines Agency made a point of claiming that the quality of life data in the algorithm (7 ) came from patients (6, p. 19, p. 28), while in reality they came from a normal population.
The Norwegian Medicines Agency used data from two other sources in addition to the one used by Bristol-Myers Squibb: a study in which health conditions that are common in the case of malignant melanoma are valued by representatives of the normal population (8 ) and an estimation of weights for quality of life based on the instrument SF-36 (9 ). This gave an estimated cost per QALY gained with the aid of ipilimumab of NOK 929 000 and 956 000 respectively for a time horizon of 15 years (6, p. 23).
The Norwegian Medicines Agency noted it as «problematic» that the three estimation methods yielded cost estimates that varied by more than NOK 200 000 per QALY, and chose to use an average of the three estimates. Moreover, they chose a time horizon of ten years instead of fifteen, and thus arrived at a cost per QALY gained of exactly NOK 990 739 (6, p. 32). According to the Norwegian Medicines Agency (1 ) and the Directorate of Health (2 ), this cost is excessive.
Discussion
The objective of this article is not to discuss whether the Norwegian Medicines Agency or Bristol-Myers Squibb have calculated correctly, the objective is to call into question this way of making political decisions.
A number of problems are associated with the sequence of reasoning from quality of life data collected during the trial of ipilimumab (4 ) to the calculated price per QALY. Small variations in the calculation methods (none of which can be seen as more correct than the others) may have a decisive impact on the conclusion, as we have seen.
The use of highly sophisticated mathematical models gives a false impression of exactitude, and conceals the fact that no estimates are better than the assumptions upon which they rest. Are psychometric data for EORTC QLQ-C30 collected from patients with myelomatosis relevant when the form is to be used to measure the benefits of treatment in patients with melanoma? Are relevant aspects of the patients’ experience of their health condition preserved throughout the almost impenetrably complex processes needed to convert the 3.5 · 1018 response alternatives in EORTC QLQ-C30 to the 81 920 response alternatives in EORTC-8D and from there to the 85 alternatives that were finally evaluated?
Can we accept that Norwegian political priorities are to be based on a Gallup survey of 350 randomly selected persons in Northern England (with approximately 60 % attrition) on how long they would be willing to live with various degrees of symptoms and reduced ability to function? Is the questionnaire technique involved, asking people to weigh years of life in full health against years of life with reduced health, at all acceptable as a method for arriving at a decimal fraction for the value of life? If so, can we estimate the value of those 81 835 health conditions that were not evaluated directly by using regression analyses based on those 85 that were actually assessed? When cost estimates differ by 24 %, is it OK to simply use the average?
The complexity of the methods partly serves to conceal plain errors, as seen in the study done by the Norwegian Medicines Agency (6 ). More seriously, value choices are here presented as if they were calculations with an exact answer.
It is fine to have an opinion regarding whether NOK 53 – 80 million per year is too much to pay to permit approximately 150 patients with advanced melanoma increase their chances of long-term survival from near zero to around 20 %. This question can be debated openly, counterarguments can be put forward, and the electorate can let their vote be swayed by the response from the politicians. However, when the Norwegian Medicines Agency calculates that ipilimumab costs NOK 990 739 per QALY, the question is transformed into technical sophistry, with which it is impossible to agree or disagree. In terms of democracy and ethics, this is reprehensible, and it means that the agency’s bureaucrats are acting as politicians, but masquerading as researchers.
In the ipilimumab case, it gradually became obvious that the pharmacoeconomic analyses returned answers that were at odds with what most people found reasonable. Therefore, Støre needed to find a way to annul all the calculations made by the civil service, but preferably without making this about-turn appear too obvious.
The pretext was to launch a government-funded research project on predictors of the effect of ipilimumab. Thereby, all relevant patients could receive this as an «investigational drug». The reality is that all patients for whom the medical professionals determine an indication are permanently assured treatment with ipilimumab.
However, the regulations on how pharmacoeconomic analyses are to be performed have not been amended (10 ). It’s about time they were. Those who have a political point of view should declare it, and not disguise it as quasi-scientific mumbo-jumbo.
