Chronic pain is a widespread health problem (1, 2), where various factors may influence the experience of pain and functional capacity. The patients are followed up by their GP, but may need follow-up by the health trusts’ interdisciplinary pain management clinics or in interdisciplinary pain management centres (3).
The pathway into the specialist pain management services requires a referral from a GP or the specialist healthcare service. The referrals are assessed within ten days, preferably by interdisciplinary admission teams (3, 4). The legal right to health care is granted if the patient ‘may be expected to benefit from the health care provided and the expected costs are reasonably commensurate with the outcome of the intervention’ (5).
Section 2–2 of the Patients’ and Users’ Rights Act specifies that rights should be granted on the basis of the referral (6), and the national guidelines for organisation and operation of interdisciplinary pain management centres (3) emphasise that the referral must provide a ‘clear description of the patient’s problem’, and contain ‘information that characterises the patient in a biopsychosocial sense’. An inadequate referral may complicate the decision of whether or not to grant the patient access to health care. The guideline therefore points out that ‘a full and concise referral is the best guarantee for ensuring that the patient is correctly prioritised’ (3).
A number of studies have shown that referrals to the specialist health services are of varying quality (7–12). Lønning and co-authors (2009) found that one-third of 198 referrals received by the medical outpatient clinic of Buskerud Hospital were inadequate (7). In an evaluation of 256 referrals to an interdisciplinary spinal clinic in St. Olavs Hospital, Gulati and co-authors (2012) found that no more than 2.1 % contained information that covered categories that were assumed to be important (8).
In addition to the quality of the referrals, it is crucial that the criteria for ‘the legal right to health care’ be interpreted and practised equally. There are examples showing that this is not the case. A study of 14 district psychiatric centres found little consistency in the assessments of rights (13).
In 2015 and 2016, the Storting allocated funds to a pilot project for development and operation of interdisciplinary services to patients with chronic pain and fatigue disorders. A primary goal was ‘to establish a good and regionally coordinated service option for this patient group’ (14–16). The Centre for Health Care Improvement at St. Olavs Hospital is evaluating the pilot project, and in this article we will report findings from the first study of the evaluation. The objective of the study was to investigate 1) the extent to which the admission teams in four pain management centres make concurrent assessments of the quality of the patient referrals received, and 2) whether the teams draw concurrent conclusions regarding the patients’ right to necessary health care.
Material and method
The study is based on 40 referrals that have been assessed by the admission teams in four pain management centres: the Centre of Pain Management and Palliative Care, Haukeland University Hospital; the Department of Pain Management, University Hospital of Northern Norway; the Department of Pain Management and Research, Oslo University Hospital; and the Department for Pain and Complex Disorders, St. Olavs hospital, Trondheim University Hospital. The admission teams are interdisciplinary and consist of doctors, psychologists and physiotherapists. During the 38th and 39th week of 2017, each admission team included ten regular referrals in the order in which they were coming in and assessed them for rights in the regular manner. In the following, these assessments will be referred to as ‘primary assessments’. The referrals were anonymised and forwarded to the project leader. The centres later received copies of the referrals from each of the other three centres. Thereby, all four admission teams undertook another 30 assessments, here referred to as ‘secondary assessments’.
Based on the recommendations in the manual ‘Organisation and operation of interdisciplinary pain management centres’ (3) regarding the content of a good referral, representatives of the four admission teams prepared a screening form for assessing the referrals (see appendix). The admission teams were asked 1) to note the information that was deemed important in the assessment of the referral (with the response alternatives ‘of major importance’, ‘some’, ‘none’ or ‘insufficiently described’), 2) to assess the quality of the referral (‘not good’, ‘good’ or ‘very good’), and 3) to state the priority (‘legal right to health care’, ‘no legal right to health care’ or ‘further information needed’).
In the primary assessment, supplementary information from the patient record could be used for the rights assessment, and this was also recorded in the screening form.
After the secondary assessment, the copies of the referrals and the screening forms, both of which were marked with unique numbers to ensure correct linkage, were returned to the project leader. All 40 referrals were assessed four times – one primary and three secondary assessments – meaning that the four teams undertook a total of 160 assessments.
The Regional Committee for Medical and Health Research Ethics (REK) considered this to be a quality assurance study and outside the scope of their responsibility. According to the notification test defined by the Norwegian Centre for Research Data (NSD) (17), the project was not subject to the duty to notify, because the data material was anonymised.
Two statistical methods were used to assess correspondence between the admission teams (‘inter-rater reliability’): per cent agreement and intra-class correlation coefficient (ICC). Per cent agreement (18) was used when there was insufficient variation in the assessments, which tends to occur when there are few response categories (19). The agreement was deemed sufficient – or acceptable – if it was in the 75–90 % range (20). ICC was used to evaluate correspondence between the primary and secondary assessments of quality, according to the following equation: ICC (absolute agreement, k assessment team) = variation in quality assessment for each referral / (variation in quality assessment for each referral + (variation in repetition + measurement error) / k) (19). Since the evaluation teams were not randomly selected, we used a two-way mixed-effects model with the quality of the referral as the dependent variable. We used the average of the assessments from the admission teams and absolute agreement (i.e. identical assessments by all four admission teams) as the definition (19). Four ICC analyses were undertaken, one for each of the centres that undertook primary assessments in relation to the three centres that made the secondary assessments. Statistical analyses were performed with the aid of IBM SPSS Statistics version 23 (IBM Corp., Armonk, NY).
There is no clear consensus as to what ICC value should be deemed to represent acceptable agreement. In line with previous recommendations (13, 21) we chose to use the following normative values: ICC < 0.20 = little agreement; 0.21–0.40 = weak agreement; 0.41–0.60 = moderate agreement; 0.61–0.80 = high agreement; > 0.80 = very high agreement.
Results
Information in the referral and its impact on the assessment
Of the 16 information categories, seven were referred to as ‘insufficiently described’ in more than one-half of the 160 assessments. In more than 70 % of the assessments this applied to the categories ‘stated motivation for the programme’ (85 %), ‘substance use/addiction disorder’ (78 %) and ‘sleep disturbances’ (76 %) (Table 1). For these three categories, the agreement between the admission teams amounted to 85 %, 83 % and 84 % respectively.
Table 1
Impact of the information in the referral, n (%).
|
Information in the referrals (N = 160)
|
Of major/ some importance
|
None
|
Insufficiently described
|
Per cent agreement (%)
|
|
Stated motivation for the programme1
|
18 (11)
|
5 (3)
|
134 (85)
|
85
|
|
Substance use/addiction disorder
|
25 (16)
|
11 (7)
|
124 (78)
|
83
|
|
Sleep disturbances
|
33 (21)
|
5 (3)
|
122 (76)
|
84
|
|
Previous treatment in a pain management clinic2
|
38 (24)
|
11 (7)
|
107 (67)
|
70
|
|
Nature of the pain3
|
40 (25)
|
19 (12)
|
100 (63)
|
71
|
|
Depressive disorders/anxiety/catastrophic thinking
|
54 (35)
|
8 (5)
|
98 (61)
|
76
|
|
Pain intensity
|
61 (39)
|
13 (8)
|
86 (54)
|
65
|
|
Activity level/physical impairments
|
77 (48)
|
5 (3)
|
78 (49)
|
73
|
|
Examination considered complete
|
117 (73)
|
9 (6)
|
34 (21)
|
71
|
|
Location of the pain
|
115 (72)
|
26 (16)
|
19 (12)
|
61
|
|
Previous treatment, not in a pain management clinic
|
101 (64)
|
12 (8)
|
47 (29)
|
52
|
|
Duration of the symptoms3
|
93 (58)
|
23 (14)
|
43 (27)
|
60
|
|
Findings from the clinical examination and supplementary examinations/diagnostics (medical imaging, EMG, MR neurography, lab, etc.)
|
85 (54)
|
6 (4)
|
69 (43)
|
68
|
|
Comorbidity
|
83 (52)
|
21 (13)
|
56 (35)
|
63
|
|
Social status (family situation, friends, social functioning)
|
68 (42)
|
30 (19)
|
62 (39)
|
64
|
|
Employment
|
62 (39)
|
58 (36)
|
40 (25)
|
70
|
Quality of the referrals
In the primary assessments, 45 % of the referrals were assessed as ‘not good’, 40 % as ‘good’ and 15 % as ‘very good’. In the secondary assessments, the total distribution was 43 %, 45 % and 12 % respectively. Table 2 shows how the primary and secondary assessments of the quality of the referrals were distributed among the categories ‘not good’, ‘good’ and ‘very good’ for each centre, and the degree of correspondence (ICC) between the primary and secondary assessments. The correspondence between the primary assessment and the respective secondary assessments was high for referrals that underwent primary assessment at Haukeland University Hospital, moderate in the primary assessments at Oslo University Hospital and the University Hospital of Northern Norway, and low for primary assessments undertaken at St Olavs Hospital. The latter hospital stood out in that the quality of all ten referrals was deemed ‘not good’ in the primary assessment, while the three other centres deemed the quality ‘good’ for a majority of the same referrals.
Table 2
Agreement in the assessment of quality between primary assessments from each centre and the corresponding secondary assessments. Number of referrals per response category. HUS = Haukeland University Hospital, OUS = Oslo University Hospital, STO = St. Olavs Hospital, UNN = University Hospital of Northern Norway.
|
|
Quality of the referral
|
Absolute agreement
|
|
|
Not good
|
Good
|
Very good
|
ICC 1
|
|
Primary assessment (HUS)
|
3
|
6
|
1
|
|
|
Secondary assessment (UNN)
|
2
|
6
|
2
|
|
|
Secondary assessment (OUS)
|
3
|
6
|
1
|
|
|
Secondary assessment (STO)
|
7
|
3
|
0
|
|
|
|
15
|
21
|
4
|
0.74 (0.36–0.92)
|
|
|
|
|
|
|
|
Primary assessment (UNN)
|
3
|
3
|
4
|
|
|
Secondary assessment (STO)
|
8
|
2
|
0
|
|
|
Secondary assessment (OUS)
|
4
|
2
|
4
|
|
|
Secondary assessment (HUS)
|
6
|
4
|
0
|
|
|
|
21
|
11
|
8
|
0.51 (−0.05–0.85)
|
|
|
|
|
|
|
|
Primary assessment (OUS)
|
2
|
7
|
1
|
|
|
Secondary assessment (STO)
|
8
|
2
|
0
|
|
|
Secondary assessment (HUS)
|
5
|
4
|
1
|
|
|
Secondary assessment (UNN)
|
3
|
5
|
2
|
|
|
|
18
|
18
|
4
|
0.57 (0.03–0.87)
|
|
|
|
|
|
|
|
Primary assessment (STO)
|
10
|
0
|
0
|
|
|
Secondary assessment (OUS)
|
1
|
6
|
3
|
|
|
Secondary assessment (HUS)
|
2
|
8
|
0
|
|
|
Secondary assessment (UNN)
|
3
|
6
|
1
|
|
|
|
16
|
20
|
4
|
0.19 (−0.27–0.68)
|
Legal right to health care
Overall, the primary and secondary assessments gave 63 % of patients ‘the legal right to health care’, while 37 % were given ‘no legal right to health care’.
Table 3 shows that 18 % of the referrals were assessed identically by all centres. Three of four centres made identical assessments for 45 % of the referrals. For 38 % of the referrals, two centres granted the legal right to health care, while two rejected the legal right to health care. The rate of agreement between all the centres amounted to 69 %, which is lower than the 75 % limit for ‘acceptable agreement’.
Table 3
Assessment of referrals (N = 40) regarding the legal right to health care. Agreement between the primary and secondary assessments, as well as between the secondary assessments, n (%)
|
|
Four centres agree
|
Three centres agree
|
Two centres agree
|
No centres agree
|
Per cent agreement (%)
|
|
Primary and secondary assessment
(four centres)
|
7 (18)
|
18 (45)
|
15 (38)
|
Not relevant
|
69
|
|
Secondary assessment (three centres)
|
Not relevant
|
10 (25)
|
26 (65)
|
4 (10)
|
68
|
In the secondary assessments, all three centres made identical assessments in 25 % of the cases, while two centres made identical assessments in 65 % of the cases. In 10 % of the cases, all the centres made different assessments. The degree of correspondence amounted to 68 %. Supplementary information from the patient records was used in 68 % of the primary assessments, while the secondary assessments were made exclusively on the basis of the referrals alone. Table 4 shows the agreement between each team that undertook the primary assessment and the respective secondary assessments.
Table 4
Referrals that were granted the legal right to health care per centre, n (%). Agreement between the primary and secondary assessments. Referrals assessed as ‘further information needed’ are excluded. HUS = Haukeland University Hospital, OUS = Oslo University Hospital, STO = St. Olavs Hospital, UNN = University Hospital of Northern Norway.
|
|
|
Granted the legal right to health care
|
|
|
|
Centre
|
Primary assessment
|
Secondary assessment
|
|
Per cent agreement
(%)
|
|
|
|
HUS
|
UNN
|
OUS
|
STO
|
|
|
HUS (n = 10)
|
6 (60)
|
–
|
8 (80)
|
4 (40)
|
9 (90)
|
78
|
|
wUNN (n = 10)
|
6 (60)
|
7 (78)1
|
–
|
3 (30)
|
2 (33)4
|
60
|
|
OUS (n = 10)
|
7 (70)
|
8 (80)
|
4 (67)2
|
–
|
5 (100)5
|
65
|
|
STO (n = 10)
|
6 (60)
|
4 (40)
|
5 (63)3
|
5 (50)
|
–
|
70
|
Discussion
The objective of this study was to investigate the degree of consistency between the four Norwegian regional pain management centres in their assessments of the quality of the referrals that they receive and in their conclusions regarding the patients’ right to necessary health care. The assessment of the content of the referrals showed that 7 of 16 information categories were to a large degree insufficiently described, and the admission teams deemed the quality of nearly one-half of the referrals to be ‘not good’. These findings are consistent with those made by previous studies (7, 8, 10) and confirm the impression that in many cases referrals are of insufficient quality and frequently lack information which is essential to undertake an adequate assessment of the right to treatment in the specialist health service.
Insufficient information was an important reason why a large proportion of the referrals were characterised as ‘not good’. Chronic pain is a complex condition, and substantial competence is required to describe the complexity of the illness. Low quality of referrals may reflect a need for competence enhancement on the part of the referrer. Referrals are frequently made by GPs, but the referrals in our study had come from both GPs and the specialist health service, and we were unable to distinguish between them. The list of recommended information items to include is long (3), but most likely, not all recommended information items will be relevant for all patients. The referral needs to communicate adequate information, but selecting the correct information items is a challenge not only for the referrer, but also for the pain management clinics. We found that there was moderate to low agreement between the admission teams in their assessment of quality. This means that the same referral was perceived differently by the various teams, which may reflect differences between them in terms of competence and experience. As a result, the information in the referrals may be weighted differently, signalling a need for harmonisation of the content assessment.
The overall rate of agreement between the admission teams regarding whom to grant the legal right to health care was lower than what is considered acceptable. The primary and secondary assessments granted 63 % the legal right to health care, but the rate of agreement was 69 %. The admission teams had no list of correct answers to consult, and the assessments were made on the basis of information in the referrals, with supplementary information in the patient records if available, as well as recommended guidelines and the experience in the team. What constitutes an acceptable level of agreement is thus debatable. Our results are consistent with findings made by previous studies. Ikezawa and co-authors (2010) investigated whether clinicians who were working in the same rehabilitation centre gave identical back-to-work recommendations. For patients with unambiguous pathologies they found a high degree of agreement (> 94 %) regarding the kind of information that was important in the assessment of the patients’ discharge status, but low agreement in cases where the causes of pain and disability were complex (56 %) (22). A Norwegian study of admission teams in 16 district psychiatric centres also found a low rate of agreement in their assessments of the rights to health care for 20 selected referrals (13).
The above findings reflect the challenges that arise when the patient’s clinical picture cannot be assessed in a classically medical and pathology-oriented perspective, and this is characteristic of the patient population in our study.
Our results indicate that whether a pain patient will be granted the legal right to health care or not, depends on which centre assesses the referral. The differences between the assessments may reflect varying competence in the centres, but also differences in treatment options within and outside the hospital. As long as competence and treatment options are not explicitly communicated, there is a risk that hidden and undesirable differences may arise between the health regions. The results may also indicate that the Guidelines to Priority Setting: Pain Conditions (4) ought to be reviewed. In addition, it might be considered whether they ought to be adapted for this complex patient group.
It is a recognised fact that the system of referrals needs structuring (23, 24). Standardisation of referral forms and guidelines, as well as involvement of clinicians in training programmes about the referral process, have been shown to produce improvements (9). Rokstad and co-authors (2013) found that the quality of specialist referrals was raised after the introduction of a normative information system. The system was used by 93 (the intervention group) of a total of 210 GPs. The quality of the referrals improved significantly, and less time was used (25). Our findings emphasise the need for a tighter structure that can guide the referrers regarding the information which is necessary.
The weaknesses in our study lie in the relatively small number of referrals that were assessed (40 in total), and that the screening form for the referrals was not validated. The centres that undertook the primary assessments had access to supplementary background information in the patient records, and this weakens the comparison with the secondary assessments. Within their normal time schedule, the admission teams had to undertake secondary assessments of 30 copies of referrals, in addition to the regular referrals. As a result, the reviews may have been less thorough and the assessments of rights may have been affected. The study’s strength lies in its realistic nature, since all the assessments were undertaken within the standard clinical framework.
Conclusion
Our study shows that a large proportion of the patient referrals to the four large interdisciplinary pain management centres are of low quality and provide insufficient information. The centres granted the legal right to health care for 63 % of the referrals, but there was a less than acceptable degree of agreement between the centres with regard to what patients qualified for health care. The findings indicate a need for a more structured system of referrals, harmonisation of assessments and more lucid guidelines to priority setting that can help ensure more equality in service provision to patients in different regions.
