High participation rates in surveys and thus more representative data on health, disease and risk have helped raise the quality of epidemiological research in Norway (1 , 2 ) and the other Scandinavian countries (3 – 5 ). This has provided us with a unique opportunity to understand disease risks in the population and plan preventive measures that target the entire population or sub-groups of patients.
In recent years, however, the proportion that participates in such studies has declined. The Nord-Trøndelag Health Study (HUNT) started out in 1984 – 86 with a questionnaire that was completed by 89 % of those invited. Participation fell to 69.5 % in 1995 – 97 and to 54 % in 2006 – 08 (6 ). Falling participation rates have been observed in other Norwegian health studies as well, such as the different versions of the Tromsø study (7 ) and the Mother and Child Cohort Study (8 ). This declining willingness to participate in surveys over recent decades (1 , 2 , 9 ) calls for a study of the opportunities to increase participation rates.
In addition to the design and content of the questionnaire and the cover letter/envelope, research has focused on three different measures in particular to increase participation in surveys: the use of various modes to respond, reminders, and incentives (10 – 20 ). An incentive is something that motivates or encourages a specific action, and can be subdivided into unconditional and conditional incentives, whereof the latter is dependent on participation.
Intuitively, incentives should have a motivating effect on participation among those who otherwise would have declined, but their effect in medical questionnaire surveys is uncertain. Numerous studies have found that economic incentives such as gifts or rewards in the form of lottery tickets, gift certificates or money have an effect, but most of these studies are small-scale, and the results are contradictory (10 , 20 – 22 ). Comparisons of different incentives are especially difficult, because the various studies have used different incentives and investigated their effects in different patient groups (10 ). There are few randomised studies from Norway that involve comparisons of the effects of different types of incentives on participation rates in medical and health-related studies, and the conclusions in the various studies available are contradictory (21 , 22 ).
The purpose of our study was to investigate whether the use of incentives resulted in higher participation rates in a population-based health survey, and what kind of incentive would produce the highest response rate.
Material and method
The survey was undertaken within the Norwegian component of the NordICC study (Nordic-European Initiative on Colorectal Cancer) (23 ), which included 26 417 randomly selected women and men aged 55 – 64 years and resident in Aust-Agder or Vest-Agder counties. These were randomised 1 : 2 to the screening group, which received an invitation for a colonoscopy, or to the control group, which received no offer of a colonoscopy.
The colonoscopies were performed at Sørlandet Hospital’s departments in Arendal and Kristiansand in the period from January 2011 to June 2014. Of those invited, 5 354 (61 %) attended the screening examination and underwent a colonoscopy.
The questionnaire
On arrival, all those who attended for colonoscopy screening in Arendal and Kristiansand completed a validated questionnaire on lifestyle and risk factors for colorectal cancer (24 , 25 ). The one-page questionnaire contained four domains: height and weight, self-reported diagnoses, smoking and family history of colorectal cancer. We used the same questionnaire in our survey (e-Figure 1).
Figure 1 Questionnaire
Distribution and collection of data
In September 2014 we sent the questionnaire to a sample of persons who were included in the NordICC study, but who had not undergone colonoscopy screening. These included all those who had been randomly assigned to colonoscopy, but had not attended the examination (either because they had declined, were unable to attend or had not responded to the invitation) and a random selection of persons from the control group (not invited for colonoscopy). In this article we refer to the former group as «those invited for screening» and to the latter as «the control group» (Figure 2).
Figure 2 Flowchart for the NordICC study in Norway and the sample included in the survey (in red)
The questionnaire was sent by mail to all those included in the survey (in red in Figure 2). The envelope contained a one-page information letter which also detailed the terms of consent for participation, the questionnaire and a pre-paid response envelope. The wording of the information letter was adapted to the study groups. The letter also provided information on how the questionnaire could be completed online (an online questionnaire prepared by Epinion).
Those who had not returned the questionnaire within three weeks of the distribution date were contacted by telephone and asked whether they wished to participate in the study. They were provided with various options for responding. If they still had the questionnaire available at home, they could complete and return it, they could receive a text message or an email with a link to the online questionnaire, or they could answer the questions directly by telephone to the person who made the call, or alternatively be called back at a more convenient time. The calls and any resulting telephone interviews were undertaken by Epinion according to a script prepared by the researchers and the Epinion project leader in collaboration.
Randomisation to different incentives
Before the distribution of the letter, each of the two study groups (those invited for screening and the control group) were randomised (1 : 1 : 1) to three different study arms (Figure 3). The first group received no incentives. The second group received an unconditional incentive, a ten-kroner «Flax for Året» scratch card from Norsk Tipping [Norway’s state lottery agency] enclosed in the letter. The third group received a conditional incentive – if they responded, they would participate in a prize draw for four tablet computers (iPads). The formulation «Your response will provide valuable information on how the occurrence of colon cancer is linked to various risk factors» was used to encourage the participants in all three arms to respond..
Figure 3 Study design
Thus, the response rate could be influenced by two factors. Firstly, by the study group (those invited for screening and the control group). Those invited for screening had previously been offered a colonoscopy examination, but had chosen not to accept this offer. The participants in the control group comprised a random selection of the general population in this age group. Since those invited for screening included persons who had chosen not to attend the screening, one would expect a lower response rate in this group compared to the general population (here represented by the control group) (1 , 25 ). The incentive arm represented the other factor that could influence the response rate. The study thus consisted of a total of six different groups (Figure 3) – three study arms with different incentives for those invited for screening and the control group respectively.
Statistics and estimation of statistical power
On the basis of previous experience from distribution of questionnaires about lifestyle and colorectal cancer to equivalent study groups, we assumed that 10 % of those invited for screening and 60 % of the control group would respond to the survey without any incentives (24 , 25 ), i.e. a total participation rate of 35 %. With its 7 000 included research subjects, the study had a > 90 % statistical power (with a two-tailed alpha of 0.05) to determine a 10 % difference in participation between the non-incentive arm and the incentive arms and > 80 % power to determine a difference of 7 %.
Statistical significance was tested with the aid of Fisher’s exact text, chi-square test or t-test, depending on the type of data. We estimated the relative risk for the differences in response rate between the randomisation arms. The Breslow-Day test of homogeneity was used to detect differences between the screening and control groups for the different incentives. The analyses were performed in SAS version 9.2 for Windows (26 ).
Ethics
The study was approved by the regional committee of medical and health research ethics. The NordICC study is registered in clinicaltrials.gov (NCT00883792). A response to the questionnaire (on paper, online or by telephone) was considered to constitute consent to participation.
Results
We distributed 6 795 questionnaires – 3 398 to non-participants among those invited for screening and 3 397 to the control group. There were no statistically significant differences in age, gender or hospital affiliation between those who had been invited for screening and the control group (Table 1).
Table 1 Participant characteristics and results by study groups
Invited for screening N = 3 398
Control group N = 3 397
Total N = 6 795
Number
(%)
Number
(%)
Number
(%)
Responded to the questionnaire
1 478
(43.5)
2 227
(65.6)
3 705
(54.5)
Gender
Woman
790
(53.4)
1 173
(52.7)
1 963
(53.0)
Man
688
(46.6)
1 054
(47.3)
1 742
(47.0)
Age (years) (average, standard deviation)
62,0
(3.0)
61,9
(3.0)
Screening centre
Arendal
627
(42.4)
927
(41.6)
1 554
(41.9)
Kristiansand
851
(57.6)
1 300
(58.4)
2 151
(58.1)
Responded before the reminder call
999
(29.4)
1 665
(49)
2 664
(39.2)
Responded after the reminder call
479
(14.1)
562
(16.5)
1 041
(15.3)
Responded by telephone interview (percentages as the proportion of telephone responses of all responses)
294
(19.9)
243
(10.9)
537
(14.5)
Did not respond
1 920
(56.6)
1 170
(34.5)
3 090
(45.5)
Of these: Not reminded (no telephone number or not contacted by Epinion for other practical reasons)
509
(15.0)
308
(9.1)
817
(12.0)
After the reminder calls, altogether 3 705 of 6 795 of the included persons had responded to the survey (54.5 %) – 2 817 (76 % of the respondents) returned the completed questionnaire by mail, 537 (14.5 %) answered by telephone and 351 (9.5 %) completed the online questionnaire. Altogether 1 478 (43.5 %) of those invited for screening responded, compared to 2 227 (65.6 %) in the control group (p < 0.001). A total of 54 letters were returned by the postal service. Of these respondents, those with a known telephone number were contacted during the round of reminder calls and could respond by way of a telephone interview or receive a link to the online questionnaire by email or text message.
The effect of incentives
In the incentive arm that had been randomised to receive no prize, altogether 1 223 persons (54.0 %) responded, compared to 1 214 persons (53.6 %) in the scratch-card arm and 1 268 persons (55.9 %) in the prize-draw arm. Figure 4 shows the response pattern for the different incentive arms among those who had been invited for screening and the control group respectively.
Figure 4 Proportion of respondents to the survey by those who were invited for screening and the control group and by the various incentives
There were no significant differences overall in the response rate between the various incentives (p = 0.24), nor within the control group (p = 0.67) or among those invited for screening (p = 0.29).
The effect of reminders and response methods
Before the round of reminders, altogether 2 664 persons (39.2 %) had responded, while a further 1 041 (15.3 %) responded after having received the reminder call. The response rate was higher in the control group, before as well as after the reminder calls (Figure 4), and the total effect of the reminders was higher in the control group than among those invited for screening (16.5 % and 14.1 % responded after the reminder calls: p < 0.001). The proportion of responses provided through telephone interviews amounted to 19.9 % (294 persons) among those invited for screening and 10.9 % (243 persons) in the control group (p < 0.001).
Altogether 817 persons (12 %) received no reminder call, primarily because we had no information on their telephone numbers (15 % among those invited for screening, compared to 9 % in the control group; p < 0.001).
Discussion
The results from our study show that the use of economic incentives, such as small gifts (scratch cards) or a prize draw for gifts of a greater value (tablet computers) did not help increase the participation in a medical survey undertaken among people aged 55 – 64 years.
The participation rate was 54.5 %. This is lower than that reported from equivalent surveys in Norway previously (1 , 6 , 7 , 27 ). Our results show that those who had declined to participate in the colonoscopy screening (those invited for screening) had a lower response rate than the control group. The proportion of those invited for screening who responded (43.5 %) was nevertheless considerably higher than those 11 % who responded in a similar study of non-attendees in a comparable survey on risk factors for colorectal cancer in 2006 (25 ). The higher rate of participation among non-attendees in our study was surprising, since the scope and topic of the study were fairly similar to the previous investigation. Possible explanations could be that our target group consisted of persons who were from five to ten years older than those in the previous study, that the questionnaire contained fewer questions and that we followed up the non-respondents with reminder calls.
The objective of surveys is to generate knowledge that is valid not only for those who have responded, but for a larger group of people. Therefore, it is desirable that the responding group be as «similar» as possible to the larger group, meaning that they constitute a representative sample. As a researcher one can control that those who are invited to participate in a survey are representative, by drawing a random sample. One cannot control, however, that those who actually respond to the survey are representative.
A high response rate may be an indication of a more representative sample, but there is no guarantee that this is the case. There may still be people with special characteristics who are less inclined to respond to the survey, and this may give rise to a measurement bias when compared to a situation in which all those included have responded. For example, people with low socioeconomic status and a high prevalence of lifestyle-related diseases are often underrepresented (1 , 2 ), which results in artificially low prevalence figures for a number of diseases. Such bias is more discernible in measurements of the prevalence of a condition than in measurements of the association between exposure and an outcome (28 ).
In a Cochrane review on the use of incentives in surveys (not only medical ones), unconditional incentives that could be used by the participants irrespective of whether they participated or not (such as the scratch card in our study) had a greater effect than conditional incentives that were provided after the questionnaire had been completed (10 ). Our results showed no significant effect of the unconditional incentive (the ten-kroner scratch card). This tallies well with the results of another Norwegian study conducted in 2006, which found no increased response rate after distribution of scratch cards (22 ). Yet another Norwegian study from 2012 drew the opposite conclusion, however: enclosing a scratch card raised the response rate by 10 % compared to not giving out any prizes (21 ).
Feedback from those who undertook the reminder calls and telephone interviews indicates that many participants were highly motivated to respond. Many of them stated that colon cancer was a matter that concerned them and that they regarded as an important research topic. We thus have indications that the participants in our study initially had a strong internal motivation to respond. This may have contributed to the absence of any effect of the external economic incentives. Other factors that may help explain the incentives’ lack of effect include the fact that the study was undertaken in an age group that we may assume to be wealthier than young people; they can afford to buy their own scratch cards and tablet computers. Moreover, the questionnaire was relatively short, and we may surmise that this requires less external motivation than a long, time-consuming questionnaire.
More than 40 % of those invited for screening chose to respond to the questionnaire, despite having declined the colonoscopy screening examination. We expected to find a large co-variation between non-attendance of the screening and non-response to the survey. That this co-variation was lower than expected may indicate a difference in selection mechanisms. The reasons why someone failed to attend the colonoscopy screening may be health-related or of a practical nature, such as work duties or transport problems, but he or she may nevertheless have responded to the questionnaire which only required a few minutes to complete at home. In addition, there may have been a fear that the colonoscopy examination would be an uncomfortable experience, while this mechanism has no influence for the survey.
We found no increased response rate as a result of providing incentives to the participants in this medical survey – neither those incentives that were conditional upon participation, nor the unconditional incentives had any effect. In contrast, we observed a significant effect of reminder telephone calls to participants that had not responded and to whom we provided an opportunity to respond to the survey by telephone. This increase in response rate was especially pronounced among those who had been invited for screening, a group which is difficult to reach in such health studies.
Whether telephone reminder calls are more effective than sending reminders by mail, and whether alternative response modes alone increase participation, requires further study, as do any methodological, economic and ethical aspects of the various alternatives. When planning new studies within equivalent population groups, however, it may be appropriate to give priority to reminders and alternative response modes above provision of incentives.
