Questionnaires can be used to measure people’s thoughts and emotions. What is needed to make such information reliable and useful?
Depression is measurable. We can observe behaviour, examine biomarkers or use advanced brain scanning techniques. However, first and foremost we can ask a person how he or she feels. Measurement of mental states mediated by language is a science in itself (1, 2). Questionnaires are one variant of language-based measurement and currently used in a whole range of areas. But how useful are such forms really, and what are the preconditions for their usefulness?
‘Are you depressed? (Yes/No)’. This is one possible approach to measuring depression in a questionnaire, using a single, explicit question and dichotomous response categories. The problem inherent in using this approach is its limited reliability and validity. The respondents may interpret ‘depressed’ differently, we do not know exactly what we are measuring, and we fail to capture nuances and degrees of depression. It will often be better to ask multiple questions and add up the answers, preferably about more specific symptoms and with several response categories. For example: ‘During the last week, have you been bothered by any of the following symptoms? a) Feeling hopeless about the future, b) Feeling blue, c) Feeling everything is an effort’. Each statement comes with response categories ranging from ‘not bothered at all’ to ‘Extremely bothered’. Such a set of questions that measure a specific underlying phenomenon is frequently referred to as a questionnaire scale or simply a scale. In this case, the examples are taken from the Symptoms Checklist (SCL-8), sub-scale for depression (3, 4).
Depression is only one of numerous phenomena that can be measured with the aid of a questionnaire. This issue of the Journal of the Norwegian Medical Association includes two articles that describe validation of questionnaire scales. They address obesity-related quality of life (measurement of the extent to which obesity patients perceive their body weight and shape as bothersome) (5) and work-related self-efficacy (confidence in the ability to return to work after illness) (6) respectively.
A questionnaire scale is a tool/instrument that can be useful – or less useful. Measurement of depression is obviously useful with regard to treatment. Information on obesity-related quality of life can be important for treatment and prevention of overweight and obesity. If the objective of obesity treatment is not only to lose weight, but also improve quality of life, validated measurements of the latter will be essential. Similarly, a quantification of self-efficacy could be use useful to understand mechanisms that facilitate employment participation, to identify factors that predict return to work and as an outcome target for rehabilitation and interventions.
To be useful, a method must be reliable and valid (7). We want our bathroom scales to produce the same result three times in a row, not to show too much or too little, and actually measure our weight and nothing else. Reliability refers to whether a measurement can be relied on, whether it is reproducible. Validity means that we actually measure what we intend to measure. Similar to how bathroom scales or blood-pressure gauges put a figure on physical phenomena, questionnaires quantify behaviour, emotions or thoughts. For all measurement, empirical documentation of reliability and validity is required.
To be useful, a method must be reliable and valid
Some academic journals publish studies of scale development and validity related to a broad range of topics. For example, over the last year Psychological Assessment has published articles on measurement of mindfulness, psychopathy, sexual orientation, life satisfaction, social support, suicide risk, empathy, machiavellianism, social phobia, gambling addiction and caffeine expectancy. A lot of this scale development is done in English, and it is thus crucial that questionnaires be translated, developed and validated in Norwegian.
Measurement of thoughts, emotions and behaviour with the aid of questionnaires entails relatively simple test administration, a standardised procedure and the possibility of structured summation and presentation of results. Questionnaires are used in research that studies patient groups or population-based samples, occasionally in combination with registry or genetic data. Potentially, validated questionnaire scales also have a considerable clinical value in identifying risks and protective factors, monitoring the development of a condition or capturing the effects of an intervention. To legitimise the benefits of questionnaires, there is a need for guidelines regarding their scope of application, administration and basis for standardisation, as well as their psychometric properties. Establishing the validity of a scale is not a one-off activity, but a continuous process. Precisely for this reason, contributions to this process are highly welcome.
