Non-representative patient samples – in contrast to heterogeneous, representative samples – are crucial for the validity of studies.
I often read that randomised clinical studies should be based on representative samples, i.e. heterogeneous patient samples that represent the distribution of the various personal characteristics that are found in the patient population, because this will enhance their external validity (1). But for whom are these findings really valid? Representative patient samples are obviously essential if the objective of the study is to undertake opinion polls among patients or reveal the prevalence of a specific disorder. Finding answers to clinical research questions is a completely different matter. A single study cannot be used to determine whether the effect is valid for other samples and under conditions different from those that the study has investigated. For example, we cannot be certain that pre- and post-menopausal women or young and old men will derive the same effect from a proposed course of treatment. Hormonal influence, different disease histories and degrees of severity, genetic differences and varying comorbidity all have an effect. In clinical research the inclusion criteria need to be strict, to keep all these distorting factors constant. In other words, we seek to have a group with the highest possible degree of homogeneity, with high internal validity, i.e. a non-representative patient sample.
Let us assume that we wish to test a new type of disease-modifying drug among patients with ankylosing spondylitis, a chronic rheumatic disorder that typically affects the joints in the spinal column and pelvis. Let us assume that the patient group is very homogenous, for example white Norwegian menopausal women aged 50–55 who use no other drugs and were diagnosed with the disease at the same age, their disease has progressed in the same way, the concentrations of various biomarkers are the same, there are no comorbidities, they all engage in the same level of physical activity and have the same level of education. One might think that these strict inclusion criteria would weaken any findings made, because the results will not be valid for other patient groups. It is indeed a limitation that the findings will not necessarily be valid for other patient groups, but using a representative sample will not solve this problem – on the contrary. Let us say that we included men and women in all age groups, with variation in ethnicity, drug use, biomarkers, comorbidities, level of physical activity and socioeconomic status – in other words, a representative sample that represents all sub-groups in the patient population. In this case, the therapeutic effect will not necessarily be valid for the entire patient population; it will be an average effect, weighted by the therapeutic effect in the various sub-groups. As long as the sample is too small to be stratified by these factors, we have in fact no information on possible sub-groups that may respond differently to the treatment. If the objective is to test whether the disease-modifying drug is effective in other patients, the test must be undertaken on these patients specifically. It is a simplification to believe that this can be solved by a single study based on a representative patient sample.
Strict inclusion criteria, high internal validity
The objective of clinical research ought to be to produce results that we know are valid for the patients whom we are studying. By having strict inclusion criteria that keep all distorting factors constant, we can be reasonably sure that the results in fact are valid for the patient group that we are studying. Non-representative patient samples are therefore completely crucial for ensuring credible results that can be put to the best possible use in clinical practice. It is precisely the sum of high-quality studies with high internal validity that provides correct evidence about the patients in the population.
