This question was investigated by a European Research Council funded project, Getting Research into Policy in Public Health (GRIP-Health), led by Justin Parkhurst. They found that, rather than use of the ‘evidence hierarchies’ common to evidence-based medicine, is it important for policymakers to consider the appropriateness of any policy-relevant evidence.
For researchers, this means that they need to help policymakers understand, not only how their research demonstrates the effectiveness of a particular action, but also why it is appropriate to the specific policy questions being asked.
Good evidence is appropriate for the policy question
Hierarchies of evidence, which are used in evidence-based medicine, place methodologies such as randomized controlled trials at the top, often referring to them as the ‘gold standard’ of evidence. However, such hierarchies were designed to judge evidence of intervention effect within clinical medicine, not to reflect their relevance or importance for policy (including health policymaking).
For policymakers, it is instead essential to consider whether the available evidence is appropriate for the current policy concern. This can be done by asking the following questions:
Does the evidence address the multiple policy concerns at stake?
Of the entire field of evidence available, only a subset of evidence might actually be relevant to the
concerns at hand. Some evidence outside of the subset will no doubt be of high quality, but this does not necessarily mean that it is evidence which is useful for the current policy decision. So, good practice in evidence-informed policymaking must start by making relevant policy concerns explicit. Without a clear indication of these it is impossible to say whether evidence is good or not.
Are the data constructed in ways that best serve policy goals?
When reflecting on which evidence is most useful for policy purposes, it is important to recognize that there is often a choice of how to construct and classify data. For example, medical sociologists have explored how concepts like ethnicity or social class are often not adequately captured in health surveys or research, making it impossible for such factors to be the target of policy action. These insights allow us to question whether the categories and concepts used in a body of evidence are, in fact, the most useful to achieve policy goals.
Is the evidence applicable in the local policy context?
It is also necessary to consider if the results of any evidence will apply in the local context. Randomized controlled trials are designed to show that they produced an effect where they were undertaken. They do not, however, say anything about whether the result would be the same elsewhere. Showing that an intervention worked in one place does not necessarily mean that it works always and everywhere. This is especially true when dealing with social issues, as interventions often work through alternative mechanisms in differing contexts. While biomedical interventions are assumed to be generalizable due to similarities in human physiology, social interventions – such as providing cash transfers, or providing group-based health education – may work differently in different settings and may not be appropriate for the given policy needs.
Good evidence meets relevant quality standards from a scientific perspective
Once you have found the appropriate evidence for policy, that evidence needs to further be judged
in terms of its quality. Research can take many forms, and the way to judge quality will often be dependent on the type of evidence considered. The following questions will however be useful:
- Is the evidence applied with integrity to scientific principles?
- Is the evidence applied systematically to include all relevant information on an issue of concern in a consistent and up-to-date manner?
- Does the evidence use high quality methodological criteria relevant to the data type?
Find out more:
Justin O. Parkhurst & Sudeepa Abeysinghe, “What Constitutes “Good” Evidence for Public Health and Social Policy-making? From Hierarchies to Appropriateness” in Social Epistemology [14 pages] *open access article*
Justin Parkhurst, The Politics of Evidence: From evidence-based policy to the good governance of evidence [196 pages] *open access book*
What is good evidence for policy? – Chapter summary [4 pages]
You may also be interested in:
Dorie E. Apollonio & Lisa A. Bero (2016) “Challenges to generating evidence-informed policy and the role of systematic reviews and (perceived) conflicts of interest“, Journal of Communication in Healthcare
(2018) “Patient value: its nature, measurement, and role in real world evidence studies and outcomes-based reimbursement“, Journal of Medical Economics
Rita Redberg & Ryan Padrez (2014) “Better evidence for real healthcare reform“, Expert Review of Pharmacoeconomics & Outcomes Research
(2018) “Cost-effectiveness thresholds: methods for setting and examples from around the world“, Expert Review of Pharmacoeconomics & Outcomes Research
(2014) “Using economic evidence as a support tool for policy decisions: Herculean or Sisyphean effort?“, Expert Review of Pharmacoeconomics & Outcomes Research