Analysts' Checklist for Undertaking a Systematic Review
This checklist should only be used in conjunction with the Guidance
Notes on Systematic Review.
Analysts proposing to undertake a systematic review for the first time
are also advised to take a structured course on the topic, such as the Government
Social Research Unit course on ‘Methods for Synthesis’. For
more information on GSR courses see the GSR
website.
Formulating an Answerable Question
Does the central question of the review
clearly address the following points?
|
| |
The policy intervention for
which evidence is sought |
| |
The population or sub-groups
that the policy is expected to effect |
| |
The outcomes that the policy intervention
is expected to achieve |
| |
The context in which the policy will
operate |
Searching for Relevant Studies
| Have the following steps of a search
strategy been planned?: |
| |
The searching of appropriate electronic/internet
sources |
| |
The searching of appropriate print sources
(e.g. journals, textbooks, research reports) |
| |
The hand searching of appropriate print
sources |
| |
The searching of 'grey' (i.e. unpublished)
literature |
Critically Appraising Studies Found
Has the existing literature been sifted
for quality and validity using the criteria below?
|
| |
The appropriateness of the
questions, populations and outcomes addressed
|
| |
Evidence of selection bias in the primary
studies |
| |
Evidence of performance bias in the primary
studies |
| |
Evidence of attrition bias in the primary
studies
|
| |
Evidence of detection bias in the primary
studies
|
| |
What criteria will be used for including and excluding
primary studies |
Extracting Data from Included Studies
| Has a strategy been planned for extracting
data from the included studies that considers the issues below? |
| |
A data collection form recording how, and
why, data were extracted from included studies |
| |
Information about the characteristics
of included studies |
| |
Verification of study eligibility for
the review |
| |
Details of study characteristics |
| |
Details of study methods |
| |
Details of study participants (i.e populations
and sub-groups) |
| |
Details of study interventions |
| |
Details of study outcomes and findings |
| |
Reliability check for data collection/extraction |
Analysing and Presenting the Findings
| The analysis and presentation of the
findings of the included studies should address the questions below |
| |
What comparisons should be
made (e.g. by interventions studied, participants included, outcomes
measured)? |
| |
What study results are needed for each
comparison? |
| |
What assessments of validity are to be
used in the analysis? |
| |
Is any other/additional data or information
needed from authors of studies included in the review? |
| |
Do the data from different studies need to be transformed
for the review's analysis? |
| |
How is the heterogeneity/homogeneity
of studies to be determined? |
| |
Is a meta-analysis of findings possible? |
| |
What are the main findings of the review? |
| |
What are the likely effect sizes of the
proposed policy intervention, net of the counterfactual? |
| |
What are the main caveats and/or qualifications
of the findings of this review? |
Interpreting the Findings
| The interpretation of the findings
of the review must address the following questions |
| |
What is the strength of the evidence
from the review? |
| |
How applicable are the results of the review
to 'real life' policy and practice? |
| |
What does the review say about the costs and benefits
of the proposed intervention? |
| |
What trade-offs are suggested by the review between expected
benefits, harm and costs (including opportunity
costs)? |
| |
What mediating factors emerge from the review that
might affect the implications for policy and practice in different contexts? |
Summarising the Implications for Policy and Practice
| |
What are the 'take home' messages for policy making and/or
practice? |
| |
What are the 'take home' messages for future
research in this area? |
