Synthesis
Data Extraction
Data extraction is the process of transcribing information from the primary studies under review to a standard form or template that has been designed to capture all of the detail relevant to the meta-analysis.
Double data extraction is preferred for accuracy - this is when two researchers extract the data independently and discuss any discrepancies or consult a moderator to arrive at consensus.
Further Reading
Buscemi, N., Hartling, L., Vandermeer, B., Tjosvold, L., & Klassen, T. P. (2006). Single data extraction generated more errors than double data extraction in systematic reviews. Journal of clinical epidemiology, 59(7), 697-703. Full Text
Meta-analysis and Forest Plots
It's best to set up a template for extracting the data from the research papers you're critiquing. One way of doing this is to adapt EndNote fields for the raw data - so each summary is associated with both the citation record and the .pdf. You can export or copy the data to a spreadsheet.
It will be important to be able to present all of your data accurately and effectively, although it will have come from a (usually large) number of quantitative studies.
One way of doing this is with a Forest plot. A Forest plot is a graphical representation of the mean, range and variance of results in each study, all visually aligned so that the key range of values common to all studies is readily seen.
Figure 1. Forest plot embedded in Excel worksheet. Note the Confidence Intervals around the determined Standard Errors overlap, with a central tendency indicated by the diamond.
Source: Neyeloff et al (2012) UNDER Creative Commons Attribution License
Further Reading
Lewis, Steff; Clarke, Mike. Forest plots: Trying to see the wood and the trees. British Medical Journal, International edition 322.7300 (Jun 16, 2001): 1479-80. Full Text
Lewis, Steff; Clarke, Mike. Forest plots: Trying to see the wood and the trees. British Medical Journal, International edition 322.7300 (Jun 16, 2001): 1479-80. Full Text
Jeruza L Neyeloff, Sandra C Fuchs and Leila B Moreira. Meta-analyses and Forest plots using a microsoft excel spreadsheet: step-by-step guide focusing on descriptive data analysis. (2012). BMC Research Notes, 5(1), 52-57. doi:10.1186/1756-0500 Full Text
Review Matrix and Summary Tables
A review matrix and summary tables are convenient ways to summarise and quickly rearrange qualitative information and observations to achieve a draft outline.
You can easily set up a table in a word processor or spreadsheet. Aim to itemise the most important content column by column, with one article per row. The rows can be rearranged and regrouped at need or at whim, so it doesn't matter which articles you start with.
| Author/s | Publication year | Patients | Intervention & Control | Outcomes | Appraisal | Further questions |
Further Reading
Garrard, J. (2014). Health sciences literature review made easy: The matrix method, 4th ed. Burlington, MA: Jones & Bartlett Learning. Catalogue Link
Narrative and Realist Syntheses
Realist and meta-narrative syntheses, appropriately conducted, are qualitative equivalents to (quantitative) meta-analyses. They are more likely to used in reviewing health policies and guidelines rather than answering clinical questions.
Realist reviews systematically examine complex social interventions to discover and understand any factors influencing the links between an intervention and its outcomes (summed up in the question "what works, how, for whom, in what circumstances and to what extent?").
Meta-narrative reviews are similar but adapted for use when a policy-related topic has been researched in different ways by different teams, especially when differing terms and vocabularies have emerged in the literature.
The UK National Institute of Health Research has developed guidelines and standards through its RAMESES initiatives.
Realist syntheses
RAMESES publication standards: realist syntheses
BMC Medicine 2013, 11:21
http://www.biomedcentral.com/1741-7015/11/21
For a summary see http://www.ramesesproject.org/media/RS_qual_standards_researchers.pdf
Meta-narrative reviews
RAMESES publication standards: meta-narrative reviews
BMC Medicine 2013, 11:20
http://www.biomedcentral.com/1741-7015/11/20
For a summary see http://www.ramesesproject.org/media/MNR_qual_standards_researchers.pdf
Further Reading
Greenhalgh, T., Wong, G., Westhorp, G., & Pawson, R. (2011). Protocol-realist and meta-narrative evidence synthesis: evolving standards (RAMESES). BMC medical research methodology, 11(1), 115. Full Text
For information about different review types see the library guide Which Review is That?
Best-Fit Framework Synthesis
Best-fit framework synthesis is a literature review technique in two distinct stages. It's an approach suited to urgent and important policy questions, where an iterative approach to selecting search terms and refining the search strategies can give a more timely and scalable result.
In the first stage, an initial conceptual model is selected, realizing this can only be an approximation to the content and contexts of the research domain. This is used as basis for an initial search.
This framework is then modified in response to the evidence reported in the studies found. The final framework may include elements of the first model, modified factors, and new factors that were not originally anticipated.
Further Reading
Dixon-Woods, M. (2011). Using framework-based synthesis for conducting reviews of qualitative studies. BMC Medicine 9: 39. Full Text
Carroll C., Booth A., Leaviss J., & Rick J. (2013). "Best fit" framework synthesis: refining the method. BMC Medical Research Methodology 13:37. Full Text