Critical appraisal is the process of judging the validity and quality of a research paper.
Major points to consider when appraising systematic reviews include:
Is the study valid?
What are the results, and are they significant?
Are the results applicable?
Critical Appraisal Skills Programme (CASP)
Ten questions to help you make sense of a systematic review
AMSTAR 2
A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both
STROBE - designed for epidemiological studies
EQUATOR network listing over 300 sets of reporting guidlelines
Crowe and Sheppard (2011) summarising 44 critical appraisal tools
PRISMA checklist - not a quality instrument but helpful for identifying elements to look for in a systematic review.
SPIRIT checklist - not a quality instrument but helpful for identifying elements to look for in a systematic review.
Greenhalgh, T. (1997). How to read a paper. The Medline database. BMJ, 315(7101), 180-3
Greenhalgh, T. (1997). How to read a paper. Getting your bearings (deciding what the paper is about). BMJ, 315(7102), 243-6
Greenhalgh, T. (1997). How to read a paper: Assessing the methodological quality of published papers. BMJ, 315(7103), 305-8
Greenhalgh, T. (1997). How to read a paper. statistics for the non-statistician. i: Different types of data need different statistical tests. BMJ, 315(7104), 364-6
Greenhalgh, T. (1997). How to read a paper. statistics for the non-statistician. ii: "significant" relations and their pitfalls. BMJ, 315(7105), 422-5
Greenhalgh, T. (1997). How to read a paper. papers that report drug trials. BMJ, 315(7106), 480-3
Greenhalgh, T. (1997). How to read a paper. papers that report diagnostic or screening tests. BMJ, 315(7107), 540-3
Greenhalgh, T. (1997). How to read a paper. papers that tell you what things cost (economic analyses). BMJ, 315(7108), 596-9
Greenhalgh, T. (1997). Papers that summarise other papers (systematic reviews and meta-analyses). BMJ, 315(7109), 672-5
Greenhalgh, T, & Taylor, R. (1997). Papers that go beyond numbers (qualitative research). BMJ, 315(7110), 740-3
Data extraction is the process of transcribing information from the primary studies under review to a standard form or template that has been designed to capture all of the detail relevant to the meta-analysis.
Double data extraction is preferred for accuracy - this is when two researchers extract the data independently and discuss any discrepancies or consult a moderator to arrive at consensus.
Buscemi, N., Hartling, L., Vandermeer, B., Tjosvold, L., & Klassen, T. P. (2006). Single data extraction generated more errors than double data extraction in systematic reviews. Journal of clinical epidemiology, 59(7), 697-703. Full Text