prisma methodology for systematic review

Loading metrics

Open Access

Guidelines and Guidance

The Guidelines and Guidance section contains advice on conducting and reporting medical research.

See all article types »

The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration

* E-mail: [email protected]

Affiliations Università di Modena e Reggio Emilia, Modena, Italy, Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri, Milan, Italy

Affiliation Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom

Affiliation Ottawa Methods Centre, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada

Affiliation Annals of Internal Medicine, Philadelphia, Pennsylvania, United States of America

Affiliation The Nordic Cochrane Centre, Copenhagen, Denmark

Affiliation Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece

Affiliations UK Cochrane Centre, Oxford, United Kingdom, School of Nursing and Midwifery, Trinity College, Dublin, Ireland

Affiliation Departments of Medicine, Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada

Affiliations Kleijnen Systematic Reviews Ltd, York, United Kingdom, School for Public Health and Primary Care (CAPHRI), University of Maastricht, Maastricht, The Netherlands

Affiliations Ottawa Methods Centre, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada, Department of Epidemiology and Community Medicine, Faculty of Medicine, Ottawa, Ontario, Canada

• Alessandro Liberati, 
• Douglas G. Altman, 
• Jennifer Tetzlaff, 
• Cynthia Mulrow, 
• Peter C. Gøtzsche, 
• John P. A. Ioannidis, 
• Mike Clarke, 
• P. J. Devereaux, 
• Jos Kleijnen, 
• David Moher

Published: July 21, 2009

• https://doi.org/10.1371/journal.pmed.1000100
• Reader Comments

Systematic reviews and meta-analyses are essential to summarize evidence relating to efficacy and safety of health care interventions accurately and reliably. The clarity and transparency of these reports, however, is not optimal. Poor reporting of systematic reviews diminishes their value to clinicians, policy makers, and other users.

Since the development of the QUOROM ( QU ality O f R eporting O f M eta-analysis) Statement—a reporting guideline published in 1999—there have been several conceptual, methodological, and practical advances regarding the conduct and reporting of systematic reviews and meta-analyses. Also, reviews of published systematic reviews have found that key information about these studies is often poorly reported. Realizing these issues, an international group that included experienced authors and methodologists developed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) as an evolution of the original QUOROM guideline for systematic reviews and meta-analyses of evaluations of health care interventions.

The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram. The checklist includes items deemed essential for transparent reporting of a systematic review. In this Explanation and Elaboration document, we explain the meaning and rationale for each checklist item. For each item, we include an example of good reporting and, where possible, references to relevant empirical studies and methodological literature. The PRISMA Statement, this document, and the associated Web site ( http://www.prisma-statement.org/ ) should be helpful resources to improve reporting of systematic reviews and meta-analyses.

Citation: Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. (2009) The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med 6(7): e1000100. https://doi.org/10.1371/journal.pmed.1000100

Copyright: © 2009 Liberati et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: PRISMA was funded by the Canadian Institutes of Health Research; Università di Modena e Reggio Emilia, Italy; Cancer Research UK; Clinical Evidence BMJ Knowledge; The Cochrane Collaboration; and GlaxoSmithKline, Canada. AL is funded, in part, through grants of the Italian Ministry of University (COFIN - PRIN 2002 prot. 2002061749 and COFIN - PRIN 2006 prot. 2006062298). DGA is funded by Cancer Research UK. DM is funded by a University of Ottawa Research Chair. None of the sponsors had any involvement in the planning, execution, or write-up of the PRISMA documents. Additionally, no funder played a role in drafting the manuscript.

Competing interests: MC's employment is as Director of the UK Cochrane Centre. He is employed by the Oxford Radcliffe Hospitals Trust on behalf of the Department of Health and the National Institute for Health Research in England. This is a fixed term contract, the renewal of which is dependent upon the value placed upon his work, that of the UK Cochrane Centre, and of The Cochrane Collaboration more widely by the Department of Health. His work involves the conduct of systematic reviews and the support of the conduct and use of systematic reviews. Therefore, work–such as this manuscript–relating to systematic reviews might have an impact on his employment.

Abbreviations: PICOS, participants, interventions, comparators, outcomes, and study design; PRISMA, Preferred Reporting Items for Systematic reviews and Meta-Analyses; QUOROM, QU ality O f R eporting O f M eta-analyses

Provenance: Not commissioned; externally peer reviewed. In order to encourage dissemination of the PRISMA explanatory paper, this article is freely accessible on the PLoS Medicine , Annals of Internal Medicine , and BMJ Web sites. The authors jointly hold the copyright of this article. For details on further use see the PRISMA Web site ( http://www.prisma-statement.org/ ).

Introduction

Systematic reviews and meta-analyses are essential tools for summarizing evidence accurately and reliably. They help clinicians keep up-to-date; provide evidence for policy makers to judge risks, benefits, and harms of health care behaviors and interventions; gather together and summarize related research for patients and their carers; provide a starting point for clinical practice guideline developers; provide summaries of previous research for funders wishing to support new research [1] ; and help editors judge the merits of publishing reports of new studies [2] . Recent data suggest that at least 2,500 new systematic reviews reported in English are indexed in MEDLINE annually [3] .

Unfortunately, there is considerable evidence that key information is often poorly reported in systematic reviews, thus diminishing their potential usefulness [3] , [4] , [5] , [6] . As is true for all research, systematic reviews should be reported fully and transparently to allow readers to assess the strengths and weaknesses of the investigation [7] . That rationale led to the development of the QUOROM ( QU ality O f R eporting O f M eta-analyses) Statement; those detailed reporting recommendations were published in 1999 [8] . In this paper we describe the updating of that guidance. Our aim is to ensure clear presentation of what was planned, done, and found in a systematic review.

Terminology used to describe systematic reviews and meta-analyses has evolved over time and varies across different groups of researchers and authors (see Box 1 ). In this document we adopt the definitions used by the Cochrane Collaboration [9] . A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods that are selected to minimize bias, thus providing reliable findings from which conclusions can be drawn and decisions made. Meta-analysis is the use of statistical methods to summarize and combine the results of independent studies. Many systematic reviews contain meta-analyses, but not all.

Box 1. Terminology

The terminology used to describe systematic reviews and meta-analyses has evolved over time and varies between fields. Different terms have been used by different groups, such as educators and psychologists. The conduct of a systematic review comprises several explicit and reproducible steps, such as identifying all likely relevant records, selecting eligible studies, assessing the risk of bias, extracting data, qualitative synthesis of the included studies, and possibly meta-analyses.

Initially this entire process was termed a meta-analysis and was so defined in the QUOROM Statement [8] . More recently, especially in health care research, there has been a trend towards preferring the term systematic review. If quantitative synthesis is performed, this last stage alone is referred to as a meta-analysis. The Cochrane Collaboration uses this terminology [9] , under which a meta-analysis, if performed, is a component of a systematic review. Regardless of the question addressed and the complexities involved, it is always possible to complete a systematic review of existing data, but not always possible, or desirable, to quantitatively synthesize results, due to clinical, methodological, or statistical differences across the included studies. Conversely, with prospective accumulation of studies and datasets where the plan is eventually to combine them, the term “(prospective) meta-analysis” may make more sense than “systematic review.”

For retrospective efforts, one possibility is to use the term systematic review for the whole process up to the point when one decides whether to perform a quantitative synthesis. If a quantitative synthesis is performed, some researchers refer to this as a meta-analysis. This definition is similar to that found in the current edition of the Dictionary of Epidemiology [183] .

While we recognize that the use of these terms is inconsistent and there is residual disagreement among the members of the panel working on PRISMA, we have adopted the definitions used by the Cochrane Collaboration [9] .

Systematic review: A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing reliable findings from which conclusions can be drawn and decisions made [184] , [185] . The key characteristics of a systematic review are: (a) a clearly stated set of objectives with an explicit, reproducible methodology; (b) a systematic search that attempts to identify all studies that would meet the eligibility criteria; (c) an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and (d) systematic presentation, and synthesis, of the characteristics and findings of the included studies.

Meta-analysis: Meta-analysis is the use of statistical techniques to integrate and summarize the results of included studies. Many systematic reviews contain meta-analyses, but not all. By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.

The QUOROM Statement and Its Evolution into PRISMA

The QUOROM Statement, developed in 1996 and published in 1999 [8] , was conceived as a reporting guidance for authors reporting a meta-analysis of randomized trials. Since then, much has happened. First, knowledge about the conduct and reporting of systematic reviews has expanded considerably. For example, The Cochrane Library's Methodology Register (which includes reports of studies relevant to the methods for systematic reviews) now contains more than 11,000 entries (March 2009). Second, there have been many conceptual advances, such as “outcome-level” assessments of the risk of bias [10] , [11] , that apply to systematic reviews. Third, authors have increasingly used systematic reviews to summarize evidence other than that provided by randomized trials.

However, despite advances, the quality of the conduct and reporting of systematic reviews remains well short of ideal [3] , [4] , [5] , [6] . All of these issues prompted the need for an update and expansion of the QUOROM Statement. Of note, recognizing that the updated statement now addresses the above conceptual and methodological issues and may also have broader applicability than the original QUOROM Statement, we changed the name of the reporting guidance to PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses).

Development of PRISMA

The PRISMA Statement was developed by a group of 29 review authors, methodologists, clinicians, medical editors, and consumers [12] . They attended a three-day meeting in 2005 and participated in extensive post-meeting electronic correspondence. A consensus process that was informed by evidence, whenever possible, was used to develop a 27-item checklist ( Table 1 ; see also Text S1 for a downloadable template checklist for researchers to re-use) and a four-phase flow diagram ( Figure 1 ; see Figure S1 for a downloadable template document for researchers to re-use). Items deemed essential for transparent reporting of a systematic review were included in the checklist. The flow diagram originally proposed by QUOROM was also modified to show numbers of identified records, excluded articles, and included studies. After 11 revisions the group approved the checklist, flow diagram, and this explanatory paper.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pmed.1000100.g001

The PRISMA Statement itself provides further details regarding its background and development [12] . This accompanying Explanation and Elaboration document explains the meaning and rationale for each checklist item. A few PRISMA Group participants volunteered to help draft specific items for this document, and four of these (DGA, AL, DM, and JT) met on several occasions to further refine the document, which was circulated and ultimately approved by the larger PRISMA Group.

https://doi.org/10.1371/journal.pmed.1000100.t001

Scope of PRISMA

PRISMA focuses on ways in which authors can ensure the transparent and complete reporting of systematic reviews and meta-analyses. It does not address directly or in a detailed manner the conduct of systematic reviews, for which other guides are available [13] , [14] , [15] , [16] .

We developed the PRISMA Statement and this explanatory document to help authors report a wide array of systematic reviews to assess the benefits and harms of a health care intervention. We consider most of the checklist items relevant when reporting systematic reviews of non-randomized studies assessing the benefits and harms of interventions. However, we recognize that authors who address questions relating to etiology, diagnosis, or prognosis, for example, and who review epidemiological or diagnostic accuracy studies may need to modify or incorporate additional items for their systematic reviews.

How To Use This Paper

We modeled this Explanation and Elaboration document after those prepared for other reporting guidelines [17] , [18] , [19] . To maximize the benefit of this document, we encourage people to read it in conjunction with the PRISMA Statement [11] .

We present each checklist item and follow it with a published exemplar of good reporting for that item. (We edited some examples by removing citations or Web addresses, or by spelling out abbreviations.) We then explain the pertinent issue, the rationale for including the item, and relevant evidence from the literature, whenever possible. No systematic search was carried out to identify exemplars and evidence. We also include seven Boxes that provide a more comprehensive explanation of certain thematic aspects of the methodology and conduct of systematic reviews.

Although we focus on a minimal list of items to consider when reporting a systematic review, we indicate places where additional information is desirable to improve transparency of the review process. We present the items numerically from 1 to 27; however, authors need not address items in this particular order in their reports. Rather, what is important is that the information for each item is given somewhere within the report.

The PRISMA Checklist

Title and abstract, item 1: title..

Identify the report as a systematic review, meta-analysis, or both.

Examples. “Recurrence rates of video-assisted thoracoscopic versus open surgery in the prevention of recurrent pneumothoraces: a systematic review of randomised and non-randomised trials” [20] “Mortality in randomized trials of antioxidant supplements for primary and secondary prevention: systematic review and meta-analysis” [21]

Explanation.

Authors should identify their report as a systematic review or meta-analysis. Terms such as “review” or “overview” do not describe for readers whether the review was systematic or whether a meta-analysis was performed. A recent survey found that 50% of 300 authors did not mention the terms “systematic review” or “meta-analysis” in the title or abstract of their systematic review [3] . Although sensitive search strategies have been developed to identify systematic reviews [22] , inclusion of the terms systematic review or meta-analysis in the title may improve indexing and identification.

We advise authors to use informative titles that make key information easily accessible to readers. Ideally, a title reflecting the PICOS approach (participants, interventions, comparators, outcomes, and study design) (see Item 11 and Box 2 ) may help readers as it provides key information about the scope of the review. Specifying the design(s) of the studies included, as shown in the examples, may also help some readers and those searching databases.

Box 2. Helping To Develop the Research Question(s): The PICOS Approach

Formulating relevant and precise questions that can be answered in a systematic review can be complex and time consuming. A structured approach for framing questions that uses five components may help facilitate the process. This approach is commonly known by the acronym “PICOS” where each letter refers to a component: the patient population or the disease being addressed (P), the interventions or exposure (I), the comparator group (C), the outcome or endpoint (O), and the study design chosen (S) [186] . Issues relating to PICOS impact several PRISMA items (i.e., Items 6, 8, 9, 10, 11, and 18).

Providing information about the population requires a precise definition of a group of participants (often patients), such as men over the age of 65 years, their defining characteristics of interest (often disease), and possibly the setting of care considered, such as an acute care hospital.

The interventions (exposures) under consideration in the systematic review need to be transparently reported. For example, if the reviewers answer a question regarding the association between a woman's prenatal exposure to folic acid and subsequent offspring's neural tube defects, reporting the dose, frequency, and duration of folic acid used in different studies is likely to be important for readers to interpret the review's results and conclusions. Other interventions (exposures) might include diagnostic, preventative, or therapeutic treatments, arrangements of specific processes of care, lifestyle changes, psychosocial or educational interventions, or risk factors.

Clearly reporting the comparator (control) group intervention(s), such as usual care, drug, or placebo, is essential for readers to fully understand the selection criteria of primary studies included in systematic reviews, and might be a source of heterogeneity investigators have to deal with. Comparators are often very poorly described. Clearly reporting what the intervention is compared with is very important and may sometimes have implications for the inclusion of studies in a review—many reviews compare with “standard care,” which is otherwise undefined; this should be properly addressed by authors.

The outcomes of the intervention being assessed, such as mortality, morbidity, symptoms, or quality of life improvements, should be clearly specified as they are required to interpret the validity and generalizability of the systematic review's results.

Finally, the type of study design(s) included in the review should be reported. Some reviews only include reports of randomized trials whereas others have broader design criteria and include randomized trials and certain types of observational studies. Still other reviews, such as those specifically answering questions related to harms, may include a wide variety of designs ranging from cohort studies to case reports. Whatever study designs are included in the review, these should be reported.

Independently from how difficult it is to identify the components of the research question, the important point is that a structured approach is preferable, and this extends beyond systematic reviews of effectiveness. Ideally the PICOS criteria should be formulated a priori, in the systematic review's protocol, although some revisions might be required due to the iterative nature of the review process. Authors are encouraged to report their PICOS criteria and whether any modifications were made during the review process. A useful example in this realm is the Appendix of the “Systematic Reviews of Water Fluoridation” undertaken by the Centre for Reviews and Dissemination [187] .

Some journals recommend “indicative titles” that indicate the topic matter of the review, while others require declarative titles that give the review's main conclusion. Busy practitioners may prefer to see the conclusion of the review in the title, but declarative titles can oversimplify or exaggerate findings. Thus, many journals and methodologists prefer indicative titles as used in the examples above.

Item 2: STRUCTURED SUMMARY.

Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; funding for the systematic review; and systematic review registration number.

Example. “ Context : The role and dose of oral vitamin D supplementation in nonvertebral fracture prevention have not been well established. Objective : To estimate the effectiveness of vitamin D supplementation in preventing hip and nonvertebral fractures in older persons. Data Sources : A systematic review of English and non-English articles using MEDLINE and the Cochrane Controlled Trials Register (1960–2005), and EMBASE (1991–2005). Additional studies were identified by contacting clinical experts and searching bibliographies and abstracts presented at the American Society for Bone and Mineral Research (1995–2004). Search terms included randomized controlled trial (RCT), controlled clinical trial, random allocation, double-blind method, cholecalciferol, ergocalciferol, 25-hydroxyvitamin D, fractures, humans, elderly, falls, and bone density. Study Selection : Only double-blind RCTs of oral vitamin D supplementation (cholecalciferol, ergocalciferol) with or without calcium supplementation vs calcium supplementation or placebo in older persons (>60 years) that examined hip or nonvertebral fractures were included. Data Extraction : Independent extraction of articles by 2 authors using predefined data fields, including study quality indicators. Data Synthesis : All pooled analyses were based on random-effects models. Five RCTs for hip fracture (n = 9294) and 7 RCTs for nonvertebral fracture risk (n = 9820) met our inclusion criteria. All trials used cholecalciferol. Heterogeneity among studies for both hip and nonvertebral fracture prevention was observed, which disappeared after pooling RCTs with low-dose (400 IU/d) and higher-dose vitamin D (700–800 IU/d), separately. A vitamin D dose of 700 to 800 IU/d reduced the relative risk (RR) of hip fracture by 26% (3 RCTs with 5572 persons; pooled RR, 0.74; 95% confidence interval [CI], 0.61–0.88) and any nonvertebral fracture by 23% (5 RCTs with 6098 persons; pooled RR, 0.77; 95% CI, 0.68–0.87) vs calcium or placebo. No significant benefit was observed for RCTs with 400 IU/d vitamin D (2 RCTs with 3722 persons; pooled RR for hip fracture, 1.15; 95% CI, 0.88–1.50; and pooled RR for any nonvertebral fracture, 1.03; 95% CI, 0.86–1.24). Conclusions : Oral vitamin D supplementation between 700 to 800 IU/d appears to reduce the risk of hip and any nonvertebral fractures in ambulatory or institutionalized elderly persons. An oral vitamin D dose of 400 IU/d is not sufficient for fracture prevention.” [23]

Abstracts provide key information that enables readers to understand the scope, processes, and findings of a review and to decide whether to read the full report. The abstract may be all that is readily available to a reader, for example, in a bibliographic database. The abstract should present a balanced and realistic assessment of the review's findings that mirrors, albeit briefly, the main text of the report.

We agree with others that the quality of reporting in abstracts presented at conferences and in journal publications needs improvement [24] , [25] . While we do not uniformly favor a specific format over another, we generally recommend structured abstracts. Structured abstracts provide readers with a series of headings pertaining to the purpose, conduct, findings, and conclusions of the systematic review being reported [26] , [27] . They give readers more complete information and facilitate finding information more easily than unstructured abstracts [28] , [29] , [30] , [31] , [32] .

A highly structured abstract of a systematic review could include the following headings: Context (or Background); Objective (or Purpose); Data Sources; Study Selection (or Eligibility Criteria); Study Appraisal and Synthesis Methods (or Data Extraction and Data Synthesis); Results; Limitations; and Conclusions (or Implications). Alternatively, a simpler structure could cover but collapse some of the above headings (e.g., label Study Selection and Study Appraisal as Review Methods) or omit some headings such as Background and Limitations.

In the highly structured abstract mentioned above, authors use the Background heading to set the context for readers and explain the importance of the review question. Under the Objectives heading, they ideally use elements of PICOS (see Box 2 ) to state the primary objective of the review. Under a Data Sources heading, they summarize sources that were searched, any language or publication type restrictions, and the start and end dates of searches. Study Selection statements then ideally describe who selected studies using what inclusion criteria. Data Extraction Methods statements describe appraisal methods during data abstraction and the methods used to integrate or summarize the data. The Data Synthesis section is where the main results of the review are reported. If the review includes meta-analyses, authors should provide numerical results with confidence intervals for the most important outcomes. Ideally, they should specify the amount of evidence in these analyses (numbers of studies and numbers of participants). Under a Limitations heading, authors might describe the most important weaknesses of included studies as well as limitations of the review process. Then authors should provide clear and balanced Conclusions that are closely linked to the objective and findings of the review. Additionally, it would be helpful if authors included some information about funding for the review. Finally, although protocol registration for systematic reviews is still not common practice, if authors have registered their review or received a registration number, we recommend providing the registration information at the end of the abstract.

Taking all the above considerations into account, the intrinsic tension between the goal of completeness of the abstract and its keeping into the space limit often set by journal editors is recognized as a major challenge.

INTRODUCTION

Item 3: rationale..

Describe the rationale for the review in the context of what is already known.

Example. “Reversing the trend of increasing weight for height in children has proven difficult. It is widely accepted that increasing energy expenditure and reducing energy intake form the theoretical basis for management. Therefore, interventions aiming to increase physical activity and improve diet are the foundation of efforts to prevent and treat childhood obesity. Such lifestyle interventions have been supported by recent systematic reviews, as well as by the Canadian Paediatric Society, the Royal College of Paediatrics and Child Health, and the American Academy of Pediatrics. However, these interventions are fraught with poor adherence. Thus, school-based interventions are theoretically appealing because adherence with interventions can be improved. Consequently, many local governments have enacted or are considering policies that mandate increased physical activity in schools, although the effect of such interventions on body composition has not been assessed.” [33]

Readers need to understand the rationale behind the study and what the systematic review may add to what is already known. Authors should tell readers whether their report is a new systematic review or an update of an existing one. If the review is an update, authors should state reasons for the update, including what has been added to the evidence base since the previous version of the review.

An ideal background or introduction that sets context for readers might include the following. First, authors might define the importance of the review question from different perspectives (e.g., public health, individual patient, or health policy). Second, authors might briefly mention the current state of knowledge and its limitations. As in the above example, information about the effects of several different interventions may be available that helps readers understand why potential relative benefits or harms of particular interventions need review. Third, authors might whet readers' appetites by clearly stating what the review aims to add. They also could discuss the extent to which the limitations of the existing evidence base may be overcome by the review.

Item 4: OBJECTIVES.

Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).

Example. “To examine whether topical or intraluminal antibiotics reduce catheter-related bloodstream infection, we reviewed randomized, controlled trials that assessed the efficacy of these antibiotics for primary prophylaxis against catheter-related bloodstream infection and mortality compared with no antibiotic therapy in adults undergoing hemodialysis.” [34]

The questions being addressed, and the rationale for them, are one of the most critical parts of a systematic review. They should be stated precisely and explicitly so that readers can understand quickly the review's scope and the potential applicability of the review to their interests [35] . Framing questions so that they include the following five “PICOS” components may improve the explicitness of review questions: (1) the patient population or disease being addressed (P), (2) the interventions or exposure of interest (I), (3) the comparators (C), (4) the main outcome or endpoint of interest (O), and (5) the study designs chosen (S). For more detail regarding PICOS, see Box 2 .

Good review questions may be narrowly focused or broad, depending on the overall objectives of the review. Sometimes broad questions might increase the applicability of the results and facilitate detection of bias, exploratory analyses, and sensitivity analyses [35] , [36] . Whether narrowly focused or broad, precisely stated review objectives are critical as they help define other components of the review process such as the eligibility criteria (Item 6) and the search for relevant literature (Items 7 and 8).

Item 5: PROTOCOL AND REGISTRATION.

Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address) and, if available, provide registration information including the registration number.

Example. “Methods of the analysis and inclusion criteria were specified in advance and documented in a protocol.” [37]

A protocol is important because it pre-specifies the objectives and methods of the systematic review. For instance, a protocol specifies outcomes of primary interest, how reviewers will extract information about those outcomes, and methods that reviewers might use to quantitatively summarize the outcome data (see Item 13). Having a protocol can help restrict the likelihood of biased post hoc decisions in review methods, such as selective outcome reporting. Several sources provide guidance about elements to include in the protocol for a systematic review [16] , [38] , [39] . For meta-analyses of individual patient-level data, we advise authors to describe whether a protocol was explicitly designed and whether, when, and how participating collaborators endorsed it [40] , [41] .

Authors may modify protocols during the research, and readers should not automatically consider such modifications inappropriate. For example, legitimate modifications may extend the period of searches to include older or newer studies, broaden eligibility criteria that proved too narrow, or add analyses if the primary analyses suggest that additional ones are warranted. Authors should, however, describe the modifications and explain their rationale.

Although worthwhile protocol amendments are common, one must consider the effects that protocol modifications may have on the results of a systematic review, especially if the primary outcome is changed. Bias from selective outcome reporting in randomized trials has been well documented [42] , [43] . An examination of 47 Cochrane reviews revealed indirect evidence for possible selective reporting bias for systematic reviews. Almost all ( n  = 43) contained a major change, such as the addition or deletion of outcomes, between the protocol and the full publication [44] . Whether (or to what extent) the changes reflected bias, however, was not clear. For example, it has been rather common not to describe outcomes that were not presented in any of the included studies.

Registration of a systematic review, typically with a protocol and registration number, is not yet common, but some opportunities exist [45] , [46] . Registration may possibly reduce the risk of multiple reviews addressing the same question [45] , [46] , [47] , [48] , reduce publication bias, and provide greater transparency when updating systematic reviews. Of note, a survey of systematic reviews indexed in MEDLINE in November 2004 found that reports of protocol use had increased to about 46% [3] from 8% noted in previous surveys [49] . The improvement was due mostly to Cochrane reviews, which, by requirement, have a published protocol [3] .

Item 6: ELIGIBILITY CRITERIA.

Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale.

Examples. Types of studies : “Randomised clinical trials studying the administration of hepatitis B vaccine to CRF [chronic renal failure] patients, with or without dialysis. No language, publication date, or publication status restrictions were imposed…” Types of participants : “Participants of any age with CRF or receiving dialysis (haemodialysis or peritoneal dialysis) were considered. CRF was defined as serum creatinine greater than 200 µmol/L for a period of more than six months or individuals receiving dialysis (haemodialysis or peritoneal dialysis)…Renal transplant patients were excluded from this review as these individuals are immunosuppressed and are receiving immunosuppressant agents to prevent rejection of their transplanted organs, and they have essentially normal renal function…” Types of intervention : “Trials comparing the beneficial and harmful effects of hepatitis B vaccines with adjuvant or cytokine co-interventions [and] trials comparing the beneficial and harmful effects of immunoglobulin prophylaxis. This review was limited to studies looking at active immunization. Hepatitis B vaccines (plasma or recombinant (yeast) derived) of all types, dose, and regimens versus placebo, control vaccine, or no vaccine…” Types of outcome measures : “Primary outcome measures: Seroconversion, ie, proportion of patients with adequate anti-HBs response (>10 IU/L or Sample Ratio Units). Hepatitis B infections (as measured by hepatitis B core antigen (HBcAg) positivity or persistent HBsAg positivity), both acute and chronic. Acute (primary) HBV [hepatitis B virus] infections were defined as seroconversion to HBsAg positivity or development of IgM anti-HBc. Chronic HBV infections were defined as the persistence of HBsAg for more than six months or HBsAg positivity and liver biopsy compatible with a diagnosis or chronic hepatitis B. Secondary outcome measures: Adverse events of hepatitis B vaccinations…[and]…mortality.” [50]

Knowledge of the eligibility criteria is essential in appraising the validity, applicability, and comprehensiveness of a review. Thus, authors should unambiguously specify eligibility criteria used in the review. Carefully defined eligibility criteria inform various steps of the review methodology. They influence the development of the search strategy and serve to ensure that studies are selected in a systematic and unbiased manner.

A study may be described in multiple reports, and one report may describe multiple studies. Therefore, we separate eligibility criteria into the following two components: study characteristics and report characteristics. Both need to be reported. Study eligibility criteria are likely to include the populations, interventions, comparators, outcomes, and study designs of interest (PICOS; see Box 2 ), as well as other study-specific elements, such as specifying a minimum length of follow-up. Authors should state whether studies will be excluded because they do not include (or report) specific outcomes to help readers ascertain whether the systematic review may be biased as a consequence of selective reporting [42] , [43] .

Report eligibility criteria are likely to include language of publication, publication status (e.g., inclusion of unpublished material and abstracts), and year of publication. Inclusion or not of non-English language literature [51] , [52] , [53] , [54] , [55] , unpublished data, or older data can influence the effect estimates in meta-analyses [56] , [57] , [58] , [59] . Caution may need to be exercised in including all identified studies due to potential differences in the risk of bias such as, for example, selective reporting in abstracts [60] , [61] , [62] .

Item 7: INFORMATION SOURCES.

Describe all information sources in the search (e.g., databases with dates of coverage, contact with study authors to identify additional studies) and date last searched.

Example. “Studies were identified by searching electronic databases, scanning reference lists of articles and consultation with experts in the field and drug companies…No limits were applied for language and foreign papers were translated. This search was applied to Medline (1966–Present), CancerLit (1975–Present), and adapted for Embase (1980–Present), Science Citation Index Expanded (1981–Present) and Pre-Medline electronic databases. Cochrane and DARE (Database of Abstracts of Reviews of Effectiveness) databases were reviewed…The last search was run on 19 June 2001. In addition, we handsearched contents pages of Journal of Clinical Oncology 2001, European Journal of Cancer 2001 and Bone 2001, together with abstracts printed in these journals 1999–2001. A limited update literature search was performed from 19 June 2001 to 31 December 2003.” [63]

The National Library of Medicine's MEDLINE database is one of the most comprehensive sources of health care information in the world. Like any database, however, its coverage is not complete and varies according to the field. Retrieval from any single database, even by an experienced searcher, may be imperfect, which is why detailed reporting is important within the systematic review.

At a minimum, for each database searched, authors should report the database, platform, or provider (e.g., Ovid, Dialog, PubMed) and the start and end dates for the search of each database. This information lets readers assess the currency of the review, which is important because the publication time-lag outdates the results of some reviews [64] . This information should also make updating more efficient [65] . Authors should also report who developed and conducted the search [66] .

In addition to searching databases, authors should report the use of supplementary approaches to identify studies, such as hand searching of journals, checking reference lists, searching trials registries or regulatory agency Web sites [67] , contacting manufacturers, or contacting authors. Authors should also report if they attempted to acquire any missing information (e.g., on study methods or results) from investigators or sponsors; it is useful to describe briefly who was contacted and what unpublished information was obtained.

Item 8: SEARCH.

Present the full electronic search strategy for at least one major database, including any limits used, such that it could be repeated.

Examples. In text : “We used the following search terms to search all trials registers and databases: immunoglobulin*; IVIG; sepsis; septic shock; septicaemia; and septicemia…” [68] In appendix : “Search strategy: MEDLINE (OVID) 01. immunoglobulins/ 02. immunoglobulin$.tw. 03. ivig.tw. 04. 1 or 2 or 3 05. sepsis/ 06. sepsis.tw. 07. septic shock/ 08. septic shock.tw. 09. septicemia/ 10. septicaemia.tw. 11. septicemia.tw. 12. 5 or 6 or 7 or 8 or 9 or 10 or 11 13. 4 and 12 14. randomized controlled trials/ 15. randomized-controlled-trial.pt. 16. controlled-clinical-trial.pt. 17. random allocation/ 18. double-blind method/ 19. single-blind method/ 20. 14 or 15 or 16 or 17 or 18 or 19 21. exp clinical trials/ 22. clinical-trial.pt. 23. (clin$ adj trial$).ti,ab. 24. ((singl$ or doubl$ or trebl$ or tripl$) adj (blind$)).ti,ab. 25. placebos/ 26. placebo$.ti,ab. 27. random$.ti,ab. 28. 21 or 22 or 23 or 24 or 25 or 26 or 27 29. research design/ 30. comparative study/ 31. exp evaluation studies/ 32. follow-up studies/ 33. prospective studies/ 34. (control$ or prospective$ or volunteer$).ti,ab. 35. 30 or 31 or 32 or 33 or 34 36. 20 or 28 or 29 or 35 37. 13 and 36” [68]

The search strategy is an essential part of the report of any systematic review. Searches may be complicated and iterative, particularly when reviewers search unfamiliar databases or their review is addressing a broad or new topic. Perusing the search strategy allows interested readers to assess the comprehensiveness and completeness of the search, and to replicate it. Thus, we advise authors to report their full electronic search strategy for at least one major database. As an alternative to presenting search strategies for all databases, authors could indicate how the search took into account other databases searched, as index terms vary across databases. If different searches are used for different parts of a wider question (e.g., questions relating to benefits and questions relating to harms), we recommend authors provide at least one example of a strategy for each part of the objective [69] . We also encourage authors to state whether search strategies were peer reviewed as part of the systematic review process [70] .

We realize that journal restrictions vary and that having the search strategy in the text of the report is not always feasible. We strongly encourage all journals, however, to find ways, such as a “Web extra,” appendix, or electronic link to an archive, to make search strategies accessible to readers. We also advise all authors to archive their searches so that (1) others may access and review them (e.g., replicate them or understand why their review of a similar topic did not identify the same reports), and (2) future updates of their review are facilitated.

Several sources provide guidance on developing search strategies [71] , [72] , [73] . Most searches have constraints, for example relating to limited time or financial resources, inaccessible or inadequately indexed reports and databases, unavailability of experts with particular language or database searching skills, or review questions for which pertinent evidence is not easy to find. Authors should be straightforward in describing their search constraints. Apart from the keywords used to identify or exclude records, they should report any additional limitations relevant to the search, such as language and date restrictions (see also eligibility criteria, Item 6) [51] .

Item 9: STUDY SELECTION.

State the process for selecting studies (i.e., for screening, for determining eligibility, for inclusion in the systematic review, and, if applicable, for inclusion in the meta-analysis).

Example. “Eligibility assessment…[was] performed independently in an unblinded standardized manner by 2 reviewers…Disagreements between reviewers were resolved by consensus.” [74]

There is no standard process for selecting studies to include in a systematic review. Authors usually start with a large number of identified records from their search and sequentially exclude records according to eligibility criteria. We advise authors to report how they screened the retrieved records (typically a title and abstract), how often it was necessary to review the full text publication, and if any types of record (e.g., letters to the editor) were excluded. We also advise using the PRISMA flow diagram to summarize study selection processes (see Item 17; Box 3 ).

Box 3. Identification of Study Reports and Data Extraction

Comprehensive searches usually result in a large number of identified records, a much smaller number of studies included in the systematic review, and even fewer of these studies included in any meta-analyses. Reports of systematic reviews often provide little detail as to the methods used by the review team in this process. Readers are often left with what can be described as the “X-files” phenomenon, as it is unclear what occurs between the initial set of identified records and those finally included in the review.

Sometimes, review authors simply report the number of included studies; more often they report the initial number of identified records and the number of included studies. Rarely, although this is optimal for readers, do review authors report the number of identified records, the smaller number of potentially relevant studies, and the even smaller number of included studies, by outcome. Review authors also need to differentiate between the number of reports and studies. Often there will not be a 1∶1 ratio of reports to studies and this information needs to be described in the systematic review report.

Ideally, the identification of study reports should be reported as text in combination with use of the PRISMA flow diagram. While we recommend use of the flow diagram, a small number of reviews might be particularly simple and can be sufficiently described with a few brief sentences of text. More generally, review authors will need to report the process used for each step: screening the identified records; examining the full text of potentially relevant studies (and reporting the number that could not be obtained); and applying eligibility criteria to select the included studies.

Such descriptions should also detail how potentially eligible records were promoted to the next stage of the review (e.g., full text screening) and to the final stage of this process, the included studies. Often review teams have three response options for excluding records or promoting them to the next stage of the winnowing process: “yes,” “no,” and “maybe.”

Similarly, some detail should be reported on who participated and how such processes were completed. For example, a single person may screen the identified records while a second person independently examines a small sample of them. The entire winnowing process is one of “good book keeping” whereby interested readers should be able to work backwards from the included studies to come up with the same numbers of identified records.

There is often a paucity of information describing the data extraction processes in reports of systematic reviews. Authors may simply report that “relevant” data were extracted from each included study with little information about the processes used for data extraction. It may be useful for readers to know whether a systematic review's authors developed, a priori or not, a data extraction form, whether multiple forms were used, the number of questions, whether the form was pilot tested, and who completed the extraction. For example, it is important for readers to know whether one or more people extracted data, and if so, whether this was completed independently, whether “consensus” data were used in the analyses, and if the review team completed an informal training exercise or a more formal reliability exercise.

Efforts to enhance objectivity and avoid mistakes in study selection are important. Thus authors should report whether each stage was carried out by one or several people, who these people were, and, whenever multiple independent investigators performed the selection, what the process was for resolving disagreements. The use of at least two investigators may reduce the possibility of rejecting relevant reports [75] . The benefit may be greatest for topics where selection or rejection of an article requires difficult judgments [76] . For these topics, authors should ideally tell readers the level of inter-rater agreement, how commonly arbitration about selection was required, and what efforts were made to resolve disagreements (e.g., by contact with the authors of the original studies).

Item 10: DATA COLLECTION PROCESS.

Describe the method of data extraction from reports (e.g., piloted forms, independently by two reviewers) and any processes for obtaining and confirming data from investigators.

Example. “We developed a data extraction sheet (based on the Cochrane Consumers and Communication Review Group's data extraction template), pilot-tested it on ten randomly-selected included studies, and refined it accordingly. One review author extracted the following data from included studies and the second author checked the extracted data…Disagreements were resolved by discussion between the two review authors; if no agreement could be reached, it was planned a third author would decide. We contacted five authors for further information. All responded and one provided numerical data that had only been presented graphically in the published paper.” [77]

Reviewers extract information from each included study so that they can critique, present, and summarize evidence in a systematic review. They might also contact authors of included studies for information that has not been, or is unclearly, reported. In meta-analysis of individual patient data, this phase involves collection and scrutiny of detailed raw databases. The authors should describe these methods, including any steps taken to reduce bias and mistakes during data collection and data extraction [78] ( Box 3 ).

Some systematic reviewers use a data extraction form that could be reported as an appendix or “Web extra” to their report. These forms could show the reader what information reviewers sought (see Item 11) and how they extracted it. Authors could tell readers if the form was piloted. Regardless, we advise authors to tell readers who extracted what data, whether any extractions were completed in duplicate, and, if so, whether duplicate abstraction was done independently and how disagreements were resolved.

Published reports of the included studies may not provide all the information required for the review. Reviewers should describe any actions they took to seek additional information from the original researchers (see Item 7). The description might include how they attempted to contact researchers, what they asked for, and their success in obtaining the necessary information. Authors should also tell readers when individual patient data were sought from the original researchers [41] (see Item 11) and indicate the studies for which such data were used in the analyses. The reviewers ideally should also state whether they confirmed the accuracy of the information included in their review with the original researchers, for example, by sending them a copy of the draft review [79] .

Some studies are published more than once. Duplicate publications may be difficult to ascertain, and their inclusion may introduce bias [80] , [81] . We advise authors to describe any steps they used to avoid double counting and piece together data from multiple reports of the same study (e.g., juxtaposing author names, treatment comparisons, sample sizes, or outcomes). We also advise authors to indicate whether all reports on a study were considered, as inconsistencies may reveal important limitations. For example, a review of multiple publications of drug trials showed that reported study characteristics may differ from report to report, including the description of the design, number of patients analyzed, chosen significance level, and outcomes [82] . Authors ideally should present any algorithm that they used to select data from overlapping reports and any efforts they used to solve logical inconsistencies across reports.

Item 11: DATA ITEMS.

List and define all variables for which data were sought (e.g., PICOS, funding sources), and any assumptions and simplifications made.

Examples. “Information was extracted from each included trial on: (1) characteristics of trial participants (including age, stage and severity of disease, and method of diagnosis), and the trial's inclusion and exclusion criteria; (2) type of intervention (including type, dose, duration and frequency of the NSAID [non-steroidal anti-inflammatory drug]; versus placebo or versus the type, dose, duration and frequency of another NSAID; or versus another pain management drug; or versus no treatment); (3) type of outcome measure (including the level of pain reduction, improvement in quality of life score (using a validated scale), effect on daily activities, absence from work or school, length of follow up, unintended effects of treatment, number of women requiring more invasive treatment).” [83]

It is important for readers to know what information review authors sought, even if some of this information was not available [84] . If the review is limited to reporting only those variables that were obtained, rather than those that were deemed important but could not be obtained, bias might be introduced and the reader might be misled. It is therefore helpful if authors can refer readers to the protocol (see Item 5), and archive their extraction forms (see Item 10), including definitions of variables. The published systematic review should include a description of the processes used with, if relevant, specification of how readers can get access to additional materials.

We encourage authors to report whether some variables were added after the review started. Such variables might include those found in the studies that the reviewers identified (e.g., important outcome measures that the reviewers initially overlooked). Authors should describe the reasons for adding any variables to those already pre-specified in the protocol so that readers can understand the review process.

We advise authors to report any assumptions they made about missing or unclear information and to explain those processes. For example, in studies of women aged 50 or older it is reasonable to assume that none were pregnant, even if this is not reported. Likewise, review authors might make assumptions about the route of administration of drugs assessed. However, special care should be taken in making assumptions about qualitative information. For example, the upper age limit for “children” can vary from 15 years to 21 years, “intense” physiotherapy might mean very different things to different researchers at different times and for different patients, and the volume of blood associated with “heavy” blood loss might vary widely depending on the setting.

Item 12: RISK OF BIAS IN INDIVIDUAL STUDIES.

Describe methods used for assessing risk of bias in individual studies (including specification of whether this was done at the study or outcome level, or both), and how this information is to be used in any data synthesis.

Example. “To ascertain the validity of eligible randomized trials, pairs of reviewers working independently and with adequate reliability determined the adequacy of randomization and concealment of allocation, blinding of patients, health care providers, data collectors, and outcome assessors; and extent of loss to follow-up (i.e. proportion of patients in whom the investigators were not able to ascertain outcomes).” [85] “To explore variability in study results (heterogeneity) we specified the following hypotheses before conducting the analysis. We hypothesised that effect size may differ according to the methodological quality of the studies.” [86]

The likelihood that the treatment effect reported in a systematic review approximates the truth depends on the validity of the included studies, as certain methodological characteristics may be associated with effect sizes [87] , [88] . For example, trials without reported adequate allocation concealment exaggerate treatment effects on average compared to those with adequate concealment [88] . Therefore, it is important for authors to describe any methods that they used to gauge the risk of bias in the included studies and how that information was used [89] . Additionally, authors should provide a rationale if no assessment of risk of bias was undertaken. The most popular term to describe the issues relevant to this item is “quality,” but for the reasons that are elaborated in Box 4 we prefer to name this item as “assessment of risk of bias.”

Box 4. Study Quality and Risk of Bias

In this paper, and elsewhere [11] , we sought to use a new term for many readers, namely, risk of bias, for evaluating each included study in a systematic review. Previous papers [89] , [188] tended to use the term “quality”. When carrying out a systematic review we believe it is important to distinguish between quality and risk of bias and to focus on evaluating and reporting the latter. Quality is often the best the authors have been able to do. For example, authors may report the results of surgical trials in which blinding of the outcome assessors was not part of the trial's conduct. Even though this may have been the best methodology the researchers were able to do, there are still theoretical grounds for believing that the study was susceptible to (risk of) bias.

Assessing the risk of bias should be part of the conduct and reporting of any systematic review. In all situations, we encourage systematic reviewers to think ahead carefully about what risks of bias (methodological and clinical) may have a bearing on the results of their systematic reviews.

For systematic reviewers, understanding the risk of bias on the results of studies is often difficult, because the report is only a surrogate of the actual conduct of the study. There is some suggestion [189] , [190] that the report may not be a reasonable facsimile of the study, although this view is not shared by all [88] , [191] . There are three main ways to assess risk of bias: individual components, checklists, and scales. There are a great many scales available [192] , although we caution their use based on theoretical grounds [193] and emerging empirical evidence [194] . Checklists are less frequently used and potentially run the same problems as scales. We advocate using a component approach and one that is based on domains for which there is good empirical evidence and perhaps strong clinical grounds. The new Cochrane risk of bias tool [11] is one such component approach.

The Cochrane risk of bias tool consists of five items for which there is empirical evidence for their biasing influence on the estimates of an intervention's effectiveness in randomized trials (sequence generation, allocation concealment, blinding, incomplete outcome data, and selective outcome reporting) and a catch-all item called “other sources of bias” [11] . There is also some consensus that these items can be applied for evaluation of studies across very diverse clinical areas [93] . Other risk of bias items may be topic or even study specific, i.e., they may stem from some peculiarity of the research topic or some special feature of the design of a specific study. These peculiarities need to be investigated on a case-by-case basis, based on clinical and methodological acumen, and there can be no general recipe. In all situations, systematic reviewers need to think ahead carefully about what aspects of study quality may have a bearing on the results.

Many methods exist to assess the overall risk of bias in included studies, including scales, checklists, and individual components [90] , [91] . As discussed in Box 4 , scales that numerically summarize multiple components into a single number are misleading and unhelpful [92] , [93] . Rather, authors should specify the methodological components that they assessed. Common markers of validity for randomized trials include the following: appropriate generation of random allocation sequence [94] ; concealment of the allocation sequence [93] ; blinding of participants, health care providers, data collectors, and outcome adjudicators [95] , [96] , [97] , [98] ; proportion of patients lost to follow-up [99] , [100] ; stopping of trials early for benefit [101] ; and whether the analysis followed the intention-to-treat principle [100] , [102] . The ultimate decision regarding which methodological features to evaluate requires consideration of the strength of the empiric data, theoretical rationale, and the unique circumstances of the included studies.

Authors should report how they assessed risk of bias; whether it was in a blind manner; and if assessments were completed by more than one person, and if so, whether they were completed independently [103] , [104] . Similarly, we encourage authors to report any calibration exercises among review team members that were done. Finally, authors need to report how their assessments of risk of bias are used subsequently in the data synthesis (see Item 16). Despite the often difficult task of assessing the risk of bias in included studies, authors are sometimes silent on what they did with the resultant assessments [89] . If authors exclude studies from the review or any subsequent analyses on the basis of the risk of bias, they should tell readers which studies they excluded and explain the reasons for those exclusions (see Item 6). Authors should also describe any planned sensitivity or subgroup analyses related to bias assessments (see Item 16).

Item 13: SUMMARY MEASURES.

State the principal summary measures (e.g., risk ratio, difference in means).

Examples. “Relative risk of mortality reduction was the primary measure of treatment effect.” [105] “The meta-analyses were performed by computing relative risks (RRs) using random-effects model. Quantitative analyses were performed on an intention-to-treat basis and were confined to data derived from the period of follow-up. RR and 95% confidence intervals for each side effect (and all side effects) were calculated.” [106] “The primary outcome measure was the mean difference in log 10 HIV-1 viral load comparing zinc supplementation to placebo…” [107]

When planning a systematic review, it is generally desirable that authors pre-specify the outcomes of primary interest (see Item 5) as well as the intended summary effect measure for each outcome. The chosen summary effect measure may differ from that used in some of the included studies. If possible the choice of effect measures should be explained, though it is not always easy to judge in advance which measure is the most appropriate.

For binary outcomes, the most common summary measures are the risk ratio, odds ratio, and risk difference [108] . Relative effects are more consistent across studies than absolute effects [109] , [110] , although absolute differences are important when interpreting findings (see Item 24).

For continuous outcomes, the natural effect measure is the difference in means [108] . Its use is appropriate when outcome measurements in all studies are made on the same scale. The standardized difference in means is used when the studies do not yield directly comparable data. Usually this occurs when all studies assess the same outcome but measure it in a variety of ways (e.g., different scales to measure depression).

For time-to-event outcomes, the hazard ratio is the most common summary measure. Reviewers need the log hazard ratio and its standard error for a study to be included in a meta-analysis [111] . This information may not be given for all studies, but methods are available for estimating the desired quantities from other reported information [111] . Risk ratio and odds ratio (in relation to events occurring by a fixed time) are not equivalent to the hazard ratio, and median survival times are not a reliable basis for meta-analysis [112] . If authors have used these measures they should describe their methods in the report.

Item 14: PLANNED METHODS OF ANALYSIS.

Describe the methods of handling data and combining results of studies, if done, including measures of consistency (e.g., I 2 ) for each meta-analysis.

Examples. “We tested for heterogeneity with the Breslow-Day test, and used the method proposed by Higgins et al. to measure inconsistency (the percentage of total variation across studies due to heterogeneity) of effects across lipid-lowering interventions. The advantages of this measure of inconsistency (termed I 2 ) are that it does not inherently depend on the number of studies and is accompanied by an uncertainty interval.” [113] “In very few instances, estimates of baseline mean or mean QOL [Quality of life] responses were obtained without corresponding estimates of variance (standard deviation [SD] or standard error). In these instances, an SD was imputed from the mean of the known SDs. In a number of cases, the response data available were the mean and variance in a pre study condition and after therapy. The within-patient variance in these cases could not be calculated directly and was approximated by assuming independence.” [114]

The data extracted from the studies in the review may need some transformation (processing) before they are suitable for analysis or for presentation in an evidence table. Although such data handling may facilitate meta-analyses, it is sometimes needed even when meta-analyses are not done. For example, in trials with more than two intervention groups it may be necessary to combine results for two or more groups (e.g., receiving similar but non-identical interventions), or it may be desirable to include only a subset of the data to match the review's inclusion criteria. When several different scales (e.g., for depression) are used across studies, the sign of some scores may need to be reversed to ensure that all scales are aligned (e.g., so low values represent good health on all scales). Standard deviations may have to be reconstructed from other statistics such as p -values and t statistics [115] , [116] , or occasionally they may be imputed from the standard deviations observed in other studies [117] . Time-to-event data also usually need careful conversions to a consistent format [111] . Authors should report details of any such data processing.

Statistical combination of data from two or more separate studies in a meta-analysis may be neither necessary nor desirable (see Box 5 and Item 21). Regardless of the decision to combine individual study results, authors should report how they planned to evaluate between-study variability (heterogeneity or inconsistency) ( Box 6 ). The consistency of results across trials may influence the decision of whether to combine trial results in a meta-analysis.

Box 5. Whether or Not To Combine Data

Deciding whether or not to combine data involves statistical, clinical, and methodological considerations. The statistical decisions are perhaps the most technical and evidence-based. These are more thoroughly discussed in Box 6 . The clinical and methodological decisions are generally based on discussions within the review team and may be more subjective.

Clinical considerations will be influenced by the question the review is attempting to address. Broad questions might provide more “license” to combine more disparate studies, such as whether “Ritalin is effective in increasing focused attention in people diagnosed with attention deficit hyperactivity disorder (ADHD).” Here authors might elect to combine reports of studies involving children and adults. If the clinical question is more focused, such as whether “Ritalin is effective in increasing classroom attention in previously undiagnosed ADHD children who have no comorbid conditions,” it is likely that different decisions regarding synthesis of studies are taken by authors. In any case authors should describe their clinical decisions in the systematic review report.

Deciding whether or not to combine data also has a methodological component. Reviewers may decide not to combine studies of low risk of bias with those of high risk of bias (see Items 12 and 19). For example, for subjective outcomes, systematic review authors may not wish to combine assessments that were completed under blind conditions with those that were not.

For any particular question there may not be a “right” or “wrong” choice concerning synthesis, as such decisions are likely complex. However, as the choice may be subjective, authors should be transparent as to their key decisions and describe them for readers.

Box 6. Meta-Analysis and Assessment of Consistency (Heterogeneity)

Meta-analysis: statistical combination of the results of multiple studies.

If it is felt that studies should have their results combined statistically, other issues must be considered because there are many ways to conduct a meta-analysis. Different effect measures can be used for both binary and continuous outcomes (see Item 13). Also, there are two commonly used statistical models for combining data in a meta-analysis [195] . The fixed-effect model assumes that there is a common treatment effect for all included studies [196] ; it is assumed that the observed differences in results across studies reflect random variation [196] . The random-effects model assumes that there is no common treatment effect for all included studies but rather that the variation of the effects across studies follows a particular distribution [197] . In a random-effects model it is believed that the included studies represent a random sample from a larger population of studies addressing the question of interest [198] .

There is no consensus about whether to use fixed- or random-effects models, and both are in wide use. The following differences have influenced some researchers regarding their choice between them. The random-effects model gives more weight to the results of smaller trials than does the fixed-effect analysis, which may be undesirable as small trials may be inferior and most prone to publication bias. The fixed-effect model considers only within-study variability whereas the random-effects model considers both within- and between-study variability. This is why a fixed-effect analysis tends to give narrower confidence intervals (i.e., provide greater precision) than a random-effects analysis [110] , [196] , [199] . In the absence of any between-study heterogeneity, the fixed- and random-effects estimates will coincide.

In addition, there are different methods for performing both types of meta-analysis [200] . Common fixed-effect approaches are Mantel-Haenszel and inverse variance, whereas random-effects analyses usually use the DerSimonian and Laird approach, although other methods exist, including Bayesian meta-analysis [201] .

In the presence of demonstrable between-study heterogeneity (see below), some consider that the use of a fixed-effect analysis is counterintuitive because their main assumption is violated. Others argue that it is inappropriate to conduct any meta-analysis when there is unexplained variability across trial results. If the reviewers decide not to combine the data quantitatively, a danger is that eventually they may end up using quasi-quantitative rules of poor validity (e.g., vote counting of how many studies have nominally significant results) for interpreting the evidence. Statistical methods to combine data exist for almost any complex situation that may arise in a systematic review, but one has to be aware of their assumptions and limitations to avoid misapplying or misinterpreting these methods.

Assessment of Consistency (Heterogeneity)

We expect some variation (inconsistency) in the results of different studies due to chance alone. Variability in excess of that due to chance reflects true differences in the results of the trials, and is called “heterogeneity.” The conventional statistical approach to evaluating heterogeneity is a chi-squared test (Cochran's Q), but it has low power when there are few studies and excessive power when there are many studies [202] . By contrast, the I 2 statistic quantifies the amount of variation in results across studies beyond that expected by chance and so is preferable to Q [202] , [203] . I 2 represents the percentage of the total variation in estimated effects across studies that is due to heterogeneity rather than to chance; some authors consider an I 2 value less than 25% as low [202] . However, I 2 also suffers from large uncertainty in the common situation where only a few studies are available [204] , and reporting the uncertainty in I 2 (e.g., as the 95% confidence interval) may be helpful [145] . When there are few studies, inferences about heterogeneity should be cautious.

When considerable heterogeneity is observed, it is advisable to consider possible reasons [205] . In particular, the heterogeneity may be due to differences between subgroups of studies (see Item 16). Also, data extraction errors are a common cause of substantial heterogeneity in results with continuous outcomes [139] .

When meta-analysis is done, authors should specify the effect measure (e.g., relative risk or mean difference) (see Item 13), the statistical method (e.g., inverse variance), and whether a fixed- or random-effects approach, or some other method (e.g., Bayesian) was used (see Box 6 ). If possible, authors should explain the reasons for those choices.

Item 15: RISK OF BIAS ACROSS STUDIES.

Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies).

Examples. “For each trial we plotted the effect by the inverse of its standard error. The symmetry of such ‘funnel plots’ was assessed both visually, and formally with Egger's test, to see if the effect decreased with increasing sample size.” [118] “We assessed the possibility of publication bias by evaluating a funnel plot of the trial mean differences for asymmetry, which can result from the non publication of small trials with negative results…Because graphical evaluation can be subjective, we also conducted an adjusted rank correlation test and a regression asymmetry test as formal statistical tests for publication bias…We acknowledge that other factors, such as differences in trial quality or true study heterogeneity, could produce asymmetry in funnel plots.” [119]

Reviewers should explore the possibility that the available data are biased. They may examine results from the available studies for clues that suggest there may be missing studies (publication bias) or missing data from the included studies (selective reporting bias) (see Box 7 ). Authors should report in detail any methods used to investigate possible bias across studies.

Box 7. Bias Caused by Selective Publication of Studies or Results within Studies

Systematic reviews aim to incorporate information from all relevant studies. The absence of information from some studies may pose a serious threat to the validity of a review. Data may be incomplete because some studies were not published, or because of incomplete or inadequate reporting within a published article. These problems are often summarized as “publication bias” although in fact the bias arises from non-publication of full studies and selective publication of results in relation to their findings. Non-publication of research findings dependent on the actual results is an important risk of bias to a systematic review and meta-analysis.

Missing Studies

Several empirical investigations have shown that the findings from clinical trials are more likely to be published if the results are statistically significant ( p <0.05) than if they are not [125] , [206] , [207] . For example, of 500 oncology trials with more than 200 participants for which preliminary results were presented at a conference of the American Society of Clinical Oncology, 81% with p <0.05 were published in full within five years compared to only 68% of those with p >0.05 [208] .

Also, among published studies, those with statistically significant results are published sooner than those with non-significant findings [209] . When some studies are missing for these reasons, the available results will be biased towards exaggerating the effect of an intervention.

Missing Outcomes

In many systematic reviews only some of the eligible studies (often a minority) can be included in a meta-analysis for a specific outcome. For some studies, the outcome may not be measured or may be measured but not reported. The former will not lead to bias, but the latter could.

Evidence is accumulating that selective reporting bias is widespread and of considerable importance [42] , [43] . In addition, data for a given outcome may be analyzed in multiple ways and the choice of presentation influenced by the results obtained. In a study of 102 randomized trials, comparison of published reports with trial protocols showed that a median of 38% efficacy and 50% safety outcomes per trial, respectively, were not available for meta-analysis. Statistically significant outcomes had a higher odds of being fully reported in publications when compared with non-significant outcomes for both efficacy (pooled odds ratio 2.4; 95% confidence interval 1.4 to 4.0) and safety (4.7, 1.8 to 12) data. Several other studies have had similar findings [210] , [211] .

Detection of Missing Information

Missing studies may increasingly be identified from trials registries. Evidence of missing outcomes may come from comparison with the study protocol, if available, or by careful examination of published articles [11] . Study publication bias and selective outcome reporting are difficult to exclude or verify from the available results, especially when few studies are available.

If the available data are affected by either (or both) of the above biases, smaller studies would tend to show larger estimates of the effects of the intervention. Thus one possibility is to investigate the relation between effect size and sample size (or more specifically, precision of the effect estimate). Graphical methods, especially the funnel plot [212] , and analytic methods (e.g., Egger's test) are often used [213] , [214] , [215] , although their interpretation can be problematic [216] , [217] . Strictly speaking, such analyses investigate “small study bias”; there may be many reasons why smaller studies have systematically different effect sizes than larger studies, of which reporting bias is just one [218] . Several alternative tests for bias have also been proposed, beyond the ones testing small study bias [215] , [219] , [220] , but none can be considered a gold standard. Although evidence that smaller studies had larger estimated effects than large ones may suggest the possibility that the available evidence is biased, misinterpretation of such data is common [123] .

It is difficult to assess whether within-study selective reporting is present in a systematic review. If a protocol of an individual study is available, the outcomes in the protocol and the published report can be compared. Even in the absence of a protocol, outcomes listed in the methods section of the published report can be compared with those for which results are presented [120] . In only half of 196 trial reports describing comparisons of two drugs in arthritis were all the effect variables in the methods and results sections the same [82] . In other cases, knowledge of the clinical area may suggest that it is likely that the outcome was measured even if it was not reported. For example, in a particular disease, if one of two linked outcomes is reported but the other is not, then one should question whether the latter has been selectively omitted [121] , [122] .

Only 36% (76 of 212) of therapeutic systematic reviews published in November 2004 reported that study publication bias was considered, and only a quarter of those intended to carry out a formal assessment for that bias [3] . Of 60 meta-analyses in 24 articles published in 2005 in which formal assessments were reported, most were based on fewer than ten studies; most displayed statistically significant heterogeneity; and many reviewers misinterpreted the results of the tests employed [123] . A review of trials of antidepressants found that meta-analysis of only the published trials gave effect estimates 32% larger on average than when all trials sent to the drug agency were analyzed [67] .

Item 16: ADDITIONAL ANALYSES.

Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified.

Example. “Sensitivity analyses were pre-specified. The treatment effects were examined according to quality components (concealed treatment allocation, blinding of patients and caregivers, blinded outcome assessment), time to initiation of statins, and the type of statin. One post-hoc sensitivity analysis was conducted including unpublished data from a trial using cerivastatin.” [124]

Authors may perform additional analyses to help understand whether the results of their review are robust, all of which should be reported. Such analyses include sensitivity analysis, subgroup analysis, and meta-regression [125] .

Sensitivity analyses are used to explore the degree to which the main findings of a systematic review are affected by changes in its methods or in the data used from individual studies (e.g., study inclusion criteria, results of risk of bias assessment). Subgroup analyses address whether the summary effects vary in relation to specific (usually clinical) characteristics of the included studies or their participants. Meta-regression extends the idea of subgroup analysis to the examination of the quantitative influence of study characteristics on the effect size [126] . Meta-regression also allows authors to examine the contribution of different variables to the heterogeneity in study findings. Readers of systematic reviews should be aware that meta-regression has many limitations, including a danger of over-interpretation of findings [127] , [128] .

Even with limited data, many additional analyses can be undertaken. The choice of which analysis to undertake will depend on the aims of the review. None of these analyses, however, are exempt from producing potentially misleading results. It is important to inform readers whether these analyses were performed, their rationale, and which were pre-specified.

Item 17: STUDY SELECTION.

Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram.

Examples. In text : “A total of 10 studies involving 13 trials were identified for inclusion in the review. The search of Medline, PsycInfo and Cinahl databases provided a total of 584 citations. After adjusting for duplicates 509 remained. Of these, 479 studies were discarded because after reviewing the abstracts it appeared that these papers clearly did not meet the criteria. Three additional studies…were discarded because full text of the study was not available or the paper could not be feasibly translated into English. The full text of the remaining 27 citations was examined in more detail. It appeared that 22 studies did not meet the inclusion criteria as described. Five studies…met the inclusion criteria and were included in the systematic review. An additional five studies…that met the criteria for inclusion were identified by checking the references of located, relevant papers and searching for studies that have cited these papers. No unpublished relevant studies were obtained.” [129] See flow diagram Figure 2 .

DDW, Digestive Disease Week; UEGW, United European Gastroenterology Week. Reproduced with permission from [130] .

https://doi.org/10.1371/journal.pmed.1000100.g002

Authors should report, ideally with a flow diagram, the total number of records identified from electronic bibliographic sources (including specialized database or registry searches), hand searches of various sources, reference lists, citation indices, and experts. It is useful if authors delineate for readers the number of selected articles that were identified from the different sources so that they can see, for example, whether most articles were identified through electronic bibliographic sources or from references or experts. Literature identified primarily from references or experts may be prone to citation or publication bias [131] , [132] .

The flow diagram and text should describe clearly the process of report selection throughout the review. Authors should report: unique records identified in searches; records excluded after preliminary screening (e.g., screening of titles and abstracts); reports retrieved for detailed evaluation; potentially eligible reports that were not retrievable; retrieved reports that did not meet inclusion criteria and the primary reasons for exclusion; and the studies included in the review. Indeed, the most appropriate layout may vary for different reviews.

Authors should also note the presence of duplicate or supplementary reports so that readers understand the number of individual studies compared to the number of reports that were included in the review. Authors should be consistent in their use of terms, such as whether they are reporting on counts of citations, records, publications, or studies. We believe that reporting the number of studies is the most important.

A flow diagram can be very useful; it should depict all the studies included based upon fulfilling the eligibility criteria, whether or not data have been combined for statistical analysis. A recent review of 87 systematic reviews found that about half included a QUOROM flow diagram [133] . The authors of this research recommended some important ways that reviewers can improve the use of a flow diagram when describing the flow of information throughout the review process, including a separate flow diagram for each important outcome reported [133] .

Item 18: STUDY CHARACTERISTICS.

For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citation.

Examples. In text : “Characteristics of included studies Methods All four studies finally selected for the review were randomised controlled trials published in English. The duration of the intervention was 24 months for the RIO-North America and 12 months for the RIO-Diabetes, RIO-Lipids and RIO-Europe study. Although the last two described a period of 24 months during which they were conducted, only the first 12-months results are provided. All trials had a run-in, as a single blind period before the randomisation. Participants The included studies involved 6625 participants. The main inclusion criteria entailed adults (18 years or older), with a body mass index greater than 27 kg/m 2 and less than 5 kg variation in body weight within the three months before study entry. Intervention All trials were multicentric. The RIO-North America was conducted in the USA and Canada, RIO-Europe in Europe and the USA, RIO-Diabetes in the USA and 10 other different countries not specified, and RIO-Lipids in eight unspecified different countries. The intervention received was placebo, 5 mg of rimonabant or 20 mg of rimonabant once daily in addition to a mild hypocaloric diet (600 kcal/day deficit). Outcomes Primary In all studies the primary outcome assessed was weight change from baseline after one year of treatment and the RIO-North America study also evaluated the prevention of weight regain between the first and second year. All studies evaluated adverse effects, including those of any kind and serious events. Quality of life was measured in only one study, but the results were not described (RIO-Europe). Secondary and additional outcomes These included prevalence of metabolic syndrome after one year and change in cardiometabolic risk factors such as blood pressure, lipid profile, etc. No study included mortality and costs as outcome. The timing of outcome measures was variable and could include monthly investigations, evaluations every three months or a single final evaluation after one year.” [134] In table : See Table 2 .

https://doi.org/10.1371/journal.pmed.1000100.t002

For readers to gauge the validity and applicability of a systematic review's results, they need to know something about the included studies. Such information includes PICOS ( Box 2 ) and specific information relevant to the review question. For example, if the review is examining the long-term effects of antidepressants for moderate depressive disorder, authors should report the follow-up periods of the included studies. For each included study, authors should provide a citation for the source of their information regardless of whether or not the study is published. This information makes it easier for interested readers to retrieve the relevant publications or documents.

Reporting study-level data also allows the comparison of the main characteristics of the studies included in the review. Authors should present enough detail to allow readers to make their own judgments about the relevance of included studies. Such information also makes it possible for readers to conduct their own subgroup analyses and interpret subgroups, based on study characteristics.

Authors should avoid, whenever possible, assuming information when it is missing from a study report (e.g., sample size, method of randomization). Reviewers may contact the original investigators to try to obtain missing information or confirm the data extracted for the systematic review. If this information is not obtained, this should be noted in the report. If information is imputed, the reader should be told how this was done and for which items. Presenting study-level data makes it possible to clearly identify unpublished information obtained from the original researchers and make it available for the public record.

Typically, study-level characteristics are presented as a table as in the example in Table 2 . Such presentation ensures that all pertinent items are addressed and that missing or unclear information is clearly indicated. Although paper-based journals do not generally allow for the quantity of information available in electronic journals or Cochrane reviews, this should not be accepted as an excuse for omission of important aspects of the methods or results of included studies, since these can, if necessary, be shown on a Web site.

Following the presentation and description of each included study, as discussed above, reviewers usually provide a narrative summary of the studies. Such a summary provides readers with an overview of the included studies. It may for example address the languages of the published papers, years of publication, and geographic origins of the included studies.

The PICOS framework is often helpful in reporting the narrative summary indicating, for example, the clinical characteristics and disease severity of the participants and the main features of the intervention and of the comparison group. For non-pharmacological interventions, it may be helpful to specify for each study the key elements of the intervention received by each group. Full details of the interventions in included studies were reported in only three of 25 systematic reviews relevant to general practice [84] .

Item 19: RISK OF BIAS WITHIN STUDIES.

Present data on risk of bias of each study and, if available, any outcome-level assessment (see Item 12).

Example. See Table 3 .

https://doi.org/10.1371/journal.pmed.1000100.t003

We recommend that reviewers assess the risk of bias in the included studies using a standard approach with defined criteria (see Item 12). They should report the results of any such assessments [89] .

Reporting only summary data (e.g., “two of eight trials adequately concealed allocation”) is inadequate because it fails to inform readers which studies had the particular methodological shortcoming. A more informative approach is to explicitly report the methodological features evaluated for each study. The Cochrane Collaboration's new tool for assessing the risk of bias also requests that authors substantiate these assessments with any relevant text from the original studies [11] . It is often easiest to provide these data in a tabular format, as in the example. However, a narrative summary describing the tabular data can also be helpful for readers.

Item 20: RESULTS OF INDIVIDUAL STUDIES.

For all outcomes considered (benefits and harms), present, for each study: (a) simple summary data for each intervention group and (b) effect estimates and confidence intervals, ideally with a forest plot.

Examples. See Table 4 and Figure 3 .

CI, confidence interval. Reproduced with permission from [137] .

https://doi.org/10.1371/journal.pmed.1000100.g003

https://doi.org/10.1371/journal.pmed.1000100.t004

Publication of summary data from individual studies allows the analyses to be reproduced and other analyses and graphical displays to be investigated. Others may wish to assess the impact of excluding particular studies or consider subgroup analyses not reported by the review authors. Displaying the results of each treatment group in included studies also enables inspection of individual study features. For example, if only odds ratios are provided, readers cannot assess the variation in event rates across the studies, making the odds ratio impossible to interpret [138] . Additionally, because data extraction errors in meta-analyses are common and can be large [139] , the presentation of the results from individual studies makes it easier to identify errors. For continuous outcomes, readers may wish to examine the consistency of standard deviations across studies, for example, to be reassured that standard deviation and standard error have not been confused [138] .

For each study, the summary data for each intervention group are generally given for binary outcomes as frequencies with and without the event (or as proportions such as 12/45). It is not sufficient to report event rates per intervention group as percentages. The required summary data for continuous outcomes are the mean, standard deviation, and sample size for each group. In reviews that examine time-to-event data, the authors should report the log hazard ratio and its standard error (or confidence interval) for each included study. Sometimes, essential data are missing from the reports of the included studies and cannot be calculated from other data but may need to be imputed by the reviewers. For example, the standard deviation may be imputed using the typical standard deviations in the other trials [116] , [117] (see Item 14). Whenever relevant, authors should indicate which results were not reported directly and had to be estimated from other information (see Item 13). In addition, the inclusion of unpublished data should be noted.

For all included studies it is important to present the estimated effect with a confidence interval. This information may be incorporated in a table showing study characteristics or may be shown in a forest plot [140] . The key elements of the forest plot are the effect estimates and confidence intervals for each study shown graphically, but it is preferable also to include, for each study, the numerical group-specific summary data, the effect size and confidence interval, and the percentage weight (see second example [ Figure 3 ]). For discussion of the results of meta-analysis, see Item 21.

In principle, all the above information should be provided for every outcome considered in the review, including both benefits and harms. When there are too many outcomes for full information to be included, results for the most important outcomes should be included in the main report with other information provided as a Web appendix. The choice of the information to present should be justified in light of what was originally stated in the protocol. Authors should explicitly mention if the planned main outcomes cannot be presented due to lack of information. There is some evidence that information on harms is only rarely reported in systematic reviews, even when it is available in the original studies [141] . Selective omission of harms results biases a systematic review and decreases its ability to contribute to informed decision making.

Item 21: SYNTHESES OF RESULTS.

Present the main results of the review. If meta-analyses are done, include for each, confidence intervals and measures of consistency.

Examples. “Mortality data were available for all six trials, randomizing 311 patients and reporting data for 305 patients. There were no deaths reported in the three respiratory syncytial virus/severe bronchiolitis trials; thus our estimate is based on three trials randomizing 232 patients, 64 of whom died. In the pooled analysis, surfactant was associated with significantly lower mortality (relative risk = 0.7, 95% confidence interval = 0.4–0.97, P = 0.04). There was no evidence of heterogeneity (I 2  = 0%)”. [142] “Because the study designs, participants, interventions, and reported outcome measures varied markedly, we focused on describing the studies, their results, their applicability, and their limitations and on qualitative synthesis rather than meta-analysis.” [143] “We detected significant heterogeneity within this comparison (I 2  = 46.6%; χ 2  = 13.11, df = 7; P = 0.07). Retrospective exploration of the heterogeneity identified one trial that seemed to differ from the others. It included only small ulcers (wound area less than 5 cm 2 ). Exclusion of this trial removed the statistical heterogeneity and did not affect the finding of no evidence of a difference in healing rate between hydrocolloids and simple low adherent dressings (relative risk = 0.98, [95% confidence interval] 0.85 to 1.12; I 2  = 0%).” [144]

Results of systematic reviews should be presented in an orderly manner. Initial narrative descriptions of the evidence covered in the review (see Item 18) may tell readers important things about the study populations and the design and conduct of studies. These descriptions can facilitate the examination of patterns across studies. They may also provide important information about applicability of evidence, suggest the likely effects of any major biases, and allow consideration, in a systematic manner, of multiple explanations for possible differences of findings across studies.

If authors have conducted one or more meta-analyses, they should present the results as an estimated effect across studies with a confidence interval. It is often simplest to show each meta-analysis summary with the actual results of included studies in a forest plot (see Item 20) [140] . It should always be clear which of the included studies contributed to each meta-analysis. Authors should also provide, for each meta-analysis, a measure of the consistency of the results from the included studies such as I 2 (heterogeneity; see Box 6 ); a confidence interval may also be given for this measure [145] . If no meta-analysis was performed, the qualitative inferences should be presented as systematically as possible with an explanation of why meta-analysis was not done, as in the second example above [143] . Readers may find a forest plot, without a summary estimate, helpful in such cases.

Authors should in general report syntheses for all the outcome measures they set out to investigate (i.e., those described in the protocol; see Item 4) to allow readers to draw their own conclusions about the implications of the results. Readers should be made aware of any deviations from the planned analysis. Authors should tell readers if the planned meta-analysis was not thought appropriate or possible for some of the outcomes and the reasons for that decision.

It may not always be sensible to give meta-analysis results and forest plots for each outcome. If the review addresses a broad question, there may be a very large number of outcomes. Also, some outcomes may have been reported in only one or two studies, in which case forest plots are of little value and may be seriously biased.

Of 300 systematic reviews indexed in MEDLINE in 2004, a little more than half (54%) included meta-analyses, of which the majority (91%) reported assessing for inconsistency in results.

Item 22: RISK OF BIAS ACROSS STUDIES.

Present results of any assessment of risk of bias across studies (see Item 15).

Examples. “Strong evidence of heterogeneity (I 2  = 79%, P <0.001) was observed. To explore this heterogeneity, a funnel plot was drawn. The funnel plot in Figure 4 shows evidence of considerable asymmetry.” [146] “Specifically, four sertraline trials involving 486 participants and one citalopram trial involving 274 participants were reported as having failed to achieve a statistically significant drug effect, without reporting mean HRSD [Hamilton Rating Scale for Depression] scores. We were unable to find data from these trials on pharmaceutical company Web sites or through our search of the published literature. These omissions represent 38% of patients in sertraline trials and 23% of patients in citalopram trials. Analyses with and without inclusion of these trials found no differences in the patterns of results; similarly, the revealed patterns do not interact with drug type. The purpose of using the data obtained from the FDA was to avoid publication bias, by including unpublished as well as published trials. Inclusion of only those sertraline and citalopram trials for which means were reported to the FDA would constitute a form of reporting bias similar to publication bias and would lead to overestimation of drug–placebo differences for these drug types. Therefore, we present analyses only on data for medications for which complete clinical trials' change was reported.” [147]

SE, standard error. Adapted from [146] , with permission.

https://doi.org/10.1371/journal.pmed.1000100.g004

Authors should present the results of any assessments of risk of bias across studies. If a funnel plot is reported, authors should specify the effect estimate and measure of precision used, presented typically on the x -axis and y -axis, respectively. Authors should describe if and how they have tested the statistical significance of any possible asymmetry (see Item 15). Results of any investigations of selective reporting of outcomes within studies (as discussed in Item 15) should also be reported. Also, we advise authors to tell readers if any pre-specified analyses for assessing risk of bias across studies were not completed and the reasons (e.g., too few included studies).

Item 23: ADDITIONAL ANALYSES.

Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]).

Examples. “…benefits of chondroitin were smaller in trials with adequate concealment of allocation compared with trials with unclear concealment (P for interaction = 0.050), in trials with an intention-to-treat analysis compared with those that had excluded patients from the analysis (P for interaction = 0.017), and in large compared with small trials (P for interaction = 0.022).” [148] “Subgroup analyses according to antibody status, antiviral medications, organ transplanted, treatment duration, use of antilymphocyte therapy, time to outcome assessment, study quality and other aspects of study design did not demonstrate any differences in treatment effects. Multivariate meta-regression showed no significant difference in CMV [cytomegalovirus] disease after allowing for potential confounding or effect-modification by prophylactic drug used, organ transplanted or recipient serostatus in CMV positive recipients and CMV negative recipients of CMV positive donors.” [149]

Authors should report any subgroup or sensitivity analyses and whether or not they were pre-specified (see Items 5 and 16). For analyses comparing subgroups of studies (e.g., separating studies of low- and high-dose aspirin), the authors should report any tests for interactions, as well as estimates and confidence intervals from meta-analyses within each subgroup. Similarly, meta-regression results (see Item 16) should not be limited to p -values, but should include effect sizes and confidence intervals [150] , as the first example reported above does in a table. The amount of data included in each additional analysis should be specified if different from that considered in the main analyses. This information is especially relevant for sensitivity analyses that exclude some studies; for example, those with high risk of bias.

Importantly, all additional analyses conducted should be reported, not just those that were statistically significant. This information will help avoid selective outcome reporting bias within the review as has been demonstrated in reports of randomized controlled trials [42] , [44] , [121] , [151] , [152] . Results from exploratory subgroup or sensitivity analyses should be interpreted cautiously, bearing in mind the potential for multiple analyses to mislead.

Item 24: SUMMARY OF EVIDENCE.

Summarize the main findings, including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., health care providers, users, and policy makers).

Example. “Overall, the evidence is not sufficiently robust to determine the comparative effectiveness of angioplasty (with or without stenting) and medical treatment alone. Only 2 randomized trials with long-term outcomes and a third randomized trial that allowed substantial crossover of treatment after 3 months directly compared angioplasty and medical treatment…the randomized trials did not evaluate enough patients or did not follow patients for a sufficient duration to allow definitive conclusions to be made about clinical outcomes, such as mortality and cardiovascular or kidney failure events. Some acceptable evidence from comparison of medical treatment and angioplasty suggested no difference in long-term kidney function but possibly better blood pressure control after angioplasty, an effect that may be limited to patients with bilateral atherosclerotic renal artery stenosis. The evidence regarding other outcomes is weak. Because the reviewed studies did not explicitly address patients with rapid clinical deterioration who may need acute intervention, our conclusions do not apply to this important subset of patients.” [143]

Authors should give a brief and balanced summary of the nature and findings of the review. Sometimes, outcomes for which little or no data were found should be noted due to potential relevance for policy decisions and future research. Applicability of the review's findings, to different patients, settings, or target audiences, for example, should be mentioned. Although there is no standard way to assess applicability simultaneously to different audiences, some systems do exist [153] . Sometimes, authors formally rate or assess the overall body of evidence addressed in the review and can present the strength of their summary recommendations tied to their assessments of the quality of evidence (e.g., the GRADE system) [10] .

Authors need to keep in mind that statistical significance of the effects does not always suggest clinical or policy relevance. Likewise, a non-significant result does not demonstrate that a treatment is ineffective. Authors should ideally clarify trade-offs and how the values attached to the main outcomes would lead different people to make different decisions. In addition, adroit authors consider factors that are important in translating the evidence to different settings and that may modify the estimates of effects reported in the review [153] . Patients and health care providers may be primarily interested in which intervention is most likely to provide a benefit with acceptable harms, while policy makers and administrators may value data on organizational impact and resource utilization.

Item 25: LIMITATIONS.

Discuss limitations at study and outcome level (e.g., risk of bias), and at review level (e.g., incomplete retrieval of identified research, reporting bias).

Examples. Outcome level: “The meta-analysis reported here combines data across studies in order to estimate treatment effects with more precision than is possible in a single study. The main limitation of this meta-analysis, as with any overview, is that the patient population, the antibiotic regimen and the outcome definitions are not the same across studies.” [154] Study and review level: “Our study has several limitations. The quality of the studies varied. Randomization was adequate in all trials; however, 7 of the articles did not explicitly state that analysis of data adhered to the intention-to-treat principle, which could lead to overestimation of treatment effect in these trials, and we could not assess the quality of 4 of the 5 trials reported as abstracts. Analyses did not identify an association between components of quality and re-bleeding risk, and the effect size in favour of combination therapy remained statistically significant when we excluded trials that were reported as abstracts. Publication bias might account for some of the effect we observed. Smaller trials are, in general, analyzed with less methodological rigor than larger studies, and an asymmetrical funnel plot suggests that selective reporting may have led to an overestimation of effect sizes in small trials.” [155]

A discussion of limitations should address the validity (i.e., risk of bias) and reporting (informativeness) of the included studies, limitations of the review process, and generalizability (applicability) of the review. Readers may find it helpful if authors discuss whether studies were threatened by serious risks of bias, whether the estimates of the effect of the intervention are too imprecise, or if there were missing data for many participants or important outcomes.

Limitations of the review process might include limitations of the search (e.g., restricting to English-language publications), and any difficulties in the study selection, appraisal, and meta-analysis processes. For example, poor or incomplete reporting of study designs, patient populations, and interventions may hamper interpretation and synthesis of the included studies [84] . Applicability of the review may be affected if there are limited data for certain populations or subgroups where the intervention might perform differently or few studies assessing the most important outcomes of interest; or if there is a substantial amount of data relating to an outdated intervention or comparator or heavy reliance on imputation of missing values for summary estimates (Item 14).

Item 26: CONCLUSIONS.

Provide a general interpretation of the results in the context of other evidence, and implications for future research.

Example. Implications for practice: “Between 1995 and 1997 five different meta-analyses of the effect of antibiotic prophylaxis on infection and mortality were published. All confirmed a significant reduction in infections, though the magnitude of the effect varied from one review to another. The estimated impact on overall mortality was less evident and has generated considerable controversy on the cost effectiveness of the treatment. Only one among the five available reviews, however, suggested that a weak association between respiratory tract infections and mortality exists and lack of sufficient statistical power may have accounted for the limited effect on mortality.” Implications for research : “A logical next step for future trials would thus be the comparison of this protocol against a regimen of a systemic antibiotic agent only to see whether the topical component can be dropped. We have already identified six such trials but the total number of patients so far enrolled (n = 1056) is too small for us to be confident that the two treatments are really equally effective. If the hypothesis is therefore considered worth testing more and larger randomised controlled trials are warranted. Trials of this kind, however, would not resolve the relevant issue of treatment induced resistance. To produce a satisfactory answer to this, studies with a different design would be necessary. Though a detailed discussion goes beyond the scope of this paper, studies in which the intensive care unit rather than the individual patient is the unit of randomisation and in which the occurrence of antibiotic resistance is monitored over a long period of time should be undertaken.” [156]

Systematic reviewers sometimes draw conclusions that are too optimistic [157] or do not consider the harms equally as carefully as the benefits, although some evidence suggests these problems are decreasing [158] . If conclusions cannot be drawn because there are too few reliable studies, or too much uncertainty, this should be stated. Such a finding can be as important as finding consistent effects from several large studies.

Authors should try to relate the results of the review to other evidence, as this helps readers to better interpret the results. For example, there may be other systematic reviews about the same general topic that have used different methods or have addressed related but slightly different questions [159] , [160] . Similarly, there may be additional information relevant to decision makers, such as the cost-effectiveness of the intervention (e.g., health technology assessment). Authors may discuss the results of their review in the context of existing evidence regarding other interventions.

We advise authors also to make explicit recommendations for future research. In a sample of 2,535 Cochrane reviews, 82% included recommendations for research with specific interventions, 30% suggested the appropriate type of participants, and 52% suggested outcome measures for future research [161] . There is no corresponding assessment about systematic reviews published in medical journals, but we believe that such recommendations are much less common in those reviews.

Clinical research should not be planned without a thorough knowledge of similar, existing research [162] . There is evidence that this still does not occur as it should and that authors of primary studies do not consider a systematic review when they design their studies [163] . We believe systematic reviews have great potential for guiding future clinical research.

Item 27: FUNDING.

Describe sources of funding or other support (e.g., supply of data) for the systematic review; role of funders for the systematic review.

Examples: “The evidence synthesis upon which this article was based was funded by the Centers for Disease Control and Prevention for the Agency for Healthcare Research and Quality and the U.S. Prevention Services Task Force.” [164] “Role of funding source: the funders played no role in study design, collection, analysis, interpretation of data, writing of the report, or in the decision to submit the paper for publication. They accept no responsibility for the contents.” [165]

Authors of systematic reviews, like those of any other research study, should disclose any funding they received to carry out the review, or state if the review was not funded. Lexchin and colleagues [166] observed that outcomes of reports of randomized trials and meta-analyses of clinical trials funded by the pharmaceutical industry are more likely to favor the sponsor's product compared to studies with other sources of funding. Similar results have been reported elsewhere [167] , [168] . Analogous data suggest that similar biases may affect the conclusions of systematic reviews [169] .

Given the potential role of systematic reviews in decision making, we believe authors should be transparent about the funding and the role of funders, if any. Sometimes the funders will provide services, such as those of a librarian to complete the searches for relevant literature or access to commercial databases not available to the reviewers. Any level of funding or services provided to the systematic review team should be reported. Authors should also report whether the funder had any role in the conduct or report of the review. Beyond funding issues, authors should report any real or perceived conflicts of interest related to their role or the role of the funder in the reporting of the systematic review [170] .

In a survey of 300 systematic reviews published in November 2004, funding sources were not reported in 41% of the reviews [3] . Only a minority of reviews (2%) reported being funded by for-profit sources, but the true proportion may be higher [171] .

Additional Considerations for Systematic Reviews of Non-Randomized Intervention Studies or for Other Types of Systematic Reviews

The PRISMA Statement and this document have focused on systematic reviews of reports of randomized trials. Other study designs, including non-randomized studies, quasi-experimental studies, and interrupted time series, are included in some systematic reviews that evaluate the effects of health care interventions [172] , [173] . The methods of these reviews may differ to varying degrees from the typical intervention review, for example regarding the literature search, data abstraction, assessment of risk of bias, and analysis methods. As such, their reporting demands might also differ from what we have described here. A useful principle is for systematic review authors to ensure that their methods are reported with adequate clarity and transparency to enable readers to critically judge the available evidence and replicate or update the research.

In some systematic reviews, the authors will seek the raw data from the original researchers to calculate the summary statistics. These systematic reviews are called individual patient (or participant) data reviews [40] , [41] . Individual patient data meta-analyses may also be conducted with prospective accumulation of data rather than retrospective accumulation of existing data. Here too, extra information about the methods will need to be reported.

Other types of systematic reviews exist. Realist reviews aim to determine how complex programs work in specific contexts and settings [174] . Meta-narrative reviews aim to explain complex bodies of evidence through mapping and comparing different over-arching storylines [175] . Network meta-analyses, also known as multiple treatments meta-analyses, can be used to analyze data from comparisons of many different treatments [176] , [177] . They use both direct and indirect comparisons, and can be used to compare interventions that have not been directly compared.

We believe that the issues we have highlighted in this paper are relevant to ensure transparency and understanding of the processes adopted and the limitations of the information presented in systematic reviews of different types. We hope that PRISMA can be the basis for more detailed guidance on systematic reviews of other types of research, including diagnostic accuracy and epidemiological studies.

We developed the PRISMA Statement using an approach for developing reporting guidelines that has evolved over several years [178] . The overall aim of PRISMA is to help ensure the clarity and transparency of reporting of systematic reviews, and recent data indicate that this reporting guidance is much needed [3] . PRISMA is not intended to be a quality assessment tool and it should not be used as such.

This PRISMA Explanation and Elaboration document was developed to facilitate the understanding, uptake, and dissemination of the PRISMA Statement and hopefully provide a pedagogical framework for those interested in conducting and reporting systematic reviews. It follows a format similar to that used in other explanatory documents [17] , [18] , [19] . Following the recommendations in the PRISMA checklist may increase the word count of a systematic review report. We believe, however, that the benefit of readers being able to critically appraise a clear, complete, and transparent systematic review report outweighs the possible slight increase in the length of the report.

While the aims of PRISMA are to reduce the risk of flawed reporting of systematic reviews and improve the clarity and transparency in how reviews are conducted, we have little data to state more definitively whether this “intervention” will achieve its intended goal. A previous effort to evaluate QUOROM was not successfully completed [178] . Publication of the QUOROM Statement was delayed for two years while a research team attempted to evaluate its effectiveness by conducting a randomized controlled trial with the participation of eight major medical journals. Unfortunately that trial was not completed due to accrual problems (David Moher, personal communication). Other evaluation methods might be easier to conduct. At least one survey of 139 published systematic reviews in the critical care literature [179] suggests that their quality improved after the publication of QUOROM.

If the PRISMA Statement is endorsed by and adhered to in journals, as other reporting guidelines have been [17] , [18] , [19] , [180] , there should be evidence of improved reporting of systematic reviews. For example, there have been several evaluations of whether the use of CONSORT improves reports of randomized controlled trials. A systematic review of these studies [181] indicates that use of CONSORT is associated with improved reporting of certain items, such as allocation concealment. We aim to evaluate the benefits (i.e., improved reporting) and possible adverse effects (e.g., increased word length) of PRISMA and we encourage others to consider doing likewise.

Even though we did not carry out a systematic literature search to produce our checklist, and this is indeed a limitation of our effort, PRISMA was nevertheless developed using an evidence-based approach, whenever possible. Checklist items were included if there was evidence that not reporting the item was associated with increased risk of bias, or where it was clear that information was necessary to appraise the reliability of a review. To keep PRISMA up-to-date and as evidence-based as possible requires regular vigilance of the literature, which is growing rapidly. Currently the Cochrane Methodology Register has more than 11,000 records pertaining to the conduct and reporting of systematic reviews and other evaluations of health and social care. For some checklist items, such as reporting the abstract (Item 2), we have used evidence from elsewhere in the belief that the issue applies equally well to reporting of systematic reviews. Yet for other items, evidence does not exist; for example, whether a training exercise improves the accuracy and reliability of data extraction. We hope PRISMA will act as a catalyst to help generate further evidence that can be considered when further revising the checklist in the future.

More than ten years have passed between the development of the QUOROM Statement and its update, the PRISMA Statement. We aim to update PRISMA more frequently. We hope that the implementation of PRISMA will be better than it has been for QUOROM. There are at least two reasons to be optimistic. First, systematic reviews are increasingly used by health care providers to inform “best practice” patient care. Policy analysts and managers are using systematic reviews to inform health care decision making, and to better target future research. Second, we anticipate benefits from the development of the EQUATOR Network, described below.

Developing any reporting guideline requires considerable effort, experience, and expertise. While reporting guidelines have been successful for some individual efforts [17] , [18] , [19] , there are likely others who want to develop reporting guidelines who possess little time, experience, or knowledge as to how to do so appropriately. The EQUATOR Network (Enhancing the QUAlity and Transparency Of health Research) aims to help such individuals and groups by serving as a global resource for anybody interested in developing reporting guidelines, regardless of the focus [7] , [180] , [182] . The overall goal of EQUATOR is to improve the quality of reporting of all health science research through the development and translation of reporting guidelines. Beyond this aim, the network plans to develop a large Web presence by developing and maintaining a resource center of reporting tools, and other information for reporting research ( http://www.equator-network.org/ ).

We encourage health care journals and editorial groups, such as the World Association of Medical Editors and the International Committee of Medical Journal Editors, to endorse PRISMA in much the same way as they have endorsed other reporting guidelines, such as CONSORT. We also encourage editors of health care journals to support PRISMA by updating their “Instructions to Authors” and including the PRISMA Web address, and by raising awareness through specific editorial actions.

Supporting Information

Flow of information through the different phases of a systematic review (downloadable template document for researchers to re-use).

https://doi.org/10.1371/journal.pmed.1000100.s001

(0.08 MB DOC)

Checklist of items to include when reporting a systematic review or meta-analysis (downloadable template document for researchers to re-use).

https://doi.org/10.1371/journal.pmed.1000100.s002

(0.04 MB DOC)

Acknowledgments

The following people contributed to this paper:

Doug Altman, DSc, Centre for Statistics in Medicine (Oxford, UK); Gerd Antes, PhD, University Hospital Freiburg (Freiburg, Germany); David Atkins, MD, MPH, Health Services Research and Development Service, Veterans Health Administration (Washington, D. C., US); Virginia Barbour, MRCP, DPhil, PLoS Medicine (Cambridge, UK); Nick Barrowman, PhD, Children's Hospital of Eastern Ontario (Ottawa, Canada); Jesse A. Berlin, ScD, Johnson & Johnson Pharmaceutical Research and Development (Titusville, New Jersey, US); Jocalyn Clark, PhD, PLoS Medicine (at the time of writing, BMJ , London, UK); Mike Clarke, PhD, UK Cochrane Centre (Oxford, UK) and School of Nursing and Midwifery, Trinity College (Dublin, Ireland); Deborah Cook, MD, Departments of Medicine, Clinical Epidemiology and Biostatistics, McMaster University (Hamilton, Canada); Roberto D'Amico, PhD, Università di Modena e Reggio Emilia (Modena, Italy) and Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri (Milan, Italy); Jonathan J. Deeks, PhD, University of Birmingham (Birmingham, UK); P. J. Devereaux, MD, PhD, Departments of Medicine, Clinical Epidemiology and Biostatistics, McMaster University (Hamilton, Canada); Kay Dickersin, PhD, Johns Hopkins Bloomberg School of Public Health (Baltimore, Maryland, US); Matthias Egger, MD, Department of Social and Preventive Medicine, University of Bern (Bern, Switzerland); Edzard Ernst, MD, PhD, FRCP, FRCP(Edin), Peninsula Medical School (Exeter, UK); Peter C. Gøtzsche, MD, MSc, The Nordic Cochrane Centre (Copenhagen, Denmark); Jeremy Grimshaw, MBChB, PhD, FRCFP, Ottawa Hospital Research Institute (Ottawa, Canada); Gordon Guyatt, MD, Departments of Medicine, Clinical Epidemiology and Biostatistics, McMaster University (Hamilton, Canada); Julian Higgins, PhD, MRC Biostatistics Unit (Cambridge, UK); John P. A. Ioannidis, MD, University of Ioannina Campus (Ioannina, Greece); Jos Kleijnen, MD, PhD, Kleijnen Systematic Reviews Ltd (York, UK) and School for Public Health and Primary Care (CAPHRI), University of Maastricht (Maastricht, Netherlands); Tom Lang, MA, Tom Lang Communications and Training (Davis, California, US); Alessandro Liberati, MD, Università di Modena e Reggio Emilia (Modena, Italy) and Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri (Milan, Italy); Nicola Magrini, MD, NHS Centre for the Evaluation of the Effectiveness of Health Care – CeVEAS (Modena, Italy); David McNamee, PhD, The Lancet (London, UK); Lorenzo Moja, MD, MSc, Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri (Milan, Italy); David Moher, PhD, Ottawa Methods Centre, Ottawa Hospital Research Institute (Ottawa, Canada); Cynthia Mulrow, MD, MSc, Annals of Internal Medicine (Philadelphia, Pennsylvania, US); Maryann Napoli, Center for Medical Consumers (New York, New York, US); Andy Oxman, MD, Norwegian Health Services Research Centre (Oslo, Norway); Ba' Pham, MMath, Toronto Health Economics and Technology Assessment Collaborative (Toronto, Canada) (at the time of the first meeting of the group, GlaxoSmithKline Canada, Mississauga, Canada); Drummond Rennie, MD, FRCP, FACP, University of California San Francisco (San Francisco, California, US); Margaret Sampson, MLIS, Children's Hospital of Eastern Ontario (Ottawa, Canada); Kenneth F. Schulz, PhD, MBA, Family Health International (Durham, North Carolina, US); Paul G. Shekelle, MD, PhD, Southern California Evidence Based Practice Center (Santa Monica, California, US); Jennifer Tetzlaff, BSc, Ottawa Methods Centre, Ottawa Hospital Research Institute (Ottawa, Canada); David Tovey, FRCGP, The Cochrane Library, Cochrane Collaboration (Oxford, UK) (at the time of the first meeting of the group, BMJ , London, UK); Peter Tugwell, MD, MSc, FRCPC, Institute of Population Health, University of Ottawa (Ottawa, Canada).

Dr. Lorenzo Moja helped with the preparation and the several updates of the manuscript and assisted with the preparation of the reference list.

Alessandro Liberati is the guarantor of the manuscript.

Author Contributions

ICMJE criteria for authorship read and met: AL DGA JT CM PCG JPAI MC PJD JK DM. Wrote the first draft of the paper: AL DGA JT JPAI DM. Contributed to the writing of the paper: AL DGA JT CM PCG JPAI MC PJD JK DM. Concept and design of the Explanation and Elaboration statement: AL DGA JT DM. Agree with the recommendations: AL DGA JT CM PCG JPAI MC PJD JK DM.

• View Article
• Google Scholar
• 11. Higgins JPT, Altman DG (2008) Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.0.0 [updated February 2008]. The Cochrane Collaboration. Available: http://www.cochrane-handbook.org/ . Accessed 26 May 2009.
• 14. Helfand M, Balshem H (2009) Principles for developing guidance: AHRQ and the effective health-care program. J Clin Epidemiol. In press.
• 16. Centre for Reviews and Dissemination (2009) Systematic reviews: CRD's guidance for undertaking reviews in health care. York: University of York. Available: http://www.york.ac.uk/inst/crd/systematic_reviews_book.htm . Accessed 26 May 2009.
• 38. Brunton G, Green S, Higgins JPT, Kjeldstrøm M, Jackson N, et al. (2008) Chapter 2: Preparing a Cochrane review. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.0.0 [updated February 2008]. The Cochrane Collaboration. Available: http://www.cochrane-handbook.org/ . Accessed 26 May 2009.
• 45. Centre for Reviews and Dissemination (2009) Research projects. York: University of York. Available: http://www.crd.york.ac.uk/crdweb . Accessed 26 May 2009.
• 99. Guyatt GH, Cook D, Devereaux PJ, Meade M, Straus S (2002) Therapy. Users' guides to the medical literature. AMA Press. pp. 55–79.
• 102. Guyatt GH, Devereaux PJ (2002) Therapy and validity: The principle of intention-to-treat. In: Guyatt GH, Rennie DR, editors. Users' guides to the medical literature. AMA Press. pp. 267–273.
• 108. Deeks JJ, Altman DG (2001) Effect measures for meta-analysis of trials with binary outcomes. In: Egger M, Smith GD, Altman DG, editors. Systematic reviews in healthcare: Meta-analysis in context. 2nd edition. London: BMJ Publishing Group.
• 183. Last JM (2001) A dictionary of epidemiology. Oxford: Oxford University Press & International Epidemiological Association.
• 186. O'Connor D, Green S, Higgins JPT (2008) Chapter 5: Defining the review question and developing criteria for including studies. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.0.0 [updated February 2008]. The Cochrane Collaboration. Available: http://www.cochrane-handbook.org/ . Accessed 26 May 2009.
• 187. McDonagh M, Whiting P, Bradley M, Cooper J, Sutton A, et al. (2000) A systematic review of public water fluoridation. Protocol changes (Appendix M). NHS Centre for Reviews and Dissemination. York: University of York. Available: http://www.york.ac.uk/inst/crd/pdf/appm.pdf . Accessed 26 May 2009.
• 200. Deeks JJ, Altman DG, Bradburn MJ (2001) Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Egger M, Davey Smith G, Altman DG, editors. Systematic reviews in healthcare: Meta-analysis in context. London: BMJ Publishing Group. pp. 285–312.
• 206. Dickersin K (2005) Publication bias: Recognising the problem, understanding its origin and scope, and preventing harm. In: Rothstein HR, Sutton AJ, Borenstein M, editors. Publication bias in meta-analysis—Prevention, assessment and adjustments. West Sussex: John Wiley & Sons.
• 210. Ghersi D (2006) Issues in the design, conduct and reporting of clinical trials that impact on the quality of decision making. PhD thesis. Sydney: School of Public Health, Faculty of Medicine, University of Sydney.
• 215. Rothstein HR, Sutton AJ, Borenstein M (2005) Publication bias in meta-analysis: Prevention, assessment and adjustments. West Sussex: John Wiley & Sons.
• 220. Sterne JAC, Egger M, Moher D (2008) Chapter 10: Addressing reporting biases. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.0.0 [updated February 2008]. The Cochrane Collaboration. Available: http://www.cochrane-handbook.org/ . Accessed 26 May 2009.

A Guide for Systematic Reviews: PRISMA

• Turkish Archives of Otorhinolaryngology 57(1):57-58
• 57(1):57-58

• Ankara Numune Training and Research Hospital

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Ieman A. Aljahdali
• Yasir Babiker

• STEPHENNUS STANLEY CHIAM

• Rungroj Wongmahasiri

• Anke Oenema
• J CLEAN PROD

• Masoud Hashemi
• Jingxia Wang
• Carlos Alexandre Gomes Passarinho Menezes
• Carla Cristina Zanchin Couto
• Juliana Girardi Bruniera Salustiano
• Flávio David Haddad Filho
• Evren Erkul
• M. Boyd Gillespie

• LARYNGOSCOPE

• Bulent Satar
• Alessandro Liberati

• David Moher

• Robert L. Ferris
• Derleme Yazılar
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.372; 2021

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

Matthew j page.

1 School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia

Joanne E McKenzie

Patrick m bossuyt.

2 Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, Netherlands

Isabelle Boutron

3 Université de Paris, Centre of Epidemiology and Statistics (CRESS), Inserm, F 75004 Paris, France

Tammy C Hoffmann

4 Institute for Evidence-Based Healthcare, Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Australia

Cynthia D Mulrow

5 University of Texas Health Science Center at San Antonio, San Antonio, Texas, USA; Annals of Internal Medicine

Larissa Shamseer

6 Knowledge Translation Program, Li Ka Shing Knowledge Institute, Toronto, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

Jennifer M Tetzlaff

7 Evidence Partners, Ottawa, Canada

8 Clinical Research Institute, American University of Beirut, Beirut, Lebanon; Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada

Sue E Brennan

9 Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, USA

Julie Glanville

10 York Health Economics Consortium (YHEC Ltd), University of York, York, UK

Jeremy M Grimshaw

11 Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada; Department of Medicine, University of Ottawa, Ottawa, Canada

Asbjørn Hróbjartsson

12 Centre for Evidence-Based Medicine Odense (CEBMO) and Cochrane Denmark, Department of Clinical Research, University of Southern Denmark, Odense, Denmark; Open Patient data Exploratory Network (OPEN), Odense University Hospital, Odense, Denmark

Manoj M Lalu

13 Department of Anesthesiology and Pain Medicine, The Ottawa Hospital, Ottawa, Canada; Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Canada; Regenerative Medicine Program, Ottawa Hospital Research Institute, Ottawa, Canada

Tianjing Li

14 Department of Ophthalmology, School of Medicine, University of Colorado Denver, Denver, Colorado, United States; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA

Elizabeth W Loder

15 Division of Headache, Department of Neurology, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA; Head of Research, The BMJ , London, UK

Evan Mayo-Wilson

16 Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington, Bloomington, Indiana, USA

Steve McDonald

Luke a mcguinness.

17 Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK

Lesley A Stewart

18 Centre for Reviews and Dissemination, University of York, York, UK

James Thomas

19 EPPI-Centre, UCL Social Research Institute, University College London, London, UK

Andrea C Tricco

20 Li Ka Shing Knowledge Institute of St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Epidemiology Division of the Dalla Lana School of Public Health and the Institute of Health Management, Policy, and Evaluation, University of Toronto, Toronto, Canada; Queen's Collaboration for Health Care Quality Joanna Briggs Institute Centre of Excellence, Queen's University, Kingston, Canada

Vivian A Welch

21 Methods Centre, Bruyère Research Institute, Ottawa, Ontario, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

Penny Whiting

David moher.

22 Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

Associated Data

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.

Systematic reviews serve many critical roles. They can provide syntheses of the state of knowledge in a field, from which future research priorities can be identified; they can address questions that otherwise could not be answered by individual studies; they can identify problems in primary research that should be rectified in future studies; and they can generate or evaluate theories about how or why phenomena occur. Systematic reviews therefore generate various types of knowledge for different users of reviews (such as patients, healthcare providers, researchers, and policy makers). 1 2 To ensure a systematic review is valuable to users, authors should prepare a transparent, complete, and accurate account of why the review was done, what they did (such as how studies were identified and selected) and what they found (such as characteristics of contributing studies and results of meta-analyses). Up-to-date reporting guidance facilitates authors achieving this. 3

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement published in 2009 (hereafter referred to as PRISMA 2009) 4 5 6 7 8 9 10 is a reporting guideline designed to address poor reporting of systematic reviews. 11 The PRISMA 2009 statement comprised a checklist of 27 items recommended for reporting in systematic reviews and an “explanation and elaboration” paper 12 13 14 15 16 providing additional reporting guidance for each item, along with exemplars of reporting. The recommendations have been widely endorsed and adopted, as evidenced by its co-publication in multiple journals, citation in over 60 000 reports (Scopus, August 2020), endorsement from almost 200 journals and systematic review organisations, and adoption in various disciplines. Evidence from observational studies suggests that use of the PRISMA 2009 statement is associated with more complete reporting of systematic reviews, 17 18 19 20 although more could be done to improve adherence to the guideline. 21

Many innovations in the conduct of systematic reviews have occurred since publication of the PRISMA 2009 statement. For example, technological advances have enabled the use of natural language processing and machine learning to identify relevant evidence, 22 23 24 methods have been proposed to synthesise and present findings when meta-analysis is not possible or appropriate, 25 26 27 and new methods have been developed to assess the risk of bias in results of included studies. 28 29 Evidence on sources of bias in systematic reviews has accrued, culminating in the development of new tools to appraise the conduct of systematic reviews. 30 31 Terminology used to describe particular review processes has also evolved, as in the shift from assessing “quality” to assessing “certainty” in the body of evidence. 32 In addition, the publishing landscape has transformed, with multiple avenues now available for registering and disseminating systematic review protocols, 33 34 disseminating reports of systematic reviews, and sharing data and materials, such as preprint servers and publicly accessible repositories. To capture these advances in the reporting of systematic reviews necessitated an update to the PRISMA 2009 statement.

Summary points

• To ensure a systematic review is valuable to users, authors should prepare a transparent, complete, and accurate account of why the review was done, what they did, and what they found
• The PRISMA 2020 statement provides updated reporting guidance for systematic reviews that reflects advances in methods to identify, select, appraise, and synthesise studies
• The PRISMA 2020 statement consists of a 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and revised flow diagrams for original and updated reviews
• We anticipate that the PRISMA 2020 statement will benefit authors, editors, and peer reviewers of systematic reviews, and different users of reviews, including guideline developers, policy makers, healthcare providers, patients, and other stakeholders

Development of PRISMA 2020

A complete description of the methods used to develop PRISMA 2020 is available elsewhere. 35 We identified PRISMA 2009 items that were often reported incompletely by examining the results of studies investigating the transparency of reporting of published reviews. 17 21 36 37 We identified possible modifications to the PRISMA 2009 statement by reviewing 60 documents providing reporting guidance for systematic reviews (including reporting guidelines, handbooks, tools, and meta-research studies). 38 These reviews of the literature were used to inform the content of a survey with suggested possible modifications to the 27 items in PRISMA 2009 and possible additional items. Respondents were asked whether they believed we should keep each PRISMA 2009 item as is, modify it, or remove it, and whether we should add each additional item. Systematic review methodologists and journal editors were invited to complete the online survey (110 of 220 invited responded). We discussed proposed content and wording of the PRISMA 2020 statement, as informed by the review and survey results, at a 21-member, two-day, in-person meeting in September 2018 in Edinburgh, Scotland. Throughout 2019 and 2020, we circulated an initial draft and five revisions of the checklist and explanation and elaboration paper to co-authors for feedback. In April 2020, we invited 22 systematic reviewers who had expressed interest in providing feedback on the PRISMA 2020 checklist to share their views (via an online survey) on the layout and terminology used in a preliminary version of the checklist. Feedback was received from 15 individuals and considered by the first author, and any revisions deemed necessary were incorporated before the final version was approved and endorsed by all co-authors.

The PRISMA 2020 statement

Scope of the guideline.

The PRISMA 2020 statement has been designed primarily for systematic reviews of studies that evaluate the effects of health interventions, irrespective of the design of the included studies. However, the checklist items are applicable to reports of systematic reviews evaluating other interventions (such as social or educational interventions), and many items are applicable to systematic reviews with objectives other than evaluating interventions (such as evaluating aetiology, prevalence, or prognosis). PRISMA 2020 is intended for use in systematic reviews that include synthesis (such as pairwise meta-analysis or other statistical synthesis methods) or do not include synthesis (for example, because only one eligible study is identified). The PRISMA 2020 items are relevant for mixed-methods systematic reviews (which include quantitative and qualitative studies), but reporting guidelines addressing the presentation and synthesis of qualitative data should also be consulted. 39 40 PRISMA 2020 can be used for original systematic reviews, updated systematic reviews, or continually updated (“living”) systematic reviews. However, for updated and living systematic reviews, there may be some additional considerations that need to be addressed. Where there is relevant content from other reporting guidelines, we reference these guidelines within the items in the explanation and elaboration paper 41 (such as PRISMA-Search 42 in items 6 and 7, Synthesis without meta-analysis (SWiM) reporting guideline 27 in item 13d). Box 1 includes a glossary of terms used throughout the PRISMA 2020 statement.

Glossary of terms

• Systematic review —A review that uses explicit, systematic methods to collate and synthesise findings of studies that address a clearly formulated question 43
• Statistical synthesis —The combination of quantitative results of two or more studies. This encompasses meta-analysis of effect estimates (described below) and other methods, such as combining P values, calculating the range and distribution of observed effects, and vote counting based on the direction of effect (see McKenzie and Brennan 25 for a description of each method)
• Meta-analysis of effect estimates —A statistical technique used to synthesise results when study effect estimates and their variances are available, yielding a quantitative summary of results 25
• Outcome —An event or measurement collected for participants in a study (such as quality of life, mortality)
• Result —The combination of a point estimate (such as a mean difference, risk ratio, or proportion) and a measure of its precision (such as a confidence/credible interval) for a particular outcome
• Report —A document (paper or electronic) supplying information about a particular study. It could be a journal article, preprint, conference abstract, study register entry, clinical study report, dissertation, unpublished manuscript, government report, or any other document providing relevant information
• Record —The title or abstract (or both) of a report indexed in a database or website (such as a title or abstract for an article indexed in Medline). Records that refer to the same report (such as the same journal article) are “duplicates”; however, records that refer to reports that are merely similar (such as a similar abstract submitted to two different conferences) should be considered unique.
• Study —An investigation, such as a clinical trial, that includes a defined group of participants and one or more interventions and outcomes. A “study” might have multiple reports. For example, reports could include the protocol, statistical analysis plan, baseline characteristics, results for the primary outcome, results for harms, results for secondary outcomes, and results for additional mediator and moderator analyses

PRISMA 2020 is not intended to guide systematic review conduct, for which comprehensive resources are available. 43 44 45 46 However, familiarity with PRISMA 2020 is useful when planning and conducting systematic reviews to ensure that all recommended information is captured. PRISMA 2020 should not be used to assess the conduct or methodological quality of systematic reviews; other tools exist for this purpose. 30 31 Furthermore, PRISMA 2020 is not intended to inform the reporting of systematic review protocols, for which a separate statement is available (PRISMA for Protocols (PRISMA-P) 2015 statement 47 48 ). Finally, extensions to the PRISMA 2009 statement have been developed to guide reporting of network meta-analyses, 49 meta-analyses of individual participant data, 50 systematic reviews of harms, 51 systematic reviews of diagnostic test accuracy studies, 52 and scoping reviews 53 ; for these types of reviews we recommend authors report their review in accordance with the recommendations in PRISMA 2020 along with the guidance specific to the extension.

How to use PRISMA 2020

The PRISMA 2020 statement (including the checklists, explanation and elaboration, and flow diagram) replaces the PRISMA 2009 statement, which should no longer be used. Box 2 summarises noteworthy changes from the PRISMA 2009 statement. The PRISMA 2020 checklist includes seven sections with 27 items, some of which include sub-items ( table 1 ). A checklist for journal and conference abstracts for systematic reviews is included in PRISMA 2020. This abstract checklist is an update of the 2013 PRISMA for Abstracts statement, 54 reflecting new and modified content in PRISMA 2020 ( table 2 ). A template PRISMA flow diagram is provided, which can be modified depending on whether the systematic review is original or updated ( fig 1 ).

Noteworthy changes to the PRISMA 2009 statement

• Inclusion of the abstract reporting checklist within PRISMA 2020 (see item #2 and table 2 ).
• Movement of the ‘Protocol and registration’ item from the start of the Methods section of the checklist to a new Other section, with addition of a sub-item recommending authors describe amendments to information provided at registration or in the protocol (see item #24a-24c).
• Modification of the ‘Search’ item to recommend authors present full search strategies for all databases, registers and websites searched, not just at least one database (see item #7).
• Modification of the ‘Study selection’ item in the Methods section to emphasise the reporting of how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process (see item #8).
• Addition of a sub-item to the ‘Data items’ item recommending authors report how outcomes were defined, which results were sought, and methods for selecting a subset of results from included studies (see item #10a).
• Splitting of the ‘Synthesis of results’ item in the Methods section into six sub-items recommending authors describe: the processes used to decide which studies were eligible for each synthesis; any methods required to prepare the data for synthesis; any methods used to tabulate or visually display results of individual studies and syntheses; any methods used to synthesise results; any methods used to explore possible causes of heterogeneity among study results (such as subgroup analysis, meta-regression); and any sensitivity analyses used to assess robustness of the synthesised results (see item #13a-13f).
• Addition of a sub-item to the ‘Study selection’ item in the Results section recommending authors cite studies that might appear to meet the inclusion criteria, but which were excluded, and explain why they were excluded (see item #16b).
• Splitting of the ‘Synthesis of results’ item in the Results section into four sub-items recommending authors: briefly summarise the characteristics and risk of bias among studies contributing to the synthesis; present results of all statistical syntheses conducted; present results of any investigations of possible causes of heterogeneity among study results; and present results of any sensitivity analyses (see item #20a-20d).
• Addition of new items recommending authors report methods for and results of an assessment of certainty (or confidence) in the body of evidence for an outcome (see items #15 and #22).
• Addition of a new item recommending authors declare any competing interests (see item #26).
• Addition of a new item recommending authors indicate whether data, analytic code and other materials used in the review are publicly available and if so, where they can be found (see item #27).

PRISMA 2020 item checklist

PRISMA 2020 for Abstracts checklist*

PRISMA 2020 flow diagram template for systematic reviews. The new design is adapted from flow diagrams proposed by Boers, 55 Mayo-Wilson et al. 56 and Stovold et al. 57 The boxes in grey should only be completed if applicable; otherwise they should be removed from the flow diagram. Note that a “report” could be a journal article, preprint, conference abstract, study register entry, clinical study report, dissertation, unpublished manuscript, government report or any other document providing relevant information.

We recommend authors refer to PRISMA 2020 early in the writing process, because prospective consideration of the items may help to ensure that all the items are addressed. To help keep track of which items have been reported, the PRISMA statement website ( http://www.prisma-statement.org/ ) includes fillable templates of the checklists to download and complete (also available in the data supplement on bmj.com). We have also created a web application that allows users to complete the checklist via a user-friendly interface 58 (available at https://prisma.shinyapps.io/checklist/ and adapted from the Transparency Checklist app 59 ). The completed checklist can be exported to Word or PDF. Editable templates of the flow diagram can also be downloaded from the PRISMA statement website.

We have prepared an updated explanation and elaboration paper, in which we explain why reporting of each item is recommended and present bullet points that detail the reporting recommendations (which we refer to as elements). 41 The bullet-point structure is new to PRISMA 2020 and has been adopted to facilitate implementation of the guidance. 60 61 An expanded checklist, which comprises an abridged version of the elements presented in the explanation and elaboration paper, with references and some examples removed, is available in the data supplement on bmj.com. Consulting the explanation and elaboration paper is recommended if further clarity or information is required.

Journals and publishers might impose word and section limits, and limits on the number of tables and figures allowed in the main report. In such cases, if the relevant information for some items already appears in a publicly accessible review protocol, referring to the protocol may suffice. Alternatively, placing detailed descriptions of the methods used or additional results (such as for less critical outcomes) in supplementary files is recommended. Ideally, supplementary files should be deposited to a general-purpose or institutional open-access repository that provides free and permanent access to the material (such as Open Science Framework, Dryad, figshare). A reference or link to the additional information should be included in the main report. Finally, although PRISMA 2020 provides a template for where information might be located, the suggested location should not be seen as prescriptive; the guiding principle is to ensure the information is reported.

Use of PRISMA 2020 has the potential to benefit many stakeholders. Complete reporting allows readers to assess the appropriateness of the methods, and therefore the trustworthiness of the findings. Presenting and summarising characteristics of studies contributing to a synthesis allows healthcare providers and policy makers to evaluate the applicability of the findings to their setting. Describing the certainty in the body of evidence for an outcome and the implications of findings should help policy makers, managers, and other decision makers formulate appropriate recommendations for practice or policy. Complete reporting of all PRISMA 2020 items also facilitates replication and review updates, as well as inclusion of systematic reviews in overviews (of systematic reviews) and guidelines, so teams can leverage work that is already done and decrease research waste. 36 62 63

We updated the PRISMA 2009 statement by adapting the EQUATOR Network’s guidance for developing health research reporting guidelines. 64 We evaluated the reporting completeness of published systematic reviews, 17 21 36 37 reviewed the items included in other documents providing guidance for systematic reviews, 38 surveyed systematic review methodologists and journal editors for their views on how to revise the original PRISMA statement, 35 discussed the findings at an in-person meeting, and prepared this document through an iterative process. Our recommendations are informed by the reviews and survey conducted before the in-person meeting, theoretical considerations about which items facilitate replication and help users assess the risk of bias and applicability of systematic reviews, and co-authors’ experience with authoring and using systematic reviews.

Various strategies to increase the use of reporting guidelines and improve reporting have been proposed. They include educators introducing reporting guidelines into graduate curricula to promote good reporting habits of early career scientists 65 ; journal editors and regulators endorsing use of reporting guidelines 18 ; peer reviewers evaluating adherence to reporting guidelines 61 66 ; journals requiring authors to indicate where in their manuscript they have adhered to each reporting item 67 ; and authors using online writing tools that prompt complete reporting at the writing stage. 60 Multi-pronged interventions, where more than one of these strategies are combined, may be more effective (such as completion of checklists coupled with editorial checks). 68 However, of 31 interventions proposed to increase adherence to reporting guidelines, the effects of only 11 have been evaluated, mostly in observational studies at high risk of bias due to confounding. 69 It is therefore unclear which strategies should be used. Future research might explore barriers and facilitators to the use of PRISMA 2020 by authors, editors, and peer reviewers, designing interventions that address the identified barriers, and evaluating those interventions using randomised trials. To inform possible revisions to the guideline, it would also be valuable to conduct think-aloud studies 70 to understand how systematic reviewers interpret the items, and reliability studies to identify items where there is varied interpretation of the items.

We encourage readers to submit evidence that informs any of the recommendations in PRISMA 2020 (via the PRISMA statement website: http://www.prisma-statement.org/ ). To enhance accessibility of PRISMA 2020, several translations of the guideline are under way (see available translations at the PRISMA statement website). We encourage journal editors and publishers to raise awareness of PRISMA 2020 (for example, by referring to it in journal “Instructions to authors”), endorsing its use, advising editors and peer reviewers to evaluate submitted systematic reviews against the PRISMA 2020 checklists, and making changes to journal policies to accommodate the new reporting recommendations. We recommend existing PRISMA extensions 47 49 50 51 52 53 71 72 be updated to reflect PRISMA 2020 and advise developers of new PRISMA extensions to use PRISMA 2020 as the foundation document.

We anticipate that the PRISMA 2020 statement will benefit authors, editors, and peer reviewers of systematic reviews, and different users of reviews, including guideline developers, policy makers, healthcare providers, patients, and other stakeholders. Ultimately, we hope that uptake of the guideline will lead to more transparent, complete, and accurate reporting of systematic reviews, thus facilitating evidence based decision making.

Acknowledgments

We dedicate this paper to the late Douglas G Altman and Alessandro Liberati, whose contributions were fundamental to the development and implementation of the original PRISMA statement.

We thank the following contributors who completed the survey to inform discussions at the development meeting: Xavier Armoiry, Edoardo Aromataris, Ana Patricia Ayala, Ethan M Balk, Virginia Barbour, Elaine Beller, Jesse A Berlin, Lisa Bero, Zhao-Xiang Bian, Jean Joel Bigna, Ferrán Catalá-López, Anna Chaimani, Mike Clarke, Tammy Clifford, Ioana A Cristea, Miranda Cumpston, Sofia Dias, Corinna Dressler, Ivan D Florez, Joel J Gagnier, Chantelle Garritty, Long Ge, Davina Ghersi, Sean Grant, Gordon Guyatt, Neal R Haddaway, Julian PT Higgins, Sally Hopewell, Brian Hutton, Jamie J Kirkham, Jos Kleijnen, Julia Koricheva, Joey SW Kwong, Toby J Lasserson, Julia H Littell, Yoon K Loke, Malcolm R Macleod, Chris G Maher, Ana Marušic, Dimitris Mavridis, Jessie McGowan, Matthew DF McInnes, Philippa Middleton, Karel G Moons, Zachary Munn, Jane Noyes, Barbara Nußbaumer-Streit, Donald L Patrick, Tatiana Pereira-Cenci, Ba’ Pham, Bob Phillips, Dawid Pieper, Michelle Pollock, Daniel S Quintana, Drummond Rennie, Melissa L Rethlefsen, Hannah R Rothstein, Maroeska M Rovers, Rebecca Ryan, Georgia Salanti, Ian J Saldanha, Margaret Sampson, Nancy Santesso, Rafael Sarkis-Onofre, Jelena Savović, Christopher H Schmid, Kenneth F Schulz, Guido Schwarzer, Beverley J Shea, Paul G Shekelle, Farhad Shokraneh, Mark Simmonds, Nicole Skoetz, Sharon E Straus, Anneliese Synnot, Emily E Tanner-Smith, Brett D Thombs, Hilary Thomson, Alexander Tsertsvadze, Peter Tugwell, Tari Turner, Lesley Uttley, Jeffrey C Valentine, Matt Vassar, Areti Angeliki Veroniki, Meera Viswanathan, Cole Wayant, Paul Whaley, and Kehu Yang. We thank the following contributors who provided feedback on a preliminary version of the PRISMA 2020 checklist: Jo Abbott, Fionn Büttner, Patricia Correia-Santos, Victoria Freeman, Emily A Hennessy, Rakibul Islam, Amalia (Emily) Karahalios, Kasper Krommes, Andreas Lundh, Dafne Port Nascimento, Davina Robson, Catherine Schenck-Yglesias, Mary M Scott, Sarah Tanveer and Pavel Zhelnov. We thank Abigail H Goben, Melissa L Rethlefsen, Tanja Rombey, Anna Scott, and Farhad Shokraneh for their helpful comments on the preprints of the PRISMA 2020 papers. We thank Edoardo Aromataris, Stephanie Chang, Toby Lasserson and David Schriger for their helpful peer review comments on the PRISMA 2020 papers.

Web Extra. 

Extra material supplied by the author

PRISMA 2020 checklist

PRISMA 2020 expanded checklist

Contributors: JEM and DM are joint senior authors. MJP, JEM, PMB, IB, TCH, CDM, LS, and DM conceived this paper and designed the literature review and survey conducted to inform the guideline content. MJP conducted the literature review, administered the survey and analysed the data for both. MJP prepared all materials for the development meeting. MJP and JEM presented proposals at the development meeting. All authors except for TCH, JMT, EAA, SEB, and LAM attended the development meeting. MJP and JEM took and consolidated notes from the development meeting. MJP and JEM led the drafting and editing of the article. JEM, PMB, IB, TCH, LS, JMT, EAA, SEB, RC, JG, AH, TL, EMW, SM, LAM, LAS, JT, ACT, PW, and DM drafted particular sections of the article. All authors were involved in revising the article critically for important intellectual content. All authors approved the final version of the article. MJP is the guarantor of this work. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: There was no direct funding for this research. MJP is supported by an Australian Research Council Discovery Early Career Researcher Award (DE200101618) and was previously supported by an Australian National Health and Medical Research Council (NHMRC) Early Career Fellowship (1088535) during the conduct of this research. JEM is supported by an Australian NHMRC Career Development Fellowship (1143429). TCH is supported by an Australian NHMRC Senior Research Fellowship (1154607). JMT is supported by Evidence Partners Inc. JMG is supported by a Tier 1 Canada Research Chair in Health Knowledge Transfer and Uptake. MML is supported by The Ottawa Hospital Anaesthesia Alternate Funds Association and a Faculty of Medicine Junior Research Chair. TL is supported by funding from the National Eye Institute (UG1EY020522), National Institutes of Health, United States. LAM is supported by a National Institute for Health Research Doctoral Research Fellowship (DRF-2018-11-ST2-048). ACT is supported by a Tier 2 Canada Research Chair in Knowledge Synthesis. DM is supported in part by a University Research Chair, University of Ottawa. The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

Competing interests: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/conflicts-of-interest/ and declare: EL is head of research for the BMJ ; MJP is an editorial board member for PLOS Medicine ; ACT is an associate editor and MJP, TL, EMW, and DM are editorial board members for the Journal of Clinical Epidemiology ; DM and LAS were editors in chief, LS, JMT, and ACT are associate editors, and JG is an editorial board member for Systematic Reviews . None of these authors were involved in the peer review process or decision to publish. TCH has received personal fees from Elsevier outside the submitted work. EMW has received personal fees from the American Journal for Public Health , for which he is the editor for systematic reviews. VW is editor in chief of the Campbell Collaboration, which produces systematic reviews, and co-convenor of the Campbell and Cochrane equity methods group. DM is chair of the EQUATOR Network, IB is adjunct director of the French EQUATOR Centre and TCH is co-director of the Australasian EQUATOR Centre, which advocates for the use of reporting guidelines to improve the quality of reporting in research articles. JMT received salary from Evidence Partners, creator of DistillerSR software for systematic reviews; Evidence Partners was not involved in the design or outcomes of the statement, and the views expressed solely represent those of the author.

Provenance and peer review: Not commissioned; externally peer reviewed.

Patient and public involvement: Patients and the public were not involved in this methodological research. We plan to disseminate the research widely, including to community participants in evidence synthesis organisations.

Systematic Reviews & Expert Reviews

• What are Expert Reviews?
• What Type of Review Should I Choose?
• Setting the Groundwork
• Finding the Evidence
• Selecting the Evidence
• Study Appraisal
• Data Extraction & Abstraction
• Reporting Your Findings
• PRISMA Help
• Systematic Review Toolbox
• Librarian Assistance

The PRISMA Flow Chart is a process showing how you searched for and then filtered down your search results. This process is used when conducting a systematic review, when you want your search process to be transparent. The general process looks something like this:

• Search for articles, and then remove any duplicates.
• Look at the articles titles and abstracts, exclude articles based on your screening criteria (you decide what this criteria is).
• Retrieve the full text of your remaining records. If you are unable to get the full-text of any articles, you would document this.
• Read your remaining full-text articles, and then run them through your exclusionary criteria again. 
• The remaining articles are the ones you will be using for the study.

So let's break down the flow chart, line by line.

For the Identification section , you will run your search using your search strategy (including any keywords, subject headings and synonyms)

• Databases "n" = the number of reports you found in databases. 
• Registers include clinical data registries -- these have information about studies, but might not include a traditional full-text article (like you would find in a database). An example of a registry would be ClinicalTrials.gov. 
• For the Records removed before screening  section, you'll want to remove any articles from your list that are automatic "no's." You will remove any duplicates from your results, as well as articles that you can immediately tell don't fit your base parameters. At this point in the process, you will be removing articles from consideration by using filters (i.e., articles that are over 5 years old, are in a foreign language, etc.); you won't need to dig into the article abstract yet (that comes later).

For the Screening section , you will be digging further into the records to decide whether it is worth keeping. 

• Records screened refers to the new number of results you are working with; this is the number of records  minus  the records excluded in the last step.
• In the  Records excluded  step, you review the abstracts of each citation and think about what factors would make an article ineligible for your research question.
• Some examples of exclusionary criteria could include: studies that upon closer investigation are covering the wrong population, are outdated, not peer reviewed, are written in a foreign language, are not the correct study type or have a poor design, etc.
• These exclusionary factors are up to you to decide, and should be based on your specific research question. 
• Records sought for retrieval  is your new total (records screened  minus  the records you excluded). This is your list of eligible studies. At this point, you will need to retrieve and read the full text of each article in your list.
• Reports not retrieved is the total number of articles in which you are unable to get the full text for your review.
• Reports assessed for eligibility  is the new list you have ( Reports sought for retrieval  minus   Reports not retrieved ).
• After reading the full text of the articles, you will list your reasoning for any final exclusions in the  Reports excluded  section. Again, the exclusionary criteria is up to you to decide, and is based on the question you are asking.

The last section ( Included) refers to the total number of articles you will be using.

• Studies included in review (n=15)
• Reports included in review (n=13)

• The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

• << Previous: Reporting Your Findings
• Next: Systematic Review Toolbox >>
• Last Updated: Sep 11, 2024 2:00 PM
• URL: https://researchguides.gonzaga.edu/systematic-reviews

• UofGH Library Services
• Research Guides

Systematic Reviews

• PRISMA Diagram & Checklist
• PubMed: Create a Systematic Search

PRISMA Templates

Download these fillable templates to include the PRISMA Flow Diagram and Checklist in your systematic review.

What is PRISMA?

PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.

The PRISMA statement consists of a 27-item checklist and a 4-phase flow diagram. These items have been adapted for use by students conducting systematic reviews as part of the course requirements for KIN 4400. 

For more information, consult the PRISMA Explanation and Elaboration document.

Why PRISMA?

PRISMA is the recognized standard for reporting evidence in systematic reviews and meta-analyses. The standards are endorsed by organizations and journals in the health sciences.

Benefits of using PRISMA

• Demonstrate quality of the review
• Allow readers to assess strengths and weaknesses
• Permits replication of review methods
• Structure and format the review using PRISMA headings
• << Previous: Start Here
• Next: PubMed: Create a Systematic Search >>
• Last Updated: Sep 2, 2024 1:39 PM
• URL: https://guelphhumber.libguides.com/systematic-reviews

• Open access
• Published: 29 March 2021

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

• Matthew J. Page   ORCID: orcid.org/0000-0002-4242-7526 1 ,
• Joanne E. McKenzie 1 ,
• Patrick M. Bossuyt 2 ,
• Isabelle Boutron 3 ,
• Tammy C. Hoffmann 4 ,
• Cynthia D. Mulrow 5 ,
• Larissa Shamseer 6 ,
• Jennifer M. Tetzlaff 7 ,
• Elie A. Akl 8 ,
• Sue E. Brennan 1 ,
• Roger Chou 9 ,
• Julie Glanville 10 ,
• Jeremy M. Grimshaw 11 ,
• Asbjørn Hróbjartsson 12 ,
• Manoj M. Lalu 13 ,
• Tianjing Li 14 ,
• Elizabeth W. Loder 15 ,
• Evan Mayo-Wilson 16 ,
• Steve McDonald 1 ,
• Luke A. McGuinness 17 ,
• Lesley A. Stewart 18 ,
• James Thomas 19 ,
• Andrea C. Tricco 20 ,
• Vivian A. Welch 21 ,
• Penny Whiting 17 &
• David Moher 22  

Systematic Reviews volume  10 , Article number:  89 ( 2021 ) Cite this article

327k Accesses

4049 Citations

111 Altmetric

Metrics details

An Editorial to this article was published on 19 April 2021

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews. In order to encourage its wide dissemination this article is freely accessible on BMJ, PLOS Medicine, Journal of Clinical Epidemiology and International Journal of Surgery journal websites.

Systematic reviews serve many critical roles. They can provide syntheses of the state of knowledge in a field, from which future research priorities can be identified; they can address questions that otherwise could not be answered by individual studies; they can identify problems in primary research that should be rectified in future studies; and they can generate or evaluate theories about how or why phenomena occur. Systematic reviews therefore generate various types of knowledge for different users of reviews (such as patients, healthcare providers, researchers, and policy makers) [ 1 , 2 ]. To ensure a systematic review is valuable to users, authors should prepare a transparent, complete, and accurate account of why the review was done, what they did (such as how studies were identified and selected) and what they found (such as characteristics of contributing studies and results of meta-analyses). Up-to-date reporting guidance facilitates authors achieving this [ 3 ].

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement published in 2009 (hereafter referred to as PRISMA 2009) [ 4 , 5 , 6 , 7 , 8 , 9 , 10 ] is a reporting guideline designed to address poor reporting of systematic reviews [ 11 ]. The PRISMA 2009 statement comprised a checklist of 27 items recommended for reporting in systematic reviews and an “explanation and elaboration” paper [ 12 , 13 , 14 , 15 , 16 ] providing additional reporting guidance for each item, along with exemplars of reporting. The recommendations have been widely endorsed and adopted, as evidenced by its co-publication in multiple journals, citation in over 60,000 reports (Scopus, August 2020), endorsement from almost 200 journals and systematic review organisations, and adoption in various disciplines. Evidence from observational studies suggests that use of the PRISMA 2009 statement is associated with more complete reporting of systematic reviews [ 17 , 18 , 19 , 20 ], although more could be done to improve adherence to the guideline [ 21 ].

Many innovations in the conduct of systematic reviews have occurred since publication of the PRISMA 2009 statement. For example, technological advances have enabled the use of natural language processing and machine learning to identify relevant evidence [ 22 , 23 , 24 ], methods have been proposed to synthesise and present findings when meta-analysis is not possible or appropriate [ 25 , 26 , 27 ], and new methods have been developed to assess the risk of bias in results of included studies [ 28 , 29 ]. Evidence on sources of bias in systematic reviews has accrued, culminating in the development of new tools to appraise the conduct of systematic reviews [ 30 , 31 ]. Terminology used to describe particular review processes has also evolved, as in the shift from assessing “quality” to assessing “certainty” in the body of evidence [ 32 ]. In addition, the publishing landscape has transformed, with multiple avenues now available for registering and disseminating systematic review protocols [ 33 , 34 ], disseminating reports of systematic reviews, and sharing data and materials, such as preprint servers and publicly accessible repositories. To capture these advances in the reporting of systematic reviews necessitated an update to the PRISMA 2009 statement.

Development of PRISMA 2020

A complete description of the methods used to develop PRISMA 2020 is available elsewhere [ 35 ]. We identified PRISMA 2009 items that were often reported incompletely by examining the results of studies investigating the transparency of reporting of published reviews [ 17 , 21 , 36 , 37 ]. We identified possible modifications to the PRISMA 2009 statement by reviewing 60 documents providing reporting guidance for systematic reviews (including reporting guidelines, handbooks, tools, and meta-research studies) [ 38 ]. These reviews of the literature were used to inform the content of a survey with suggested possible modifications to the 27 items in PRISMA 2009 and possible additional items. Respondents were asked whether they believed we should keep each PRISMA 2009 item as is, modify it, or remove it, and whether we should add each additional item. Systematic review methodologists and journal editors were invited to complete the online survey (110 of 220 invited responded). We discussed proposed content and wording of the PRISMA 2020 statement, as informed by the review and survey results, at a 21-member, two-day, in-person meeting in September 2018 in Edinburgh, Scotland. Throughout 2019 and 2020, we circulated an initial draft and five revisions of the checklist and explanation and elaboration paper to co-authors for feedback. In April 2020, we invited 22 systematic reviewers who had expressed interest in providing feedback on the PRISMA 2020 checklist to share their views (via an online survey) on the layout and terminology used in a preliminary version of the checklist. Feedback was received from 15 individuals and considered by the first author, and any revisions deemed necessary were incorporated before the final version was approved and endorsed by all co-authors.

The PRISMA 2020 statement

Scope of the guideline.

The PRISMA 2020 statement has been designed primarily for systematic reviews of studies that evaluate the effects of health interventions, irrespective of the design of the included studies. However, the checklist items are applicable to reports of systematic reviews evaluating other interventions (such as social or educational interventions), and many items are applicable to systematic reviews with objectives other than evaluating interventions (such as evaluating aetiology, prevalence, or prognosis). PRISMA 2020 is intended for use in systematic reviews that include synthesis (such as pairwise meta-analysis or other statistical synthesis methods) or do not include synthesis (for example, because only one eligible study is identified). The PRISMA 2020 items are relevant for mixed-methods systematic reviews (which include quantitative and qualitative studies), but reporting guidelines addressing the presentation and synthesis of qualitative data should also be consulted [ 39 , 40 ]. PRISMA 2020 can be used for original systematic reviews, updated systematic reviews, or continually updated (“living”) systematic reviews. However, for updated and living systematic reviews, there may be some additional considerations that need to be addressed. Where there is relevant content from other reporting guidelines, we reference these guidelines within the items in the explanation and elaboration paper [ 41 ] (such as PRISMA-Search [ 42 ] in items 6 and 7, Synthesis without meta-analysis (SWiM) reporting guideline [ 27 ] in item 13d). Box 1 includes a glossary of terms used throughout the PRISMA 2020 statement.

PRISMA 2020 is not intended to guide systematic review conduct, for which comprehensive resources are available [ 43 , 44 , 45 , 46 ]. However, familiarity with PRISMA 2020 is useful when planning and conducting systematic reviews to ensure that all recommended information is captured. PRISMA 2020 should not be used to assess the conduct or methodological quality of systematic reviews; other tools exist for this purpose [ 30 , 31 ]. Furthermore, PRISMA 2020 is not intended to inform the reporting of systematic review protocols, for which a separate statement is available (PRISMA for Protocols (PRISMA-P) 2015 statement [ 47 , 48 ]). Finally, extensions to the PRISMA 2009 statement have been developed to guide reporting of network meta-analyses [ 49 ], meta-analyses of individual participant data [ 50 ], systematic reviews of harms [ 51 ], systematic reviews of diagnostic test accuracy studies [ 52 ], and scoping reviews [ 53 ]; for these types of reviews we recommend authors report their review in accordance with the recommendations in PRISMA 2020 along with the guidance specific to the extension.

How to use PRISMA 2020

The PRISMA 2020 statement (including the checklists, explanation and elaboration, and flow diagram) replaces the PRISMA 2009 statement, which should no longer be used. Box  2 summarises noteworthy changes from the PRISMA 2009 statement. The PRISMA 2020 checklist includes seven sections with 27 items, some of which include sub-items (Table  1 ). A checklist for journal and conference abstracts for systematic reviews is included in PRISMA 2020. This abstract checklist is an update of the 2013 PRISMA for Abstracts statement [ 54 ], reflecting new and modified content in PRISMA 2020 (Table  2 ). A template PRISMA flow diagram is provided, which can be modified depending on whether the systematic review is original or updated (Fig.  1 ).

 PRISMA 2020 flow diagram template for systematic reviews. The new design is adapted from flow diagrams proposed by Boers [ 55 ], Mayo-Wilson et al. [ 56 ] and Stovold et al. [ 57 ] The boxes in grey should only be completed if applicable; otherwise they should be removed from the flow diagram. Note that a “report” could be a journal article, preprint, conference abstract, study register entry, clinical study report, dissertation, unpublished manuscript, government report or any other document providing relevant information

We recommend authors refer to PRISMA 2020 early in the writing process, because prospective consideration of the items may help to ensure that all the items are addressed. To help keep track of which items have been reported, the PRISMA statement website ( http://www.prisma-statement.org/ ) includes fillable templates of the checklists to download and complete (also available in Additional file 1 ). We have also created a web application that allows users to complete the checklist via a user-friendly interface [ 58 ] (available at https://prisma.shinyapps.io/checklist/ and adapted from the Transparency Checklist app [ 59 ]). The completed checklist can be exported to Word or PDF. Editable templates of the flow diagram can also be downloaded from the PRISMA statement website.

We have prepared an updated explanation and elaboration paper, in which we explain why reporting of each item is recommended and present bullet points that detail the reporting recommendations (which we refer to as elements) [ 41 ]. The bullet-point structure is new to PRISMA 2020 and has been adopted to facilitate implementation of the guidance [ 60 , 61 ]. An expanded checklist, which comprises an abridged version of the elements presented in the explanation and elaboration paper, with references and some examples removed, is available in Additional file 2 . Consulting the explanation and elaboration paper is recommended if further clarity or information is required.

Journals and publishers might impose word and section limits, and limits on the number of tables and figures allowed in the main report. In such cases, if the relevant information for some items already appears in a publicly accessible review protocol, referring to the protocol may suffice. Alternatively, placing detailed descriptions of the methods used or additional results (such as for less critical outcomes) in supplementary files is recommended. Ideally, supplementary files should be deposited to a general-purpose or institutional open-access repository that provides free and permanent access to the material (such as Open Science Framework, Dryad, figshare). A reference or link to the additional information should be included in the main report. Finally, although PRISMA 2020 provides a template for where information might be located, the suggested location should not be seen as prescriptive; the guiding principle is to ensure the information is reported.

Use of PRISMA 2020 has the potential to benefit many stakeholders. Complete reporting allows readers to assess the appropriateness of the methods, and therefore the trustworthiness of the findings. Presenting and summarising characteristics of studies contributing to a synthesis allows healthcare providers and policy makers to evaluate the applicability of the findings to their setting. Describing the certainty in the body of evidence for an outcome and the implications of findings should help policy makers, managers, and other decision makers formulate appropriate recommendations for practice or policy. Complete reporting of all PRISMA 2020 items also facilitates replication and review updates, as well as inclusion of systematic reviews in overviews (of systematic reviews) and guidelines, so teams can leverage work that is already done and decrease research waste [ 36 , 62 , 63 ].

We updated the PRISMA 2009 statement by adapting the EQUATOR Network’s guidance for developing health research reporting guidelines [ 64 ]. We evaluated the reporting completeness of published systematic reviews [ 17 , 21 , 36 , 37 ], reviewed the items included in other documents providing guidance for systematic reviews [ 38 ], surveyed systematic review methodologists and journal editors for their views on how to revise the original PRISMA statement [ 35 ], discussed the findings at an in-person meeting, and prepared this document through an iterative process. Our recommendations are informed by the reviews and survey conducted before the in-person meeting, theoretical considerations about which items facilitate replication and help users assess the risk of bias and applicability of systematic reviews, and co-authors’ experience with authoring and using systematic reviews.

Various strategies to increase the use of reporting guidelines and improve reporting have been proposed. They include educators introducing reporting guidelines into graduate curricula to promote good reporting habits of early career scientists [ 65 ]; journal editors and regulators endorsing use of reporting guidelines [ 18 ]; peer reviewers evaluating adherence to reporting guidelines [ 61 , 66 ]; journals requiring authors to indicate where in their manuscript they have adhered to each reporting item [ 67 ]; and authors using online writing tools that prompt complete reporting at the writing stage [ 60 ]. Multi-pronged interventions, where more than one of these strategies are combined, may be more effective (such as completion of checklists coupled with editorial checks) [ 68 ]. However, of 31 interventions proposed to increase adherence to reporting guidelines, the effects of only 11 have been evaluated, mostly in observational studies at high risk of bias due to confounding [ 69 ]. It is therefore unclear which strategies should be used. Future research might explore barriers and facilitators to the use of PRISMA 2020 by authors, editors, and peer reviewers, designing interventions that address the identified barriers, and evaluating those interventions using randomised trials. To inform possible revisions to the guideline, it would also be valuable to conduct think-aloud studies [ 70 ] to understand how systematic reviewers interpret the items, and reliability studies to identify items where there is varied interpretation of the items.

We encourage readers to submit evidence that informs any of the recommendations in PRISMA 2020 (via the PRISMA statement website: http://www.prisma-statement.org/ ). To enhance accessibility of PRISMA 2020, several translations of the guideline are under way (see available translations at the PRISMA statement website). We encourage journal editors and publishers to raise awareness of PRISMA 2020 (for example, by referring to it in journal “Instructions to authors”), endorsing its use, advising editors and peer reviewers to evaluate submitted systematic reviews against the PRISMA 2020 checklists, and making changes to journal policies to accommodate the new reporting recommendations. We recommend existing PRISMA extensions [ 47 , 49 , 50 , 51 , 52 , 53 , 71 , 72 ] be updated to reflect PRISMA 2020 and advise developers of new PRISMA extensions to use PRISMA 2020 as the foundation document.

We anticipate that the PRISMA 2020 statement will benefit authors, editors, and peer reviewers of systematic reviews, and different users of reviews, including guideline developers, policy makers, healthcare providers, patients, and other stakeholders. Ultimately, we hope that uptake of the guideline will lead to more transparent, complete, and accurate reporting of systematic reviews, thus facilitating evidence based decision making.

Box 1 Glossary of terms

Systematic review —A review that uses explicit, systematic methods to collate and synthesise findings of studies that address a clearly formulated question [ 43 ]

Statistical synthesis —The combination of quantitative results of two or more studies. This encompasses meta-analysis of effect estimates (described below) and other methods, such as combining P values, calculating the range and distribution of observed effects, and vote counting based on the direction of effect (see McKenzie and Brennan [ 25 ] for a description of each method)

Meta-analysis of effect estimates —A statistical technique used to synthesise results when study effect estimates and their variances are available, yielding a quantitative summary of results [ 25 ]

Outcome —An event or measurement collected for participants in a study (such as quality of life, mortality)

Result —The combination of a point estimate (such as a mean difference, risk ratio, or proportion) and a measure of its precision (such as a confidence/credible interval) for a particular outcome

Report —A document (paper or electronic) supplying information about a particular study. It could be a journal article, preprint, conference abstract, study register entry, clinical study report, dissertation, unpublished manuscript, government report, or any other document providing relevant information

Record —The title or abstract (or both) of a report indexed in a database or website (such as a title or abstract for an article indexed in Medline). Records that refer to the same report (such as the same journal article) are “duplicates”; however, records that refer to reports that are merely similar (such as a similar abstract submitted to two different conferences) should be considered unique.

Study —An investigation, such as a clinical trial, that includes a defined group of participants and one or more interventions and outcomes. A “study” might have multiple reports. For example, reports could include the protocol, statistical analysis plan, baseline characteristics, results for the primary outcome, results for harms, results for secondary outcomes, and results for additional mediator and moderator analyses

Box 2 Noteworthy changes to the PRISMA 2009 statement

• Inclusion of the abstract reporting checklist within PRISMA 2020 (see item #2 and Box 2 ).

• Movement of the ‘Protocol and registration’ item from the start of the Methods section of the checklist to a new Other section, with addition of a sub-item recommending authors describe amendments to information provided at registration or in the protocol (see item #24a-24c).

• Modification of the ‘Search’ item to recommend authors present full search strategies for all databases, registers and websites searched, not just at least one database (see item #7).

• Modification of the ‘Study selection’ item in the Methods section to emphasise the reporting of how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process (see item #8).

• Addition of a sub-item to the ‘Data items’ item recommending authors report how outcomes were defined, which results were sought, and methods for selecting a subset of results from included studies (see item #10a).

• Splitting of the ‘Synthesis of results’ item in the Methods section into six sub-items recommending authors describe: the processes used to decide which studies were eligible for each synthesis; any methods required to prepare the data for synthesis; any methods used to tabulate or visually display results of individual studies and syntheses; any methods used to synthesise results; any methods used to explore possible causes of heterogeneity among study results (such as subgroup analysis, meta-regression); and any sensitivity analyses used to assess robustness of the synthesised results (see item #13a-13f).

• Addition of a sub-item to the ‘Study selection’ item in the Results section recommending authors cite studies that might appear to meet the inclusion criteria, but which were excluded, and explain why they were excluded (see item #16b).

• Splitting of the ‘Synthesis of results’ item in the Results section into four sub-items recommending authors: briefly summarise the characteristics and risk of bias among studies contributing to the synthesis; present results of all statistical syntheses conducted; present results of any investigations of possible causes of heterogeneity among study results; and present results of any sensitivity analyses (see item #20a-20d).

• Addition of new items recommending authors report methods for and results of an assessment of certainty (or confidence) in the body of evidence for an outcome (see items #15 and #22).

• Addition of a new item recommending authors declare any competing interests (see item #26).

• Addition of a new item recommending authors indicate whether data, analytic code and other materials used in the review are publicly available and if so, where they can be found (see item #27).

Gurevitch J, Koricheva J, Nakagawa S, Stewart G. Meta-analysis and the science of research synthesis. Nature. 2018;555:175–82. https://doi.org/10.1038/nature25753 .

Article   CAS   PubMed   Google Scholar  

Gough D, Thomas J, Oliver S. Clarifying differences between reviews within evidence ecosystems. Syst Rev. 2019;8:170. https://doi.org/10.1186/s13643-019-1089-2 .

Article   PubMed   PubMed Central   Google Scholar  

Moher D. Reporting guidelines: doing better for readers. BMC Med. 2018;16:233. https://doi.org/10.1186/s12916-018-1226-0 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264–9, W64. https://doi.org/10.7326/0003-4819-151-4-200908180-00135 .

Article   PubMed   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535. https://doi.org/10.1136/bmj.b2535 .

Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097. https://doi.org/10.1371/journal.pmed.1000097 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62:1006–12. https://doi.org/10.1016/j.jclinepi.2009.06.005 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8:336–41. https://doi.org/10.1016/j.ijsu.2010.02.007 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Open Med. 2009;3:e123–30.

PubMed   PubMed Central   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Reprint--preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Phys Ther. 2009;89:873–80. https://doi.org/10.1093/ptj/89.9.873 .

Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4:e78. https://doi.org/10.1371/journal.pmed.0040078 .

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62:e1–34. https://doi.org/10.1016/j.jclinepi.2009.06.006 .

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. https://doi.org/10.1136/bmj.b2700 .

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med. 2009;151:W65–94. https://doi.org/10.7326/0003-4819-151-4-200908180-00136 .

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100. https://doi.org/10.1371/journal.pmed.1000100 .

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting. systematic reviews and meta-analyses of studies that evaluate health care. interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100. https://doi.org/10.1371/journal.pmed.1000100 .

Page MJ, Shamseer L, Altman DG, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13:e1002028. https://doi.org/10.1371/journal.pmed.1002028 .

Panic N, Leoncini E, de Belvis G, Ricciardi W, Boccia S. Evaluation of the endorsement of the preferred reporting items for systematic reviews and meta-analysis (PRISMA) statement on the quality of published systematic review and meta-analyses. PLoS One. 2013;8:e83138. https://doi.org/10.1371/journal.pone.0083138 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Agha RA, Fowler AJ, Limb C, et al. Impact of the mandatory implementation of reporting guidelines on reporting quality in a surgical journal: a before and after study. Int J Surg. 2016;30:169–72. https://doi.org/10.1016/j.ijsu.2016.04.032 .

Leclercq V, Beaudart C, Ajamieh S, Rabenda V, Tirelli E, Bruyère O. Meta-analyses indexed in PsycINFO had a better completeness of reporting when they mention PRISMA. J Clin Epidemiol. 2019;115:46–54. https://doi.org/10.1016/j.jclinepi.2019.06.014 .

Page MJ, Moher D. Evaluations of the uptake and impact of the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement and extensions: a scoping review. Syst Rev. 2017;6:263. https://doi.org/10.1186/s13643-017-0663-8 .

O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4:5. https://doi.org/10.1186/2046-4053-4-5 .

Marshall IJ, Noel-Storr A, Kuiper J, Thomas J, Wallace BC. Machine learning for identifying randomized controlled trials: an evaluation and practitioner’s guide. Res Synth Methods. 2018;9:602–14. https://doi.org/10.1002/jrsm.1287 .

Marshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev. 2019;8:163. https://doi.org/10.1186/s13643-019-1074-9 .

McKenzie JE, Brennan SE. Synthesizing and presenting findings using other methods. In: Higgins JPT, Thomas J, Chandler J, et al., editors. Cochrane handbook for systematic reviews of interventions. London: Cochrane; 2019. https://doi.org/10.1002/9781119536604.ch12 .

Chapter   Google Scholar  

Higgins JPT, López-López JA, Becker BJ, et al. Synthesising quantitative evidence in systematic reviews of complex health interventions. BMJ Glob Health. 2019;4(Suppl 1):e000858. https://doi.org/10.1136/bmjgh-2018-000858 .

Campbell M, McKenzie JE, Sowden A, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ. 2020;368:l6890. https://doi.org/10.1136/bmj.l6890 .

Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. https://doi.org/10.1136/bmj.l4898 .

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. https://doi.org/10.1136/bmj.i4919 .

Whiting P, Savović J, Higgins JP, ROBIS group, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34. https://doi.org/10.1016/j.jclinepi.2015.06.005 .

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. https://doi.org/10.1136/bmj.j4008 .

Hultcrantz M, Rind D, Akl EA, et al. The GRADE working group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017;87:4–13. https://doi.org/10.1016/j.jclinepi.2017.05.006 .

Booth A, Clarke M, Dooley G, et al. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev. 2012;1:2. https://doi.org/10.1186/2046-4053-1-2 .

Moher D, Stewart L, Shekelle P. Establishing a new journal for systematic review products. Syst Rev. 2012;1:1. https://doi.org/10.1186/2046-4053-1-1 .

Page MJ, McKenzie JE, Bossuyt PM, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. J Clin Epidemiol 2021;134:103–112. https://doi.org/10.1016/j.jclinepi.2021.02.003 .

Page MJ, Altman DG, Shamseer L, et al. Reproducible research practices are underused in systematic reviews of biomedical interventions. J Clin Epidemiol. 2018;94:8–18. https://doi.org/10.1016/j.jclinepi.2017.10.017 .

Page MJ, Altman DG, McKenzie JE, et al. Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis. J Clin Epidemiol. 2018;95:7–18. https://doi.org/10.1016/j.jclinepi.2017.11.022 .

Page MJ, McKenzie JE, Bossuyt PM, et al. Mapping of reporting guidance for systematic reviews and meta-analyses generated a comprehensive item bank for future reporting guidelines. J Clin Epidemiol. 2020;118:60–8. https://doi.org/10.1016/j.jclinepi.2019.11.010 .

Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12:181. https://doi.org/10.1186/1471-2288-12-181 .

France EF, Cunningham M, Ring N, et al. Improving reporting of meta-ethnography: the eMERGe reporting guidance. BMC Med Res Methodol. 2019;19:25. https://doi.org/10.1186/s12874-018-0600-0 .

Page MJ, Moher D, Bossuyt PM, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372:n160. https://doi.org/10.1136/bmj.n160 .

Rethlefsen ML, Kirtley S, Waffenschmidt S, et al. PRISMA-S Group PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews. Syst Rev. 2021;10:39. https://doi.org/10.1186/s13643-020-01542-z .

Higgins JPT, Thomas J, Chandler J, et al. Cochrane handbook for systematic reviews of interventions: version 6.0. London: Cochrane; 2019. Available from https://training.cochrane.org/handbook

Book   Google Scholar  

Dekkers OM, Vandenbroucke JP, Cevallos M, Renehan AG, Altman DG, Egger M. COSMOS-E: guidance on conducting systematic reviews and meta-analyses of observational studies of etiology. PLoS Med. 2019;16:e1002742. https://doi.org/10.1371/journal.pmed.1002742 .

Cooper H, Hedges LV, Valentine JV. The handbook of research synthesis and meta-analysis. New York: Russell Sage Foundation; 2019.

IOM (Institute of Medicine). Finding what works in health care: standards for systematic reviews. Washington, D.C.: The National Academies Press; 2011.

Google Scholar  

Moher D, Shamseer L, Clarke M, PRISMA-P Group, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. https://doi.org/10.1186/2046-4053-4-1 .

Shamseer L, Moher D, Clarke M, PRISMA-P Group, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350:g7647. https://doi.org/10.1136/bmj.g7647 .

Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162:777–84. https://doi.org/10.7326/M14-2385 .

Stewart LA, Clarke M, Rovers M, PRISMA-IPD Development Group, et al. Preferred reporting items for systematic review and meta-analyses of individual participant data: the PRISMA-IPD statement. JAMA. 2015;313:1657–65. https://doi.org/10.1001/jama.2015.3656 .

Zorzela L, Loke YK, Ioannidis JP, et al. PRISMAHarms Group PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ. 2016;352:i157. https://doi.org/10.1136/bmj.i157 .

McInnes MDF, Moher D, Thombs BD, the PRISMA-DTA Group, et al. Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA. 2018;319:388–96. https://doi.org/10.1001/jama.2017.19163 .

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-SCR): checklist and explanation. Ann Intern Med. 2018;169:467–73. https://doi.org/10.7326/M18-0850 .

Beller EM, Glasziou PP, Altman DG, et al. PRISMA for Abstracts Group PRISMA for Abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med. 2013;10:e1001419. https://doi.org/10.1371/journal.pmed.1001419 .

Boers M. Graphics and statistics for cardiology: designing effective tables for presentation and publication. Heart. 2018;104:192–200. https://doi.org/10.1136/heartjnl-2017-311581 .

Mayo-Wilson E, Li T, Fusco N, Dickersin K, MUDS investigators. Practical guidance for using multiple data sources in systematic reviews and meta-analyses (with examples from the MUDS study). Res Synth Methods. 2018;9:2–12. https://doi.org/10.1002/jrsm.1277 .

Stovold E, Beecher D, Foxlee R, Noel-Storr A. Study flow diagrams in Cochrane systematic review updates: an adapted PRISMA flow diagram. Syst Rev. 2014;3:54. https://doi.org/10.1186/2046-4053-3-54 .

McGuinness LA. mcguinlu/PRISMA-Checklist: Initial release for manuscript submission (Version v1.0.0). Geneva: Zenodo; 2020. https://doi.org/10.5281/zenodo.3994319 .

Aczel B, Szaszi B, Sarafoglou A, et al. A consensus-based transparency checklist. Nat Hum Behav. 2020;4:4–6. https://doi.org/10.1038/s41562-019-0772-6 .

Barnes C, Boutron I, Giraudeau B, Porcher R, Altman DG, Ravaud P. Impact of an online writing aid tool for writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial. BMC Med. 2015;13:221. https://doi.org/10.1186/s12916-015-0460-y .

Chauvin A, Ravaud P, Moher D, et al. Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study. BMC Med. 2019;17:205. https://doi.org/10.1186/s12916-019-1436-0 .

Wayant C, Page MJ, Vassar M. Evaluation of reproducible research practices in oncology systematic reviews with meta-analyses referenced by national comprehensive cancer network guidelines. JAMA Oncol. 2019;5:1550–5. https://doi.org/10.1001/jamaoncol.2019.2564 .

Article   PubMed Central   PubMed   Google Scholar  

McKenzie JE, Brennan SE. Overviews of systematic reviews: great promise, greater challenge. Syst Rev. 2017;6:185. https://doi.org/10.1186/s13643-017-0582-8 .

Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7:e1000217. https://doi.org/10.1371/journal.pmed.1000217 .

Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med. 2010;8:24. https://doi.org/10.1186/1741-7015-8-24 .

Speich B, Schroter S, Briel M, et al. Impact of a short version of the CONSORT checklist for peer reviewers to improve the reporting of randomised controlled trials published in biomedical journals: study protocol for a randomised controlled trial. BMJ Open. 2020;10:e035114. https://doi.org/10.1136/bmjopen-2019-035114 .

Stevens A, Shamseer L, Weinstein E, et al. Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic review. BMJ. 2014;348:g3804. https://doi.org/10.1136/bmj.g3804 .

Hair K, Macleod MR, Sena ES, IICARus Collaboration. A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus). Res Integr Peer Rev. 2019;4:12. https://doi.org/10.1186/s41073-019-0069-3 .

Blanco D, Altman D, Moher D, Boutron I, Kirkham JJ, Cobo E. Scoping review on interventions to improve adherence to reporting guidelines in health research. BMJ Open. 2019;9:e026589. https://doi.org/10.1136/bmjopen-2018-026589 .

Charters E. The use of think-aloud methods in qualitative research: an introduction to think-aloud methods. Brock Educ J. 2003;12:68–82. https://doi.org/10.26522/brocked.v12i2.38 .

Article   Google Scholar  

Welch V, Petticrew M, Tugwell P, PRISMA-Equity Bellagio group, et al. PRISMA-equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Med. 2012;9:e1001333. https://doi.org/10.1371/journal.pmed.1001333 .

Wang X, Chen Y, Liu Y, et al. Reporting items for systematic reviews and meta-analyses of acupuncture: the PRISMA for acupuncture checklist. BMC Complement Altern Med. 2019;19:208. https://doi.org/10.1186/s12906-019-2624-3 .

Download references

Acknowledgements

We dedicate this paper to the late Douglas G Altman and Alessandro Liberati, whose contributions were fundamental to the development and implementation of the original PRISMA statement.

We thank the following contributors who completed the survey to inform discussions at the development meeting: Xavier Armoiry, Edoardo Aromataris, Ana Patricia Ayala, Ethan M Balk, Virginia Barbour, Elaine Beller, Jesse A Berlin, Lisa Bero, Zhao-Xiang Bian, Jean Joel Bigna, Ferrán Catalá-López, Anna Chaimani, Mike Clarke, Tammy Clifford, Ioana A Cristea, Miranda Cumpston, Sofia Dias, Corinna Dressler, Ivan D Florez, Joel J Gagnier, Chantelle Garritty, Long Ge, Davina Ghersi, Sean Grant, Gordon Guyatt, Neal R Haddaway, Julian PT Higgins, Sally Hopewell, Brian Hutton, Jamie J Kirkham, Jos Kleijnen, Julia Koricheva, Joey SW Kwong, Toby J Lasserson, Julia H Littell, Yoon K Loke, Malcolm R Macleod, Chris G Maher, Ana Marušic, Dimitris Mavridis, Jessie McGowan, Matthew DF McInnes, Philippa Middleton, Karel G Moons, Zachary Munn, Jane Noyes, Barbara Nußbaumer-Streit, Donald L Patrick, Tatiana Pereira-Cenci, Ba′ Pham, Bob Phillips, Dawid Pieper, Michelle Pollock, Daniel S Quintana, Drummond Rennie, Melissa L Rethlefsen, Hannah R Rothstein, Maroeska M Rovers, Rebecca Ryan, Georgia Salanti, Ian J Saldanha, Margaret Sampson, Nancy Santesso, Rafael Sarkis-Onofre, Jelena Savović, Christopher H Schmid, Kenneth F Schulz, Guido Schwarzer, Beverley J Shea, Paul G Shekelle, Farhad Shokraneh, Mark Simmonds, Nicole Skoetz, Sharon E Straus, Anneliese Synnot, Emily E Tanner-Smith, Brett D Thombs, Hilary Thomson, Alexander Tsertsvadze, Peter Tugwell, Tari Turner, Lesley Uttley, Jeffrey C Valentine, Matt Vassar, Areti Angeliki Veroniki, Meera Viswanathan, Cole Wayant, Paul Whaley, and Kehu Yang. We thank the following contributors who provided feedback on a preliminary version of the PRISMA 2020 checklist: Jo Abbott, Fionn Büttner, Patricia Correia-Santos, Victoria Freeman, Emily A Hennessy, Rakibul Islam, Amalia (Emily) Karahalios, Kasper Krommes, Andreas Lundh, Dafne Port Nascimento, Davina Robson, Catherine Schenck-Yglesias, Mary M Scott, Sarah Tanveer and Pavel Zhelnov. We thank Abigail H Goben, Melissa L Rethlefsen, Tanja Rombey, Anna Scott, and Farhad Shokraneh for their helpful comments on the preprints of the PRISMA 2020 papers. We thank Edoardo Aromataris, Stephanie Chang, Toby Lasserson and David Schriger for their helpful peer review comments on the PRISMA 2020 papers.

Provenance and peer review

Not commissioned; externally peer reviewed.

Patient and public involvement

Patients and the public were not involved in this methodological research. We plan to disseminate the research widely, including to community participants in evidence synthesis organisations.

There was no direct funding for this research. MJP is supported by an Australian Research Council Discovery Early Career Researcher Award (DE200101618) and was previously supported by an Australian National Health and Medical Research Council (NHMRC) Early Career Fellowship (1088535) during the conduct of this research. JEM is supported by an Australian NHMRC Career Development Fellowship (1143429). TCH is supported by an Australian NHMRC Senior Research Fellowship (1154607). JMT is supported by Evidence Partners Inc. JMG is supported by a Tier 1 Canada Research Chair in Health Knowledge Transfer and Uptake. MML is supported by The Ottawa Hospital Anaesthesia Alternate Funds Association and a Faculty of Medicine Junior Research Chair. TL is supported by funding from the National Eye Institute (UG1EY020522), National Institutes of Health, United States. LAM is supported by a National Institute for Health Research Doctoral Research Fellowship (DRF-2018-11-ST2–048). ACT is supported by a Tier 2 Canada Research Chair in Knowledge Synthesis. DM is supported in part by a University Research Chair, University of Ottawa. The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

Author information

Authors and affiliations.

School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia

Matthew J. Page, Joanne E. McKenzie, Sue E. Brennan & Steve McDonald

Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, Netherlands

Patrick M. Bossuyt

Université de Paris, Centre of Epidemiology and Statistics (CRESS), Inserm, F 75004, Paris, France

Isabelle Boutron

Institute for Evidence-Based Healthcare, Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Australia

Tammy C. Hoffmann

Annals of Internal Medicine, University of Texas Health Science Center at San Antonio, San Antonio, Texas, USA

Cynthia D. Mulrow

Knowledge Translation Program, Li Ka Shing Knowledge Institute, Toronto, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

Larissa Shamseer

Evidence Partners, Ottawa, Canada

Jennifer M. Tetzlaff

Clinical Research Institute, American University of Beirut, Beirut, Lebanon; Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada

Elie A. Akl

Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, OR, USA

York Health Economics Consortium (YHEC Ltd), University of York, York, UK

Julie Glanville

Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada; Department of Medicine, University of Ottawa, Ottawa, Canada

Jeremy M. Grimshaw

Centre for Evidence-Based Medicine Odense (CEBMO) and Cochrane Denmark, Department of Clinical Research, University of Southern Denmark, JB Winsløwsvej 9b, 3rd Floor, 5000 Odense, Denmark; Open Patient data Exploratory Network (OPEN), Odense University Hospital, Odense, Denmark

Asbjørn Hróbjartsson

Department of Anesthesiology and Pain Medicine, The Ottawa Hospital, Ottawa, Canada; Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Canada; Regenerative Medicine Program, Ottawa Hospital Research Institute, Ottawa, Canada

Manoj M. Lalu

Department of Ophthalmology, School of Medicine, University of Colorado Denver, Denver, Colorado, United States; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA

Tianjing Li

Division of Headache, Department of Neurology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, USA; Head of Research, The BMJ, London, UK

Elizabeth W. Loder

Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington, Bloomington, Indiana, USA

Evan Mayo-Wilson

Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK

Luke A. McGuinness & Penny Whiting

Centre for Reviews and Dissemination, University of York, York, UK

Lesley A. Stewart

EPPI-Centre, UCL Social Research Institute, University College London, London, UK

James Thomas

Li Ka Shing Knowledge Institute of St. Michael’s Hospital, Unity Health Toronto, Toronto, Canada; Epidemiology Division of the Dalla Lana School of Public Health and the Institute of Health Management, Policy, and Evaluation, University of Toronto, Toronto, Canada; Queen’s Collaboration for Health Care Quality Joanna Briggs Institute Centre of Excellence, Queen’s University, Kingston, Canada

Andrea C. Tricco

Methods Centre, Bruyère Research Institute, Ottawa, Ontario, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

Vivian A. Welch

Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

David Moher

You can also search for this author in PubMed   Google Scholar

Contributions

JEM and DM are joint senior authors. MJP, JEM, PMB, IB, TCH, CDM, LS, and DM conceived this paper and designed the literature review and survey conducted to inform the guideline content. MJP conducted the literature review, administered the survey and analysed the data for both. MJP prepared all materials for the development meeting. MJP and JEM presented proposals at the development meeting. All authors except for TCH, JMT, EAA, SEB, and LAM attended the development meeting. MJP and JEM took and consolidated notes from the development meeting. MJP and JEM led the drafting and editing of the article. JEM, PMB, IB, TCH, LS, JMT, EAA, SEB, RC, JG, AH, TL, EMW, SM, LAM, LAS, JT, ACT, PW, and DM drafted particular sections of the article. All authors were involved in revising the article critically for important intellectual content. All authors approved the final version of the article. MJP is the guarantor of this work. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Corresponding author

Correspondence to Matthew J. Page .

Ethics declarations

Competing interests.

All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/conflicts-of-interest/ and declare: EL is head of research for the BMJ ; MJP is an editorial board member for PLOS Medicine ; ACT is an associate editor and MJP, TL, EMW, and DM are editorial board members for the Journal of Clinical Epidemiology ; DM and LAS were editors in chief, LS, JMT, and ACT are associate editors, and JG is an editorial board member for Systematic Reviews . None of these authors were involved in the peer review process or decision to publish. TCH has received personal fees from Elsevier outside the submitted work. EMW has received personal fees from the American Journal for Public Health , for which he is the editor for systematic reviews. VW is editor in chief of the Campbell Collaboration, which produces systematic reviews, and co-convenor of the Campbell and Cochrane equity methods group. DM is chair of the EQUATOR Network, IB is adjunct director of the French EQUATOR Centre and TCH is co-director of the Australasian EQUATOR Centre, which advocates for the use of reporting guidelines to improve the quality of reporting in research articles. JMT received salary from Evidence Partners, creator of DistillerSR software for systematic reviews; Evidence Partners was not involved in the design or outcomes of the statement, and the views expressed solely represent those of the author.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

PRISMA 2020 checklist.

Additional file 2.

PRISMA 2020 expanded checklist.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Page, M.J., McKenzie, J.E., Bossuyt, P.M. et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev 10 , 89 (2021). https://doi.org/10.1186/s13643-021-01626-4

Download citation

Accepted : 04 January 2021

Published : 29 March 2021

DOI : https://doi.org/10.1186/s13643-021-01626-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Ask a Librarian

Systematic Reviews

• Choosing a Review
• Getting Started
• Further Resources
• Sys Review Request Form

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses ( PRISMA ) is a gold standard process for reporting systematic reviews. Although originally developed for the health sciences, PRISMA contains important considerations for systematic reviews in any discipline. 

The main PRISMA page provides key documents, including a checklist, flow diagram, and an explanation and elaboration article.

Additionally, PRISMA has sponsored several extension  documents to help researchers with specific aspects of systematic reviews or additional review types. In particular the extension documents for protocols, scoping reviews, and searching might be of interest to researchers. 

• PRISMA Checklist : Useful for ensuring each component is described in the final manuscript.
• PRISM Protocols : Useful for completing and reporting protocols for systematic reviews (and other evidence synthesis articles)
• PRISMA flow diagram  (and  generator) : Template for reporting the results of the systematic search
• PRISMA Statement : Explanation for what PRISMA is and how researchers can use it
• How to cite PRISMA
• << Previous: Getting Started
• Next: Further Resources >>
• Last Edited: Dec 4, 2023 1:53 PM
• URL: https://guides.lib.purdue.edu/sys-review

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

Affiliations.

• 1 School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia [email protected].
• 2 School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia.
• 3 Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, Netherlands.
• 4 Université de Paris, Centre of Epidemiology and Statistics (CRESS), Inserm, F 75004 Paris, France.
• 5 Institute for Evidence-Based Healthcare, Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Australia.
• 6 University of Texas Health Science Center at San Antonio, San Antonio, Texas, USA; Annals of Internal Medicine.
• 7 Knowledge Translation Program, Li Ka Shing Knowledge Institute, Toronto, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada.
• 8 Evidence Partners, Ottawa, Canada.
• 9 Clinical Research Institute, American University of Beirut, Beirut, Lebanon; Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
• 10 Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, USA.
• 11 York Health Economics Consortium (YHEC Ltd), University of York, York, UK.
• 12 Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada; Department of Medicine, University of Ottawa, Ottawa, Canada.
• 13 Centre for Evidence-Based Medicine Odense (CEBMO) and Cochrane Denmark, Department of Clinical Research, University of Southern Denmark, Odense, Denmark; Open Patient data Exploratory Network (OPEN), Odense University Hospital, Odense, Denmark.
• 14 Department of Anesthesiology and Pain Medicine, The Ottawa Hospital, Ottawa, Canada; Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Canada; Regenerative Medicine Program, Ottawa Hospital Research Institute, Ottawa, Canada.
• 15 Department of Ophthalmology, School of Medicine, University of Colorado Denver, Denver, Colorado, United States; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA.
• 16 Division of Headache, Department of Neurology, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA; Head of Research, The BMJ, London, UK.
• 17 Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington, Bloomington, Indiana, USA.
• 18 Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK.
• 19 Centre for Reviews and Dissemination, University of York, York, UK.
• 20 EPPI-Centre, UCL Social Research Institute, University College London, London, UK.
• 21 Li Ka Shing Knowledge Institute of St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Epidemiology Division of the Dalla Lana School of Public Health and the Institute of Health Management, Policy, and Evaluation, University of Toronto, Toronto, Canada; Queen's Collaboration for Health Care Quality Joanna Briggs Institute Centre of Excellence, Queen's University, Kingston, Canada.
• 22 Methods Centre, Bruyère Research Institute, Ottawa, Ontario, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada.
• 23 Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada.
• PMID: 33782057
• PMCID: PMC8005924
• DOI: 10.1136/bmj.n71

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.

PubMed Disclaimer

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/conflicts-of-interest/ and declare: EL is head of research for the BMJ; MJP is an editorial board member for PLOS Medicine; ACT is an associate editor and MJP, TL, EMW, and DM are editorial board members for the Journal of Clinical Epidemiology; DM and LAS were editors in chief, LS, JMT, and ACT are associate editors, and JG is an editorial board member for Systematic Reviews. None of these authors were involved in the peer review process or decision to publish. TCH has received personal fees from Elsevier outside the submitted work. EMW has received personal fees from the American Journal for Public Health, for which he is the editor for systematic reviews. VW is editor in chief of the Campbell Collaboration, which produces systematic reviews, and co-convenor of the Campbell and Cochrane equity methods group. DM is chair of the EQUATOR Network, IB is adjunct director of the French EQUATOR Centre and TCH is co-director of the Australasian EQUATOR Centre, which advocates for the use of reporting guidelines to improve the quality of reporting in research articles. JMT received salary from Evidence Partners, creator of DistillerSR software for systematic reviews; Evidence Partners was not involved in the design or outcomes of the statement, and the views expressed solely represent those of the author.

PRISMA 2020 flow diagram template…

PRISMA 2020 flow diagram template for systematic reviews. The new design is adapted…

• Rigour and transparency in the family of systematic reviews: The International Journal of Health Planning and Management encourages prospective protocol registration. Jesus TS. Jesus TS. Int J Health Plann Manage. 2022 Sep;37(5):2523-2527. doi: 10.1002/hpm.3510. Epub 2022 Jun 7. Int J Health Plann Manage. 2022. PMID: 35672954 No abstract available.
• Tissue Sealants for Facial Rhytidectomy: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Chien WY, Huang YL, Chiu WK, Kang YN, Chen C. Chien WY, et al. Facial Plast Surg Aesthet Med. 2023 Mar-Apr;25(2):90-96. doi: 10.1089/fpsam.2022.0160. Epub 2022 Oct 18. Facial Plast Surg Aesthet Med. 2023. PMID: 36260353 Review.
• Efficacy and safety of galcanezumab as a treatment of refractory episodic and chronic cluster headache: Case series and narrative review. Membrilla JA, Torres-Ferrus M, Alpuente A, Caronna E, Pozo-Rosich P. Membrilla JA, et al. Headache. 2022 Nov;62(10):1395-1405. doi: 10.1111/head.14404. Epub 2022 Nov 2. Headache. 2022. PMID: 36321947 Review.

Similar articles

• PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, McKenzie JE. Page MJ, et al. BMJ. 2021 Mar 29;372:n160. doi: 10.1136/bmj.n160. BMJ. 2021. PMID: 33781993 Free PMC article.
• The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D. Page MJ, et al. J Clin Epidemiol. 2021 Jun;134:178-189. doi: 10.1016/j.jclinepi.2021.03.001. Epub 2021 Mar 29. J Clin Epidemiol. 2021. PMID: 33789819
• Methodological quality of systematic reviews comprising clinical practice guidelines for cardiovascular risk assessment and management for noncardiac surgery. Jacobsen SM, Douglas A, Smith CA, Roberts W, Ottwell R, Oglesby B, Yasler C, Torgerson T, Hartwell M, Vassar M. Jacobsen SM, et al. Br J Anaesth. 2021 Dec;127(6):905-916. doi: 10.1016/j.bja.2021.08.016. Epub 2021 Sep 20. Br J Anaesth. 2021. PMID: 34548174
• The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. Liberati A, et al. J Clin Epidemiol. 2009 Oct;62(10):e1-34. doi: 10.1016/j.jclinepi.2009.06.006. Epub 2009 Jul 23. J Clin Epidemiol. 2009. PMID: 19631507
• The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. Liberati A, et al. Ann Intern Med. 2009 Aug 18;151(4):W65-94. doi: 10.7326/0003-4819-151-4-200908180-00136. Epub 2009 Jul 20. Ann Intern Med. 2009. PMID: 19622512
• Mechanisms of manipulation: a systematic review of the literature on immediate anatomical structural or positional changes in response to manually delivered high-velocity, low-amplitude spinal manipulation. Young KJ, Leboeuf-Yde C, Gorrell L, Bergström C, Evans DW, Axén I, Chance-Larsen K, Gagey O, Georgopoulos V, Goncalves G, Harris C, Harsted S, Kerry R, Lee E, McCarthy C, Nim C, Nyirö L, Schweinhardt P, Vogel S. Young KJ, et al. Chiropr Man Therap. 2024 Sep 11;32(1):28. doi: 10.1186/s12998-024-00549-w. Chiropr Man Therap. 2024. PMID: 39261958 Review.
• Evaluation of research co-design in health: a systematic overview of reviews and development of a framework. Peters S, Guccione L, Francis J, Best S, Tavender E, Curran J, Davies K, Rowe S, Palmer VJ, Klaic M. Peters S, et al. Implement Sci. 2024 Sep 11;19(1):63. doi: 10.1186/s13012-024-01394-4. Implement Sci. 2024. PMID: 39261956 Review.
• Consideration of inequalities in effectiveness trials of mHealth applications - a systematic assessment of studies from an umbrella review. Abdelmalak N, Burns J, Suhlrie L, Laxy M, Stephan AJ. Abdelmalak N, et al. Int J Equity Health. 2024 Sep 11;23(1):181. doi: 10.1186/s12939-024-02267-4. Int J Equity Health. 2024. PMID: 39261871 Review.
• Resting-state electroencephalography and magnetoencephalography in migraine-a systematic review and meta-analysis. Zebhauser PT, Heitmann H, May ES, Ploner M. Zebhauser PT, et al. J Headache Pain. 2024 Sep 11;25(1):147. doi: 10.1186/s10194-024-01857-5. J Headache Pain. 2024. PMID: 39261817 Review.
• Association between removable prosthesis-wearing and pneumonia: a systematic review and meta-analysis. Lim TW, Li KY, Burrow MF, McGrath C. Lim TW, et al. BMC Oral Health. 2024 Sep 11;24(1):1061. doi: 10.1186/s12903-024-04814-5. BMC Oral Health. 2024. PMID: 39261813
• Gurevitch J, Koricheva J, Nakagawa S, Stewart G. Meta-analysis and the science of research synthesis. Nature 2018;555:175-82. . 10.1038/nature25753 - DOI - PubMed
• Gough D, Thomas J, Oliver S. Clarifying differences between reviews within evidence ecosystems. Syst Rev 2019;8:170. . 10.1186/s13643-019-1089-2 - DOI - PMC - PubMed
• Moher D. Reporting guidelines: doing better for readers. BMC Med 2018;16:233. . 10.1186/s12916-018-1226-0 - DOI - PMC - PubMed
• Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009;151:264-9, W64. 10.7326/0003-4819-151-4-200908180-00135 - DOI - PubMed
• Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009;339:b2535. 10.1136/bmj.b2535 - DOI - PMC - PubMed

Publication types

• Search in MeSH

Related information

• Cited in Books

Grants and funding

• UG1 EY020522/EY/NEI NIH HHS/United States

LinkOut - more resources

Full text sources.

• Europe PubMed Central
• Ovid Technologies, Inc.
• PubMed Central
• White Rose Research Online

Other Literature Sources

• scite Smart Citations

Miscellaneous

• NCI CPTAC Assay Portal
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Grand Valley State University

• University Libraries

Search the Library for Articles, Books, and More Find It!

More research tools:

• Subject Guides
• Find, Borrow, Request
• Teaching & Scholarship
• Visit, Study, Learn
• Systematic and Scoping Reviews
• 5. Organize & Screen Search Results

Systematic and Scoping Reviews: 5. Organize & Screen Search Results

• Types of Reviews
• Review Guidelines
• 0. Plan your Review
• 1. Define the Question
• 2. Check for Recent Reviews and Protocols
• 3. Write and register your protocol
• Developing your Search Terms
• Database Search Tips and Filters
• Grey Literature
• Record and Report your Search Strategy
• 6. Appraise the Studies
• 7. Extract Data
• 8. Analyze / Synthesize Data
• 9. Write the Review
• Rapid Reviews
• Scoping Reviews
• Librarian Support

To Do during this step

• Combine and de-duplicate your search results
• Title/abstract screening by 2 or more people
• Full-text screening by 2 or more people
• Conduct citation searching on included articles
• Record the data from these steps in PRISMA flow diagram

Pro tip: Save time and stress by piloting the title/abstract and full-text screening with the whole team. There will be fewer conflicts to resolve later.

Reporting Your Process

You will need to report your screening process. Use these tools to record how many results are removed during each stage of screening.

The PRISMA Statement: Preferred Reporting Items for Systematic Reviews and Meta-Analyses Evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.  Focused on randomized trials,  PRISMA can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.

The PRISMA flow diagram is a tool for recording and reporting the number of records during the different steps of a systematic review, along with reasons for exclusion. It is often included within the review or as supplemental material.

Tool for Generating a PRISMA Search Flow Diagram - from the Evidence Synthesis Hackathon

Complying with PRISMA and Standards for Cochrane: See Cochrane Handbook >  Methodological Expectations of Cochrane Intervention Reviews (MECIR)

Managing References

Manage your search results.

Export the results of your search from each database. For instructions, check the Help or Support for the database.

Import the file into your citation management tool. For instructions, see your chosen citation management tool's Support or Help documentation.

De-duplicating

In your searches across multiple databases, there will be some articles that are retrieved in more than one database. This is expected, but you don't need to evaluate that article more than once. Before you start screening, de-duplicate. The processes described below are methods for removing duplicates while minimizing the risk of accidentally removing similar non-duplicate articles. Some of the screening tools also have effective de-duplication.

SRA-DM tool:  Rathbone J, Carter M, Hoffmann T, Glasziou P.  Better duplicate detection for systematic reviewers: evaluation of Systematic Review Assistant-Deduplication Module . Syst Rev. 2015 Jan 14;4:6. doi: 10.1186/2046-4053-4-6. PubMed PMID: 25588387.

For desktop version of EndNote only: Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote . Journal of the Medical Library Association : JMLA. 2016;104(3):240-243. doi:10.3163/1536-5050.104.3.014<

Make a note of how many duplicates were removed for reporting in your paper. Your PRISMA flow diagram is a good place to keep track.

Screening the Articles

Steps for screening.

The purpose of article screening to remove studies that are not eligible for inclusion.

Use your inclusion/exclusion criteria , two or more team members will conduct the following:

• Title/abstract screening : First, screen the title and abstracts of the studies and determine whether they are relevant to your research question. Since you conducted a comprehensive search, there will be items that were captured that are clearly not relevant.
• Full text screening : For studies included based on the title/abstract screening, obtain the full text and evaluate for inclusion/exclusion.

During both steps, record the reason for excluding an item. Review support software commonly contains features to simplify this.

Review support software will typically include a record screening/study selection function. This allows more than one reviewer to independently screen the records without seeing other reviewers' decisions to include or exclude studies, and thus reduces bias. Areas of disagreement can be resolved by consensus or by a third party who is an expert in the field.

Project management software for systematic reviews and other evidence synthesis :

These tools provide support for independent screening of the title/abstracts and the full text of articles. Some have additional features, such as support for data extraction or machine-learning to sort results. The health sciences librarian are not trained in these tools and there are currently no institutional subscriptions to these tools.

• Rayyan Free web software and mobile app for SR project teams. Does not include data extraction.
• Rayyan tutorial series Short Rayyan video tutorials from Northeastern University Library
• Covidence Web-based software with support for quality appraisal/Risk of Bias phase and for data extraction. Easy export to popular statistical analysis programs. UW has site license - see Covidence guide for more information. Additional training resources below.
• Colandr Web-based software with support for data extraction phase and export in CSV format (openable in Excel). Algorithms assist during screening and extraction. Free.
• DistillerSR Web-based software with support for data extraction and analysis. Algorithms assist during screening and extraction. Free trial for individual students; fee-based for faculty and longer student projects.
• RevMan 5 Facilitates preparation of protocols and full reviews, including text, characteristics of studies, comparison tables, and study data. It can perform meta-analysis of the data entered, and present the results graphically. Free for Cochrane authors. Note: Download of desktop RevMan 5 for free academic use ends April 2023.
• JBI Sumari Web-based software with support for quality appraisal/Risk of Bias phase, data extraction, and meta-analysis. Fee-based; free 14-day trial.
• EPPI-Reviewer Web-based software with support for data extraction and analysis, including analysis of qualitative data. Algorithms assist during screening. Designed to support reviews that inform policy-makers.
• SR Toolbox Search for systematic review tools: software, guides, checklists, standards.

Chart showing the features of different tools and which step during which they are relevant: "Digital Tools for Managing Different Steps of the Systematic Review Process" . Wu W, Akers K, Hu E, Sarkozy A, Vinson P. Library Scholarly Publications. 2018; 136. https://digitalcommons.wayne.edu/libsp/136

Reviewing retrieved references for inclusion in systematic reviews using Endnote. Bramer WM, Milic J, Mast F. J Med Libr Assoc. 2017 Jan; 105(1): 84-87.

• Reference checking for systematic reviews using Endnote . Bramer WM. J Med Libr Assoc. 2018 Oct; 106(4): 542-6.

Covidence and Rayyan . Kellermeyer L, Harnke B, Knight S. J Med Libr Assoc. 2018 Oct; 106(4): 580-3. (Review)

U Conn's guide to using Rayyan

Translation Services

GVSU does not have an official translation service for research articles. The following resources are worth trying, but may not be able to provide everything you need.

• Google Translate Start here for initial title and abstract screening.
• Cochrane's Task Exchange Sign up and request translation help.
• Odegaard Writing & Research Center (OWRC) May be able to provide direction for more resources for translation.
• EthnoMed Provides student-translated handouts and may be willing to connect you with students or groups available to translate.

Citation Searching

Once you have completed the full-text screening, you will use the included articles to identify additional potentially similar articles in a process often called citation searching or citation chasing. This is done on the basis that it is probable that studies which cite or are cited by a source study will contain similar content. This practice is recommended in section 1.1.4 of the Technical Supplement to Chapter 4 of the 2022 Cochrane Handbook.

- Backward citation searching: Consult the reference lists for the included articles. Locate the title and abstract information for the references, then screen them according to your screening criteria.

- Forward citation searching: Locate articles which cite your included articles, then screen them according to your screening criteria.

Tools to make this easier:

• SpiderCite from SR-Accelerator Export your included articles as an EndNote library, then import them into SpiderCite. SpiderCite will generate libraries of the references and the articles citing your included articles that can be imported back into EndNote for review.
• citationchaser by Neal Haddaway Paste in a list of the doi's or PMIDs for your included articles or import as an Excel or RIS file. citationchaser will generate RIS libraries of the references and the articles citing your included articles that can be imported into a citation manager or screening software for review.
• OpenAlex from OurResearch (in development) Mentioned in the 2022 Cochrane Handbook. Its API is currently operative, but as of Sept. 2022 its website is still in progress.
• << Previous: Record and Report your Search Strategy
• Next: 6. Appraise the Studies >>
• Last Updated: Sep 11, 2024 8:22 AM
• URL: https://libguides.gvsu.edu/SR

Login to LibApps

Email us! [email protected]

Text Us! (616) 818-0219

Mary Idema Pew Library (616) 331-3500

Steelcase Library (616) 331-6799

Frey Foundation Learning Commons (616) 331-5933

Social Media

Committed to Equality

Federal Depository Library Program

• GVSU is an AA/EO Institution
• Privacy Policy
• Disclosures
• Copyright © 2018 GVSU

• Open access
• Published: 11 September 2024

Associations between retirement, social security policies and the health of older people: a systematic review

• Laíze Marina de Oliveira Teixeira 1 ,
• Fabio Alexis Rincón Uribe 1 ,
• Hélio Luiz Fonseca Moreira 2 &
• Janari da Silva Pedroso 1  

BMC Public Health volume  24 , Article number:  2473 ( 2024 ) Cite this article

Metrics details

As people age, they are more likely to experience several health conditions which are circumstances that arise throughout life that can interfere with an individual’s ability to work, leading them to demand the social security system. This research aims to systematically review and synthesize studies related to health conditions in the aging process with social security policy reforms.

A systematic review was performed across Embase, Web of Science, Scopus, Pubmed, CINAHL, ASSIA (Proquest) and APA PsycNet from 1979 to 2022. Methods are outlined in a published protocol registered a priori on PROSPERO (CRD42021225820). Eligible studies include original empirical articles published in English, Spanish, French and Portuguese, using the search terms “aging” and “social security”. Identified outcomes were organized into categories and a meta-ethnography was completed following the phases proposed by Noblit and Hare and the eMERGe meta-ethnography reporting guidance.

There were 17 eligible studies from 4 continents with 10 cross-sectional, 1 both cross-sectional and longitudinal and 5 longitudinal data analysis. These assessed the relationship of health conditions that occur in the aging process related to social security policies, in particular, to retirement. The categories included (i) health as a way to promote an active working life for the elderly; (ii) health as an indicator for reforms in social security policies; (iii) retirement planning as a strategic element for coping with post-retirement life; and (iv) the relationship between social security policies and psychological health.

Conclusions

This review showed that health and retirement defined in social security policies are related and have an impact on people’s lives, especially in the decision to leave the labor market. Therefore, measures to assess the possible consequences of this relationship when promoting reforms on social security policies should be encouraged.

Peer Review reports

Population aging is poised to become one of the most significant social transformations of the 21st century, with implications for every sector of society, including the labor and financial markets, the demand for goods and services such as housing, transportation, social protection, family structures and intergenerational ties [ 1 ]. It is estimated that by 2050, there will be 1.5 billion people aged 65 and older worldwide, more than doubling the number of individuals in this age group in the year 2020 [ 2 ]. The percentage of older people in the global population is expected to increase from 9.3% in 2020 to 16.0% in 2050, indicating that by the middle of the 21st century, one in six people worldwide will be 65 years of age or older [ 2 ].

The World Health Organization [ 3 ] conceptualizes elderly people based on age criteria for research purposes. Based on this criterion, an elderly person is one who is aged 60 or over who lives in developing countries, and one who is aged 65 or over who lives in developed countries. Additionally, it is recognized that aging is a continuous, multidimensional and multidirectional process of changes dictated by the concurrent action of the genetic-biological and socio-cultural determinants of the life cycle [ 4 , 5 ].

As people age, they experience a gradual decrease in physical and mental capacity, and a growing risk of disease and death which, at a biological level, results from the accumulation of molecular and cellular damage over the course of the lifetime [ 6 ]. These health conditions can be defined as the circumstances in the health of individuals that require responses from health systems professionals and users [ 7 ]. It can generate a disabling process and significantly compromise the quality of life of the elderly.

Beyond biological changes, the current context is geared towards producing a more favorable social and cultural environment for healthy aging and it is the role of public policies to help more people reach old age in the best possible state of health [ 8 ]. In order to reach that, aging is often associated with other life transitions, such as retirement. The aspects that determine retirement are interconnected to the individual’s life story, permeated by the combination of identity, family, friendship, work relationships, and professional career [ 9 , 10 , 11 , 12 ]. Nevertheless, to achieve this, it is necessary to be part of a social security system responsible for managing the granting and payment of pensions.

In all sorts of retirement, the economic situation of the state and the availability of similar social/welfare benefits can influence its meaning and consequences, since retirement must be thought about and sought after from a young age [ 13 ]. Several types of Welfare State regimes represent different responsibilities assumed by the market, the state and the family in the management of social risks and social security [ 14 ]. Previous research shows that countries with the most comprehensive Welfare State, such as Denmark, Sweden, and Norway, have better population health outcomes when compared to Neoliberal States such as the United States and the United Kingdom [ 15 , 16 ].

The discussion about social security policies can be located between the fields of health of the elderly and workers’ health, considering that the experience of this period does not occur in isolation, but is interconnected, among other factors, to their professional trajectory and to the different stages that make up the life cycle. Researchers have continued to show a strong link between older workers, health, planned retirement age [ 17 , 18 , 19 ], current retirement behaviors [ 20 , 21 , 22 ], and adjustment and satisfaction with post-retirement life [ 23 , 24 , 25 , 26 ]. In this paper, we aimed to capture current evidence in a systematic review to understand how health conditions in the aging process are related to social security reforms.

Search strategy and selection criteria

The search procedures for the studies took place between September 2021 and March 2022, with the last search being carried out on March 3, 2022. This systematic review aligns with the PRISMA checklist [ 27 , 28 ] and methods are outlined in detail in a protocol registered a priori on PROSPERO (CRD42021225820). Likewise, a protocol article was published in a peer-reviewed journal [ 29 ].

Eligibility was based on the Population, Intervention, Comparison, Outcomes (PICO) framework, with studies included if they met the following criteria: (1) participants who are in the process of transition to retirement or retired; (2) examined retirement guarantees as intervention/exposure which could be pension benefits, health insurance, subsidized assistance and other contributory schemes; (3) outcomes measured by quantitative methods that analyze the association or influence of social security policies on any outcome related to mental or physical health, such as psychological symptoms, mental disorders, illnesses, well-being. (4) original empirical studies published in English, Spanish, French and/or Portuguese, as these were the most common languages in the research, between 1979 and 2022 that examined aging from health conditions related to social security policies. Studies that identified any results associated with mental health and/or physical health, such as psychological symptoms, mental disorders, illnesses, well-being were included. The choice of 1979 to begin the search is due to the change in policies adopted by countries from a Welfare State to a neoliberal structure, marked by the election of Margaret Thatcher in the United Kingdom in May 1979.

Searches using the indexed terms “social security” AND “aging” were conducted across Embase, Web of Science, Scopus, Pubmed, CINAHL, ASSIA (Proquest) and APA PsycNet. Table  1 presents the full search criteria. Two independent reviewers (LT, FU) screened titles and abstracts for eligibility and studies that met criteria on title and abstract, underwent full text review. Using an excel spreadsheet, data from all studies were then independently extracted by the two reviewers (LT, FU), characteristics of the study (year of publication, study location, author); study design (longitudinal study, cross-sectional, case-control, other); sample size; participant characteristics (age, sex, years of education, marital status); method of data collection; method of analysis; instruments (health conditions measurements and retirement measurements) and the main conclusions of the study.

The PRISMA flowchart in Fig.  1 shows that 8,758 records were found in the databases. 1,336 duplicates were removed by automation tool, leaving 7,422 articles for title and abstract screening. Of these, 72 articles underwent full-text assessment and 17 met eligibility criteria and were included.

PRISMA Flowchart

Data analysis

Risk of bias and study quality.

Risk of bias and study quality was assessed at study-level using the Newcastle-Ottawa Scale (NOS) for cross-sectional and observational studies [ 30 ]. The NOS scale employs a star system by means of a checklist consisting of three criteria: (a) Selection: where the representativeness of the participants is assessed by analyzing the sampling and sample formation processes; (b) Comparability: where the confounding factors adjusted for sample analysis are identified; and (c) Result: where the evaluation and analysis of the results are verified. According to the scoring system, studies are scored in a range from 0 to 10 points and classified as low (10 and 9 points), medium (7 and 8 points), or high (< 7 points) risk of bias. Higher scores represent better quality. Overall, the NOS scale demonstrates good inter-rater and test-retest reliability [ 31 , 32 ].

Meta-ethnography

A narrative synthesis was performed using the meta-ethnography [ 33 ], which helps synthesizing the studies by combining the results found in an interpretive and non-aggregative way, to generate a higher level of analysis that produces a more relevant contribution than the individual findings of each investigation. Categories were created through thematic analysis of the data considering the evidence found in the selected studies.

Initial synthesis involved extraction of each paper findings, key concepts, metaphors and themes to determine how the studies are related to one another, and to develop descriptive codes. The key themes and relationships from the selected studies were tabulated. A translational process was then be undertaken to synthesize the findings using reciprocal analysis to create themes. The final findings were reported in a clear and concise manner to provide readers with a clear understanding of how we arrived at our findings. All stages were undertaken collaboratively by the research team. Data synthesis were independently undertaken by two reviewers (LT, FU); with a third author (JP) used for consensus as appropriate. The eMERGe meta-ethnography reporting guidance was followed [ 34 ].

Role of the funding source

The funders had no role in study design, data collection, analysis, interpretation, or writing. The corresponding author had full access to all data and final responsibility for the decision to submit for publication.

Seventeen cohorts of adults and elderly people were analyzed from the following countries: Australia, Austria, Belgium, Canada, China, Czech Republic, Denmark, Estonia, France, Germany, Greece, Hungary, India, Ireland, Italy, Japan, Netherlands, Philippines, Poland, Portugal, Slovenia, South Korea, Spain, Sweden, Switzerland, United Kingdom, and United States. The sample range varied between 80 and 18,345 individuals with an age range of 30 to 87 years. Of the 17 studies, 10 reported cross-sectional data analysis, 1 reported cross-sectional and longitudinal analysis and 5 analyzed longitudinal data. The main characteristics and results of the studies are presented in Table  2 .

A predominance of studies was carried out in the European continent (73.58%), with the largest number of studies concentrated in Sweden. In the Americas, in turn were 13.20% of the studies concentrated in the United States and Canada, followed by 11.32% in Asia and 1.88% in Oceania. However, there is a lack of research in African regions and in Latin and South American countries. Identified studies evaluated the relationship between health conditions that are more common in older adults, retirement and social security policy reforms - particularly those related to retirement - and were published between 1995 and 2021. Individual study risk of bias assessment is presented in Table  3 .

Health as a way to promote an active working life for the elderly

The perception that individuals have about their health condition and their permanence in the labor market is related. Four studies brought results suggesting this relationship [ 35 , 36 , 37 , 38 ]. In all studies, a good perception of health in general scope was found to be a determining factor for remaining in the labor market. Although retirement is an expected event, many older people would consider staying in the labor market for longer if there were better working conditions, such as additional senior citizen days, longer vacations, flexible work hours, and if the work was less physically demanding [ 38 ]. Also, unionized workers reported that favoring of prolonging work is not out of sheer necessity, but rather, because the expression of this desire comes from work attachment and professional identification [ 35 ]. Retirees who were in excellent health retired from their career jobs, were more likely to take bridge jobs, that bridge the gap between full-time employment and complete withdrawal from the labor force [ 36 ]. Workers who reported fair or poor physical health were less likely to remain employed after the ages of 62 and 65, moreover, there was a gradual decline in self-reported health and worsen health conditions over time [ 37 ]. According to the data found, a good self-reported health status is a factor that promotes the extension of elderly individuals in the labor market, despite meeting the legal eligibility criteria for retirement.

Health as an indicator for reforms in social security policies

Health conditions were associated with changes in countries’ laws about the eligibility criteria for receiving social security benefits. Four studies explored how health conditions could work as indicators for social security policy reforms [ 39 , 40 , 41 , 42 ]. The studies considered the following health conditions: subjective well-being, life satisfaction, and health status and related them to changes in social security of the countries subject to their analysis.

An increase in pension insecurity is associated with a reduction in life satisfaction, and it is a negative and significant relationship. The individuals most affected by pension insecurity are those who are further away from their retirement, have lower incomes, rate their life expectancy as low, have higher cognitive abilities, and do not expect private pension payments. However, while younger cohorts have more time to adapt to new pension systems or accumulate other types of savings, individuals that will retire in the foreseeable future are at risk of needing to work longer or receive lower pensions [ 40 ]. In a long term, increasing the age of formal retirement is relatively neutral with regard to subjective well-being, and suggests that later formal retirement simply delays the benefits to be enjoyed at retirement [ 39 ]. Employment rates increased in the 50–59 age group with welfare reform, but only among healthy individuals, with the odds ratio for receiving temporary benefits or not being eligible for benefits increasing for people with moderate to severe health problems [ 41 ]. Companies that aim to extend working time, where the social environment is more advantageous to their continuation after achieving the legal retirement age, and/or those who do not have experience with age discrimination, adjust more easily to the increase in retirement age. Likewise, employees with poor health have more difficulty adjusting to this augmentation, and better health status is related to fewer negative emotions and thoughts about prolonged employment, but also to increase behavior to facilitate a longer working life [ 42 ].

These results indicate that health conditions may be associated with the enhancement in the legal retirement age criterion. A good health condition can help individuals to adapt to the changes generated by the reforms. Also, there is a significant cost to people with poor health and to those who are farthest from retirement, despite presenting a certain neutrality with regard to the positive health of those who are near to retirement when a reform is sanctioned. Thus, when amending criteria to extend time in the labor force to solve fiscal problems, policymakers should analyze the impact on the health of individuals who are forced to postpone retirement, which corroborates its use as an indicator for social security policies, according to the demands of its population.

Retirement planning as a strategic element for coping with post-retirement life

Well-being in retirement is directly related to the attitudes of workers throughout their lives. Four studies looked at the relationship between individuals’ retirement planning during the aging process for their benefit receipt and their health conditions in old age [ 43 , 44 , 45 , 46 ].

Social and financial perceptions of post-retirement life were identified as factors that significantly influence retirement planning. On social perceptions, the major components that influence retirement planning detected were depression, role clarity of retired people and social involvement. About financial perceptions, the components identified were financial obligations, government support during retirement, uncertainty from financial perceptions and preparation for post-retirement life [ 46 ]. Therefore, contentment and security with participants’ financial situation exert an important factor for retirement preparedness [ 43 , 46 ]. In this sense, people who actively planned for retirement were much more likely to have a high net worth, personal savings or investment, or a defined contribution plan as their primary source of retirement income, and much less likely to have a low net worth. People who actively planned for retirement were less likely to have the government insurance plan as their primary source. Nevertheless, there was no significant difference between people who actively planned for retirement and people who did not in the percentage of poor health. Most respondents identified their health as excellent or good, except for individuals with no retirement and a low level of wealth whose showed a considerable decrease in QoL compared to individuals with retirement and a low level of wealth [ 44 , 45 ].

To have a retirement planning during life, and consequently the coverage by a pension plan, can help positively in the post-retirement life, especially in the individual’s perceptions, whether they are social, health or financial. Such help is mainly due to the psychological perceptions of financial issues that may influence how the individual will experience his or her old age. Then, social security planning can work as a strategy for coping with post-retirement life, since it not only prepares workers to meet their needs, but also supports them in the face of concerns about the losses of this phase of life.

The relationship between social security policies and psychological health

A total of five studies have analyzed the relationship of social security with psychological health, investigating symptoms of depression [ 47 , 48 , 49 , 50 ], anxiety [ 51 ] and stress [ 49 ].

Lower job control and poorer self-reported health lead to a lower retirement age, also, the risk of depressive symptoms is increased for people with a lower level of education [ 47 , 49 ]. In addition, greater satisfaction of the needs for autonomy, competence, and relatedness was related to less depressive symptoms at baseline. However, satisfaction of pre-retirement needs was not a statistically significant predictor of subsequent changes in depressive symptoms throughout the transition to retirement. As for the basic psychological needs, only autonomy showed statistical significance, which demonstrated the existence of an initial short-term increase throughout the transition to retirement [ 50 ]. Besides, workers reported being in better health, less depressed, with more energy, fewer chronic conditions, and fewer limitations in their activities. Those who were retired reported feeling more bored, helpless, and hopeless [ 47 ]. Furthermore, being absent from the workforce through early retirement due to depression and other mental health disorders results in considerably less income than being in the workforce full time, as well as less wealth than those who have no mental health condition [ 48 ]. Regarding anxiety, a cross-country study suggests that the development of a social security system where the individual holds coverage for living expenses after retirement and health care decreases people’s concern about the future [ 51 ].

The results indicate that there is a link between psychological health and social security policies established when individuals decide to take early retirement, as a result of symptoms such as depression and stress, which generate a labor disability, and the need to activate the social security protection system due to a forced exit from the labor market. As well, the opposite logic can be seen where the existence of a robust social security system that provides coverage for life’s adversities, such as illness and old age, reduces symptoms such as anxiety.

This meta-ethnography identified 17 eligible studies that examined the relationship between health conditions associated with aging and social security policies among people nearing retirement or retired. Most of the studies included in this systematic review involved cohorts aged 40 years or older and investigated associations between social security policies for people of retirement age and perceptions of, or behaviors related to, general health, psychological health or physical functioning. The synthesis of the evidence suggests that health can operate as a way to promote the working life for the elderly and as an indicator for social security policy reforms, that retirement planning is a strategic element for coping with post-retirement life, and that there is a relationship between social security policies and psychological symptoms.

About health as a way to promote the working life, four studies have found that changes related to sociodemographic dynamics point out that the phase between the ages of 50 and 70 has emerged as a type of second part of working life, which can be supported by a good self-assessment of the subject’s general health status when perceiving the possibility of staying in the labor market, albeit in an adapted way, such as by adopting bridge jobs [ 35 , 36 , 37 , 38 ]. The evidence suggests that if people can experience their old age in good health, they can be productive, still work and contribute to society, in a slightly different way from that of a younger person, promoting independence and increasing a healthy life for the elderly.

When it comes to health as indicator for social security reforms, of all the studies included in the synthesis, four studies allowed us to identify that a good health status can help individuals adapt to the changes generated by the reforms of the legal age criterion in the social security models and that people in poor health are the ones who suffer most from the crisis caused by unexpected changes in the welfare system [ 39 , 40 , 41 , 42 ]. This result is consistent with the literature reviewed, which has observed a variation in the health behavior of workers and in the health conditions of the samples researched that approaches social security reforms [ 52 , 53 , 54 ]. The results indicate that to ensure a healthy aging population, when reforming social security systems, policymakers have to enhance positive impact on health, since social protection aims to provide income security, health care and support at every stage of life, with particular attention to the most marginalized. However, the underlying mechanisms by which social security reforms appear to have this effect on health have not been evidenced, which may reflect an empirical evidence gap that is possibly developing.

Moreover, four studies included in the review enabled to indicate that there are actions in the life course that can help to obtain a satisfactory health after leaving the labor market, such as retirement planning; which according to the results found can reduce worry about retirement, keep anxiety under control, improve income and quality of life in the realization of this life event [ 43 , 44 , 45 , 46 ]. Retirement planning is defined as a goal-oriented behavior in which individuals devote efforts to prepare for their withdrawal from the labor market [ 25 ]; that could function as a strategic element for coping with post-retirement life.

Regarding the relationship between social security policies and psychological health, four studies suggested that the presentation of symptoms such as depression and stress, may demand from the social security system, as they are capable of disabling individuals, who will have a forced exit from the labor market [ 47 , 48 , 49 , 50 ]. And a cross-sectional study allowed us to infer that in countries where the level of development and comprehensiveness of its security system is higher, its population presents a lower anxiety picture when participants are asked about old age [ 51 ]. This is consistent with previous literature, where better health outcomes have been found in countries with a more extensive welfare state [ 15 , 16 ]. These findings support the idea that mental health should be thought about and promoted, especially in the workplace, once social environments can affect health. A public-health guideline to aging should consider approaches that reinforce rehabilitation, adaptation and psychosocial growth.

In general, a significant number of studies have employed self-reported instruments to measure health conditions when considered in their general aspect [ 35 , 43 , 47 , 48 ], which supports the importance of self-report as a meaningful indicator of health status. The increasing validity and adaptability of self-assessment scales have enhanced their use for academic, clinical, research, and epidemiological purposes, offering adequate levels of reliability in measuring and prognosticating short- and long-term measures of health [ 55 ]. Furthermore, the results found in this review can help to create the environments and opportunities that enable people to be and do what they value throughout their lives, increasing wellbeing and participation in society and promoting a healthy aging.

About the limitations of this review, the cross-sectional analysis of most studies restricts the validity of the results, as this prevented us from examining the cause-and-effect relationship of the variables. Also, considerable methodological variation was found in the theoretical perspectives consulted, the follow-up periods, and the questionnaires used in the studies to assess health conditions and social security measures, which hampered the meta-analytic analysis. This could have improved the interpretation and generalizability of the results and thus provided greater validity of the evidence.

The difficulty in defining and measuring retirement was also noted. On a conceptual level, a variety of theoretical approaches were found that operationalized retirement through self-report, legal concept, labor force participation, and pension receipt. However, this theoretical-conceptual variation may not be problematic as these approaches are not mutually exclusive as each assesses and analyzes a particular component of what is meant by retirement.

In spite of this significant heterogeneity in results, the multifaceted nature of health and social security allowed us to find a substantial amount of research that worked on their relationship, and made it possible to conduct the meta-ethnography. 58.82% of the studies had a low assessment score, i.e., a high risk of bias, represented by the lack of representativeness of the samples, the predominant use of self-assessment scales, and low risk factor verification. Finally, the selected publications were only from 1995 on, although our search covered research published from 1979 onwards, mainly due to the low methodological quality of the studies found in this period and the scarce quantity of studies detected between 1979 and 1994, revealing an increase in academic production and its publication from the mid-1990s.

Despite the limitations, the main strength of this systematic review was to conduct an analysis of health conditions related to social security policy reforms, synthesizing the evidence reported in a substantial number of relevant studies. These studies covered diverse population-based cohorts in large samples of middle-aged and elderly individuals, demonstrating the appropriate applicability of the theoretical construct of social security policies in diverse cultural contexts and methodological advances in the development and validation of outcome measures. This reflects not only the growing interest in research on variables based on human experience, but also in the search for empirical evidence to support the contribution of multidisciplinary constructs directed at public policy. At last, the searches of studies in four languages - English, Portuguese, French and Spanish - facilitated the understanding of the relationship between health conditions and social security policy reforms in samples of middle-aged and elderly participants from different cultures.

The results of this review included important health domains such as general health functioning, psychological health, and work disability factors. Overall, it showed that there is a link between health and retirement, where health is a relevant factor in deciding when to exit the labor market. This may encourage future researchers and policy makers to analyze the ramifications of its relationship to advance the promotion of quality of life for the elderly population.

For future research, the need arises to study and analyze the underlying mechanisms through which social security policy reforms and health conditions are related. Likewise, their potential benefits could be assessed through interventions aimed at promoting health for older workers, preventing psychological symptomatology, and planning for retirement. At the theoretical level, the conceptual diversity of retirement could represent an opportunity to operationalize this variable as a multifaceted construct, which could improve its explanatory and interpretive capacity in the face of different health outcomes for aging.

Data availability

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Abbreviations

Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols

Newcastle-Ottawa Scale

United Nations. Resolution 75/131. United Nations Decade of Healthy Ageing (2021–2030), 2020.

United Nations. World Population Ageing 2020 highlights: living arrangements of older persons. Population Division: Department of Economic and Social Affairs; 2020.

Book   Google Scholar  

World Health Organization. Active ageing: a policy framework, 2002.

Pereira A, Freitas C, Mendonça C et al. Envelhecimento, estresse e sociedade: uma visão psiconeuroendocrinológica. Ciências & Cognição. 2004(1): 34–35.

Baltes PB, Smith J. Lifespan psychology: from Developmental Contextualism to Developmental Biocultural Co-constructivism. Res Hum Dev. 2004. https://doi.org/10.1207/s15427617rhd0103_1 .

Article   Google Scholar  

Sadighi Akha AA. Aging and the immune system: an overview. J Immunol Methods. 2018;463:21–6. https://doi.org/10.1016/j.jim.2018.08.005 .

Article   CAS   PubMed   Google Scholar  

Mendes EV. O cuidado das condições crônicas na atenção primária à saúde: o imperativo da consolidação da estratégia da saúde da família. Brasília: Organização Pan-Americana da Saúde; 2012.

Google Scholar  

Beard JR, Officer A, de Carvalho IA, et al. The World report on ageing and health: a policy framework for healthy ageing. Lancet. 2016;387(10033):2145–54. https://doi.org/10.1016/S0140-6736(15)00516-4 .

Article   PubMed   Google Scholar  

Santos MFS. Identidade E aposentadoria. São Paulo: EPU; 1990.

Carlos SA, Jacques M, da GC, Larratéa SV, Herédia OC, Identidade. Aposentadoria e Terceira Idade. Estud. interdiscip. envelhec. 1. https://doi.org/10.22456/2316-2171.4653

Denton FT, Spencer BG. What is Retirement? A Review and Assessment of Alternative concepts and measures. Canadian Journal on Aging / La Revue Canadienne Du vieillissement. Volume 28. Cambridge University Press; 2009. pp. 63–76. 110.1017/S0714980809090047.

Santos J, Moraes B. Aposentados que trabalham: fatores relacionados a permanência no mercado de trabalho / retired workers: factors related to permanence in the labor market. Revista Interinstitucional Brasileira De Terapia Ocupacional – REVISBRATO. 2020;4(4):587–602. https://doi.org/10.47222/2526-3544.rbto25552 .

Guillemard A-M. Les défis Du vieillissement. Âge, Emploi, retraite, perspectives internationales. 2ª. ed. Paris: Armand Colin; 2010.

Aysan MF, Beaujot R. Welfare regimes for Aging populations: no single path for reform. Popul Dev Rev. 2009;35(4):701–20. https://doi.org/10.1111/j.1728-4457.2009.00305.x .

Bambra C. Health Status and the Worlds of Welfare. Social Policy and Society. Volume 5. Cambridge University Press; 2006. pp. 53–62. 110.1017/S1474746405002721.

Chung H, Ng E, Ibrahim S, Karlsson B, Benach J, Espelt A, et al. Welfare State regimes, gender, and Depression: a Multilevel Analysis of Middle and High Income Countries. Int J Environ Res Public Health. 2013;10:1324–41. https://doi.org/10.3390/ijerph10041324 .

Article   PubMed   PubMed Central   Google Scholar  

Sewdas R, de Wind A, van der Zwaan LGL, et al. Why older workers work beyond the retirement age: a qualitative study. BMC Public Health. 2017;17(1):672. https://doi.org/10.1186/s12889-017-4675-z .

Carrino L, Glaser K, Avendano M. Later retirement, job strain, and health: evidence from the new state pension age in the United Kingdom. Health Econ. 2020;29(8):891–912. https://doi.org/10.1002/hec.4025 .

Pilipiec P, Groot W, Pavlova M. The Effect of an increase of the Retirement Age on the Health, Well-Being, and Labor Force Participation of Older Workers: a systematic literature review. Popul Ageing. 2021;14:271–315. https://doi.org/10.1007/s12062-020-09280-9 .

Muller CF, Boaz RF. Health as a reason or a rationalization for being retired? [published correction appears in Res Aging. 1988;10(2):234]. Res Aging; 1988;10(1):37–55. https://doi.org/10.1177/0164027588101002

Kimiyagahlam F, Safari M, Mansori S. Influential behavioral factors on retirement planning behavior: the case of Malaysia. J Financial Couns Plann. 2019;30(2):244–61. https://doi.org/10.1891/1052-3073.30.2.244 .

Liu C, Bai X, Knapp M. Multidimensional Retirement Planning behaviors, Retirement confidence, and Post-retirement Health and Well-Being among Chinese older adults in Hong Kong. Appl Res Qual Life. 2022;17:833–49. https://doi.org/10.1007/s11482-020-09901-7 .

Beck SH. Adjustment to and satisfaction with retirement. J Gerontol. 1982;37(5):616–24. https://doi.org/10.1093/geronj/37.5.616 .

Hawash M, Khalil M. Factors affecting post-retirement satisfaction of Community-Dwelling older adults. Alexandria Sci Nurs J. 2017;19(2):103–18. https://doi.org/10.21608/asalexu.2017.208354 .

Yeung DY, Zhou X. Planning for Retirement: Longitudinal Effect on Retirement resources and Post-retirement Well-being. Front Psychol. 2017;8:1300. https://doi.org/10.3389/fpsyg.2017.01300 .

Rodrigues JFC, Barreto S, Griep RH, Fonseca MJM, Camelo LV, Giatti L. Is retirement associated to greater life satisfaction? Cross-sectional findings from ELSA-Brasil. Cadernos De saúde pública. 2022;38(2). https://doi.org/10.1590/0102-311X00034521 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097 .

Moher D, Shamseer L, Clarke M. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1). https://doi.org/10.1186/2046-4053-4-1 .

Teixeira LMO, Uribe FAR, Moreira HLF, Pedroso JS. Mudanças - Psicologia da Saúde. 2023;31:57–64. https://doi.org/10.15603/2176-1019/mud.v31n1p57-64 . Envelhecimento e Políticas de Seguridade Social: protocolo de revisão sistemática.

Wells G, Shea B, O’Connell D et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomized studies in meta-analyses. Ottawa, ON, Ottawa Hospital Research Institute; 2011. Retrieved from http://www.ohri.ca/programs/clinical_epidemiology/oxford.as

Oremus M, Oremus C, Hall GBC, McKinnon M. Inter-rater and test–retest reliability of quality assessments by novice student raters using the Jadad and Newcastle–Ottawa scales. BMJ Open. 2012;2:e001368. https://doi.org/10.1136/bmjopen-2012-001368 .

Margulis AV, Pladevall M, Riera-Guardia N, et al. Quality assessment of observational studies in a drug-safety systematic review, comparison of two tools: the Newcastle–Ottawa Scale and the RTI item bank. Clin Epidemiol. 2014;6:359–68. https://doi.org/10.2147/CLEP.S66677 .

Noblit G, Hare D. Meta-ethnography: synthesizing qualitative studies. Newbury Park, California: Sage; 1988.

France EF, Cunningham M, Ring N, et al. Improving reporting of meta-ethnography: the eMERGe reporting guidance. Psycho Oncol. 2019;28(3):447–58. https://doi.org/10.1002/pon.4915 .

Guérin G, Wils T, Saba T, St-Jacques N. (1995). Retraite anticipée ou extension de la vie professionnelle? Les aspirations des professionnels syndiqués de 50 ans et plus. Cahiers québécois de démographie, 1995;24(2):245–283. https://doi.org/10.7202/010189ar

Cahill KE, Giandrea MD, Quinn JF. Retirement patterns from career employment. Gerontologist. 2006;46(4):514–23. https://doi.org/10.1093/geront/46.4.514 .

Dong XS, Wang X, Ringen K, Sokas R. Baby boomers in the United States: factors associated with working longer and delaying retirement. Am J Ind Med. 2017;60(4):315–28. https://doi.org/10.1002/ajim.22694 .

Andersen LL, Jensen PH, Sundstrup E. Barriers and opportunities for prolonging working life across different occupational groups: the SeniorWorkingLife study. Eur J Pub Health. 2020;30(2):241–6. https://doi.org/10.1093/eurpub/ckz146 .

Horner EM. Subjective well-being and retirement: analysis and policy recommendations. J Happiness Studies: Interdisciplinary Forum Subjective Well-Being. 2014;15(1):125–44. https://doi.org/10.1007/s10902-012-9399-2 .

Olivera J, Ponomarenko V. Pension insecurity and wellbeing in Europe. J Social Policy Camb Univ Press. 2017;46(3):517–42. https://doi.org/10.1017/S0047279416000787 .

Jensen NK, Brønnum-Hansen H, Andersen I, et al. Too sick to work, too healthy to qualify: a cross-country analysis of the effect of changes to disability benefits. J Epidemiol Community Health. 2019;73(8):717–22. https://doi.org/10.1136/jech-2019-212191 .

Mulders JO, Henkens K, van Dalen HP. Employees’ emotional, cognitive, and behavioral responses to increasing Statutory Retirement ages. Biomed Res Int. 2021. https://doi.org/10.1155/2021/6645271 .

Brucker E, Leppel K. Retirement plans: planners and nonplanners. Educ Gerontol. 2013;39(1):1–11. https://doi.org/10.1080/03601277.2012.660859 .

Ju YJ, Han KT, Lee HJ, et al. Quality of life and national pension receipt after retirement among older adults. Geriatr Gerontol Int. 2017;17(8):1205–13. https://doi.org/10.1111/ggi.12846 .

De los Santos JAA, Labrague LJ, Milla NE. Happiness and Retirement Readiness among Pre-retiring employees: a cross-sectional study. Ageing Int. 2020;45:85–98. https://doi.org/10.1007/s12126-019-09351-z .

Murari K, Shukla S, Adhikari B. Do psychological social and financial perceptions of post-retirement life and demography influence the retirement planning behaviour? Int J Soc Econ. 2021;48(11):1545–66. https://doi.org/10.1108/IJSE-08-2020-0581 .

McMahan S, Phillips K. Aging and employment: characteristics of those Working and retired in California. J Educ Bus. 2000;76(1):11–4. https://doi.org/10.1080/08832320009599043 .

Schofield DJ, Shrestha RN, Percival R, Kelly SJ, Passey ME, Callander EJ. Quantifying the effect of early retirement on the wealth of individuals with depression or other mental illness. Br J Psychiatry Camb Univ Press. 2011;198(2):123–8. https://doi.org/10.1192/bjp.bp.110.081679 .

Mäcken J. Work stress among older employees in Germany: effects on health and retirement age. PLoS ONE. 2019;14(2):e0211487. https://doi.org/10.1371/journal.pone.0211487 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Stenling A, Henning G, Bjälkebring P, Tafvelin S, Kivi M, Johansson B, et al. Basic psychological need satisfaction across the retirement transition: changes and longitudinal associations with depressive symptoms. Motivation Emot. 2021;45:75–90. https://doi.org/10.1007/s11031-020-09854-2 .

Kadoya Y. What makes people anxious about life after the age of 65? Evidence from international survey research in Japan, the United States, China, and India. Rev Econ Househ. 2016;14:443–61. https://doi.org/10.1007/s11150-015-9310-0 .

Simpson J, Albani V, Bell Z, Bambra C, Brown H. Effects of social security policy reforms on mental health and inequalities: a systematic review of observational studies in high-income countries. Soc Sci Med. 2021;272:113717. https://doi.org/10.1016/j.socscimed.2021.113717 .

Shai O. Is retirement good for men’s health? Evidence using a change in the retirement age in Israel. J Health Econ. 2018;57:15–30. https://doi.org/10.1016/j.jhealeco.2017.10.008 .

Bertoni M, Brunello G, Mazzarella G. Does postponing minimum retirement age improve healthy behaviors before retirement? Evidence from middle-aged Italian workers. J Health Econ. 2018;58:215–27. https://doi.org/10.1016/j.jhealeco.2018.02.011 .

Schnittker J, Bacak V. The increasing predictive validity of self-rated health. PLoS ONE. 2014;9(1):e84933. https://doi.org/10.1371/journal.pone.0084933 .

Download references

Acknowledgements

The authors would like to thank all those who contributed to the elaboration of this systematic review.

This systematic review is supported by the Universidade Federal do Pará, Brasil/Pró- Reitoria de Pesquisa e Pós-graduação (PROPESP) and the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – CAPES (finance code 001).

Author information

Authors and affiliations.

Instituto de Filosofia e Ciências Humanas, Universidade Federal do Pará, Rua Augusto Corrêa, 01- Guamá, Belém, Pará, CEP 66015-110, Brazil

Laíze Marina de Oliveira Teixeira, Fabio Alexis Rincón Uribe & Janari da Silva Pedroso

Instituto de Ciências Jurídicas, Universidade Federal do Pará, Belém, Pará, Brazil

Hélio Luiz Fonseca Moreira

You can also search for this author in PubMed   Google Scholar

Contributions

L.T. contributed to the concept, data curation, formal analysis and writing - original draft of this systematic review. F.U. contributed to the data curation, formal analysis and writing - review & editing draft. H.M. contributed to the investigation, supervision and writing - review & editing draft. J.P. contributed to the investigation, supervision, methodology and writing - review & editing draft. All authors were involved in the overarching protocol, interpretation and theoretical underpinning of the data. All authors reviewed the manuscript. Finally, all authors approved the final version for publication.

Corresponding author

Correspondence to Laíze Marina de Oliveira Teixeira .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

de Oliveira Teixeira, L.M., Uribe, F.A.R., Moreira, H.L.F. et al. Associations between retirement, social security policies and the health of older people: a systematic review. BMC Public Health 24 , 2473 (2024). https://doi.org/10.1186/s12889-024-19979-5

Download citation

Received : 28 April 2023

Accepted : 04 September 2024

Published : 11 September 2024

DOI : https://doi.org/10.1186/s12889-024-19979-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Social security
• Public policies

BMC Public Health

ISSN: 1471-2458

• General enquiries: [email protected]

• Systematic Review
• Open access
• Published: 06 September 2024

Spine muscle auto segmentation techniques in MRI imaging: a systematic review

• Hyun-Bin Kim 1 ,
• Hyeon-Su Kim 1 ,
• Shin-June Kim 1 &
• Jun-Il Yoo 2  

BMC Musculoskeletal Disorders volume  25 , Article number:  716 ( 2024 ) Cite this article

Metrics details

The accurate segmentation of spine muscles plays a crucial role in analyzing musculoskeletal disorders and designing effective rehabilitation strategies. Various imaging techniques such as MRI have been utilized to acquire muscle images, but the segmentation process remains complex and challenging due to the inherent complexity and variability of muscle structures. In this systematic review, we investigate and evaluate methods for automatic segmentation of spinal muscles.

Data for this study were obtained from PubMed/MEDLINE databases, employing a search methodology that includes the terms 'Segmentation spine muscle’ within the title, abstract, and keywords to ensure a comprehensive and systematic compilation of relevant studies. Systematic reviews were not included in the study.

Out of 369 related studies, we focused on 12 specific studies. All studies focused on segmentation of spine muscle use MRI, in this systematic review subjects such as healthy volunteers, back pain patients, ASD patient were included. MRI imaging was performed on devices from several manufacturers, including Siemens, GE. The study included automatic segmentation using AI, segmentation using PDFF, and segmentation using ROI.

Despite advancements in spine muscle segmentation techniques, challenges still exist. The accuracy and precision of segmentation algorithms need to be improved to accurately delineate the different muscle structures in the spine. Robustness to variations in image quality, artifacts, and patient-specific characteristics is crucial for reliable segmentation results. Additionally, the availability of annotated datasets for training and validation purposes is essential for the development and evaluation of new segmentation algorithms. Future research should focus on addressing these challenges and developing more robust and accurate spine muscle segmentation techniques to enhance clinical assessment and treatment planning for musculoskeletal disorders.

Peer Review reports

Introduction

Musculoskeletal disorders, such as back pain and spinal deformities, have a significant impact on individuals' well-being, quality of life, and economy [ 1 ]. Spine muscles play a critical role in supporting the spine and transmitting forces within the musculoskeletal system [ 2 ]. Abnormalities or dysfunction in spinal muscles are often associated with musculoskeletal disorders. Accurate segmentation of spinal muscles is important for understanding the mechanisms underlying these disorders and for developing appropriate treatment strategies. While changes in muscle structure are typically a result of spine pathology rather than a cause, understanding these changes can provide valuable insights for both patients and physicians.

Medical imaging techniques, such as magnetic resonance imaging (MRI) are commonly used to acquire muscle images and analyze musculoskeletal structures [ 3 ]. These techniques provide detailed information about the morphology and composition of the spine muscles [ 4 ]. However, accurately segmenting spinal muscles in these images can be difficult due to several factors such as different data, and imaging protocols.

Several challenges make the segmentation of spine muscles complex. First, the complexity and variability of muscle structures, such as size, shape, and orientation, make it difficult to design a one-size-fits-all segmentation approach. Second, image artifacts, such as noise and partial volume effects, can degrade image quality and affect segmentation accuracy. Third, patient-specific variations, such as body posture and position, can introduce additional challenges in accurately delineating muscle boundaries. These challenges highlight the need for advanced and robust segmentation techniques.

Therefore, accurate segmentation of spine muscles is vital for understanding musculoskeletal disorders and designing effective rehabilitation strategies [ 5 ]. Advanced imaging techniques and computational algorithms have contributed to significant advancements in this field. However, challenges related to the complexity of muscle structures, image artifacts, and patient-specific variations still exist. The purpose of this systematic review is to evaluate the state of the art of spinal muscle segmentation using AI methods and identify optimal algorithms to identify areas for improvement to improve clinical evaluation and treatment planning for musculoskeletal disorders and apply them to further research.

Study eligibility criteria

The inclusion criteria for this study were as follows: (1) research unrelated to segmentation spine muscle, (2) studies written in English. The exclusion criteria were as follows: (1) studies that not used MRI to measure muscle, (2) studies not that did not meet other criteria. Figure  1 for more details.

PRISMA flow chart

Search method to identify appropriate studies

In this study, we conducted a literature search following the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) in the PubMed/MEDLINE library [ 6 ]. We searched for papers published from January 1992 to August 2023 using the following search term; segmentation spine muscle MRI. These search queries were employed to retrieve relevant articles for our research.

Data extraction

To conduct an analysis of relevant papers suitable for our study, the following variables were extracted: (i) Author; (ii) Year; (iii) Segmentation method; (iv) Subjects; (v) Data; (vi) Performance; Table 1 .

Ethical considerations

As this is a systematic review, ethical approval is not required. Confidential patient information will not be collected or used in this study.

After reviewing the abstracts and screening according to the PRISMA guidelines, we excluded 189 studies that were not relevant to spine muscle segmentation. Additionally, 0 studies not written in English were excluded. Furthermore, 127 studies that did not use MRI as a measurement equipment were excluded. We also excluded 41 studies that did not evaluate indicators that met the criteria. Finally, a total of 12 studies were included in our research scope [ 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 ]. The studies included in the systematic review were conducted between 1992 and 2023 and involved healthy volunteers, back pain patients, ASD patients. MRI imaging was performed on devices from several manufacturers, including Siemens, GE, and MEDSPEC. Studies included automatic segmentation using AI, segmentation using PDFF, and segmentation using ROI . Segmentation performance was higher AI method than other segmentation method. Most high DSC 0.91 was David Baur’s U-Net. (Table  2 ).

This systematic review provided insight into the different methods and outcomes of spinal muscle splitting. The identified segmentation techniques, including traditional image processing methods, statistical models, machine learning approaches, and deep learning-based algorithms, have shown promise in accurately segmenting spine muscles. Each technique has its advantages and limitations, and the choice of technique depends on the specific requirements of the segmentation task, including accuracy, computational complexity, and adaptability to different types of spine muscle images. Among the segmentation methods used in this systematic review, segmentation using AI showed the best performance. Among them, we compared how performance differs depending on the model and preprocessing method used. Tables  2  and   3 .

Advances in deep learning-based algorithms, especially CNN architectures, have significantly improved spinal muscle segmentation. David Baur developed a CNN to segment lumbar spinal muscles in lower back pain patients from consecutive MRI slices and classify fatty muscle degeneration automatically. The study used 100 lumbar spine MRIs with 3650 slices for automatic image segmentation. The U-Net-based network achieved high segmentation accuracy, particularly for overall muscle segmentation, with a Dice similarity coefficient (DSC) of 0.91. These algorithms have demonstrated outstanding performance by learning complex features directly from muscle images without the need for hand-crafted features.

Kenneth A.Weber, Madeline Hess’s T1 axial Muscle Segmentation uses V-Net. Kenneth A.Weber’s performance is (Left DSC:0.862 ± 0.017, Right DSC: 0.871 ± 0.016) lower than Madeline Hess’s performance (DSC:0.88). This is because the elements that make up v-net are different. Table 4 compares these differences. We also compared the performance of the 3D CNN and 2D CNN. In E. O. Wesselink's study, the objective was to compare the performance between 2D convolutional neural networks (CNNs) and 3D CNNs. While 2D CNNs are designed to extract features from 2-dimensional images, 3D CNNs do so from 3-dimensional volumetric data. In this study, data augmentation techniques were applied, and the True positive rate (TPR) for right-sided muscles specifically the multifidus, erector spinae, and psoas major was compared between the two models. As indicated in Fig.  2 , the 2D model demonstrated superior performance in identifying muscles when compared to the ground truth, outperforming the 3D model. The performance of the segmentation model varies depending on the presence and severity of spine pathology [ 19 ]. In Benjamin Dourthe's study, the Dice Similarity Coefficient (DSC) values for three specific Regions of Interest (ROI) were compared between healthy individuals and those with Adult Spinal Deformity (ASD). The ROIs included the vertebral body, psoas major, and multifidus erector spinae. The study uses data from five different sets to make an in-depth comparison of how well these anatomical regions are identified in both groups. Based on the analysis, the lumbar region in healthy individuals performed better in terms of ROI identification compared to those with ASD. Figure  3 .

Comparison of True Positive Rate for Right-sided Muscles: 2D vs 3D with Data Augmentation

Comparison of DSC Values for Healthy and ASD Lumbar Across Multiple Sets

Frank Niemeyer et al. [ 20 ] highlights the differences in segmentation performance between individuals with lumbar spine pathology, such as adult spinal deformity (ASD), and those without. Based on the provided data and the referenced study, there are several factors that could contribute to the observed differences in segmentation performance. One of the primary reasons for the difference in segmentation performance could be attributed to the higher heterogeneity of lumbar spine pathology in ASD patients. In healthy individuals, the anatomical structures are more consistent and predictable, allowing segmentation algorithms to perform better. However, in ASD patients, the anatomical structures are more varied due to the deformities and associated pathological changes. This variability makes it challenging for segmentation models to accurately identify regions of interest (ROI), leading to decreased performance. The difference in segmentation performance between healthy individuals and those with ASD can be primarily attributed to the higher heterogeneity and complexity of pathological anatomy in ASD patients.

The performance differences in spinal muscle segmentation algorithms can be attributed to several factors. such as model architectures, dataset sizes, and batch size. Different neural network architectures, U-Net, CNN, and V-Net, have unique structural characteristics that influence their performance. For instance, U-Net is designed for biomedical image segmentation and excels at capturing fine details and contextual information, whereas CNNs are more general-purpose and can vary significantly in their complexity and depth. The performance differences in spinal muscle segmentation algorithms can be attributed to a combination of hyperparameters, model architectures, and dataset characteristics. While the choice of hyperparameters such as learning rate, optimizer, activation function, and regularization techniques (dropout) significantly impact model performance, the dataset size and the specific loss functions used are equally crucial. To optimize segmentation performance, it is essential to carefully tune these parameters and consider the specific requirements of the task at hand. Future research could focus on systematically evaluating these factors across different models to establish more standardized guidelines for optimal performance in spinal muscle segmentation.

Spine muscle segmentation is crucial due to its pivotal role in the analysis of musculoskeletal disorders and the design of effective rehabilitation strategies. The reviewed studies showcased various segmentation techniques, with deep learning-based algorithms demonstrating superior performance. However, challenges related to accuracy, robustness, and dataset availability persist. CT imaging can also perform automatic segmentation of spinal muscles well. For example, among studies on automatic segmentation of spinal muscles in CT images, there is a study using Bayesian U-Net to investigate the relationship between the accuracy of muscle segmentation around the spine in torso CT images [ 21 ], and a method of 3D segmentation of skeletal muscles, including paraspinal muscles, by region in the L3 slice of body CT images using simultaneous learning using 2D U-Net [ 22 ], or multi-scale iterative random forest classification was used. A fully automated segmentation study of paraspinal muscles in 3D trunk CT images [ 23 ]. etc. There is this. These studies should consider incorporating both MRI and CT modalities in paravertebral muscle segmentation. CT imaging can be particularly useful for evaluating patient groups where MRI imaging is not feasible, such as those with pacemakers. Also, because the comparative segmentation methods of the included studies are all different, it cannot be concluded that the best algorithm among the studies is the artificial intelligence-based segmentation. In the future, a method to integrate all studies and conduct quantitative evaluation will need to be developed. Addressing these challenges will lead to more accurate segmentation techniques and enhance clinical assessment and treatment planning for musculoskeletal disorders.

Spinal muscle segmentation is a variety of techniques, ranging from traditional methods to deep learning algorithms such as David Baur's U-Net, have shown promise in accurately segmenting spinal muscles. Deep learning, in particular, excels at this task by learning complex features directly from images. Spinal muscle segmentation plays an important role in musculoskeletal disease analysis and rehabilitation planning. Deep learning has shown excellent performance, but issues related to accuracy, robustness, and dataset availability still remain. Addressing these challenges will further improve clinical evaluation and treatment strategies for musculoskeletal disorders.

Availability of data and materials

All data generated or analyzed during this study are included in this published article or are available from the corresponding author on reasonable request.

Roux CH, et al. Impact of musculoskeletal disorders on quality of life: an inception cohort study. Ann Rheum Dis. 2005;64(4):606–11. https://doi.org/10.1136/ard.2004.020784 .

Article   CAS   PubMed   Google Scholar  

Lynders C. The critical role of development of the transversus abdominis in the prevention and treatment of low back pain. HSS J. 2019;15(3):214–20. https://doi.org/10.1007/s11420-019-09717-8 .

Article   PubMed   PubMed Central   Google Scholar  

Dean Deyle G. The role of MRI in musculoskeletal practice: a clinical perspective. J Man Manip Ther. 2011;19(3):152–61. https://doi.org/10.1179/2042618611Y.0000000009 .

Crawford RJ, Fortin M, Weber KA, Smith A, Elliott JM. Are MRI technologies crucial to our understanding of spinal conditions? J Orthop Sports Phys Ther. 2019;49(5):320–9. https://doi.org/10.2519/jospt.2019.8793 .

Noonan AM, Brown SHM. Paraspinal muscle pathophysiology associated with low back pain and spine degenerative disorders. JOR Spine. 2021;4(3):e1171. https://doi.org/10.1002/jsp2.1171 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097

Niemeyer F, Zanker A, Jonas R, Tao Y, Galbusera F, Wilke HJ. An externally validated deep learning model for the accurate segmentation of the lumbar paravertebral muscles. Eur Spine J. 2022;31(8):2156–64. https://doi.org/10.1007/s00586-022-07320-w .

Article   PubMed   Google Scholar  

Baur D, et al. Analysis of the paraspinal muscle morphology of the lumbar spine using a convolutional neural network (CNN). Eur Spine J. 2022;31(3):774–82. https://doi.org/10.1007/s00586-021-07073-y .

Dourthe B, et al. Automated segmentation of spinal muscles from upright open MRI using a multiscale pyramid 2D convolutional neural network. Spine (Phila Pa 1976). 2022;47(16):1179–86. https://doi.org/10.1097/BRS.0000000000004308 .

Wesselink EO, et al. Convolutional neural networks for the automatic segmentation of lumbar paraspinal muscles in people with low back pain. Sci Rep. 2022;12(1):13485. https://doi.org/10.1038/s41598-022-16710-5 .

Weber KA, et al. Deep learning convolutional neural networks for the automatic quantification of muscle fat infiltration following whiplash injury. Sci Rep. 2019;9(1):7973. https://doi.org/10.1038/s41598-019-44416-8 .

Hess M, et al. Deep learning for multi-tissue segmentation and fully automatic personalized biomechanical models from BACPAC clinical lumbar spine MRI. Pain Med. 2023;24(Suppl 1):S139–48. https://doi.org/10.1093/pm/pnac142 .

Burian E, et al. Lumbar muscle and vertebral bodies segmentation of chemical shift encoding-based water-fat MRI: the reference database MyoSegmenTUM spine. BMC Musculoskelet Disord. 2019;20(1):152. https://doi.org/10.1186/s12891-019-2528-x .

Weber KA, et al. Multi-muscle deep learning segmentation to automate the quantification of muscle fat infiltration in cervical spine conditions. Sci Rep. 2021;11(1):1. https://doi.org/10.1038/s41598-021-95972-x .

Article   CAS   Google Scholar  

Pai S A, et al. Quantitative identification and segmentation repeatability of thoracic spinal muscle morphology. JOR Spine. 2020;3(3):e1103. https://doi.org/10.1002/jsp2.1103

Sollmann N, et al. Regional variation in paraspinal muscle composition using chemical shift encoding-based water-fat MRI. Quant Imaging Med Surg. 2020;10(2):496–507. https://doi.org/10.21037/qims.2020.01.10 .

Mhuiris ÁN, Volken T, Elliott JM, Hoggarth M, Samartzis D, Crawford RJ. Reliability of quantifying the spatial distribution of fatty infiltration in lumbar paravertebral muscles using a new segmentation method for T1-weighted MRI. BMC Musculoskelet Disord. 2016;17:234. https://doi.org/10.1186/s12891-016-1090-z .

Dieckmeyer M, et al. Texture features of proton density fat fraction maps from chemical shift encoding-based MRI predict paraspinal muscle strength. Diagnostics (Basel). 2021;11(2):239. https://doi.org/10.3390/diagnostics11020239 .

Saeed MU, Dikaios N, Dastgir A, Ali G, Hamid M, Hajjej F. An Automated deep learning approach for spine segmentation and vertebrae recognition using computed tomography images. Diagnostics (Basel). 2023;13(16):2658. https://doi.org/10.3390/diagnostics13162658 .

Ferrero E, et al. Volume of spinopelvic muscles: comparison between adult spinal deformity patients and asymptomatic subjects. Spine Deform. 2021;9(6):1617–24. https://doi.org/10.1007/s43390-021-00357-9 .

Wakamatsu Y, Kamiya N, Zhou X, Hara T, Fujita H. Relationship between number of annotations and accuracy in segmentation of the erector spinae muscle using Bayesian U-Net in torso CT images. In: International Forum on Medical Imaging in Asia 2021. SPIE; 2021. p. 30–3. https://doi.org/10.1117/12.2590780 .

Kawamoto M, Kamiya N, Zhou X, Kato H, Hara T, Fujita H. Simultaneous Learning of Erector Spinae Muscles for Automatic Segmentation of Site-Specific Skeletal Muscles in Body CT Images. IEEE Access. 2024;12:15468–76. https://doi.org/10.1109/ACCESS.2023.3335948 .

Article   Google Scholar  

Kamiya N, Li J, Kume M, Fujita H, Shen D, Zheng G. Fully automatic segmentation of paraspinal muscles from 3D torso CT images via multi-scale iterative random forest classifications. Int J Comput Assist Radiol Surg. 2018;13(11):1697–706. https://doi.org/10.1007/s11548-018-1852-1 .

Download references

Acknowledgements

This research was supported by a grant of the Korea Health Technology R&D Project through the Korea, Health Industry Development Institute(KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number : HI22C0494).

No funding for this study.

Author information

Authors and affiliations.

Department of Biomedical Research Institute, Inha University Hospital, 27 Inhang-ro, Jung-gu, Incheon, Republic of Korea

Hyun-Bin Kim, Hyeon-Su Kim & Shin-June Kim

Department of Orthopaedic Surgery, School of Medicine, Inha University Hospital, 27 Inhang-ro, Jung-gu, Incheon, Republic of Korea

You can also search for this author in PubMed   Google Scholar

Contributions

Study concept and design: Jun-Il Yoo Study analysis and interpretation: Hyun-Bin Kim Study Investigation: Hyeon-Su Kim, Shin-June Kim.

Corresponding author

Correspondence to Jun-Il Yoo .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing of interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Kim, HB., Kim, HS., Kim, SJ. et al. Spine muscle auto segmentation techniques in MRI imaging: a systematic review. BMC Musculoskelet Disord 25 , 716 (2024). https://doi.org/10.1186/s12891-024-07777-4

Download citation

Received : 22 November 2023

Accepted : 14 August 2024

Published : 06 September 2024

DOI : https://doi.org/10.1186/s12891-024-07777-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Spine muscles
• Segmentation
• Magnetic resonance imaging (MRI)
• Machine learning
• Deep learning

BMC Musculoskeletal Disorders

ISSN: 1471-2474

• General enquiries: [email protected]