inclusion and exclusion criteria of literature review

Jump to navigation

Cochrane Training

Chapter 3: defining the criteria for including studies and how they will be grouped for the synthesis.

Joanne E McKenzie, Sue E Brennan, Rebecca E Ryan, Hilary J Thomson, Renea V Johnston, James Thomas

Key Points:

• The scope of a review is defined by the types of population (participants), types of interventions (and comparisons), and the types of outcomes that are of interest. The acronym PICO (population, interventions, comparators and outcomes) helps to serve as a reminder of these.
• The population, intervention and comparison components of the question, with the additional specification of types of study that will be included, form the basis of the pre-specified eligibility criteria for the review. It is rare to use outcomes as eligibility criteria: studies should be included irrespective of whether they report outcome data, but may legitimately be excluded if they do not measure outcomes of interest, or if they explicitly aim to prevent a particular outcome.
• Cochrane Reviews should include all outcomes that are likely to be meaningful and not include trivial outcomes. Critical and important outcomes should be limited in number and include adverse as well as beneficial outcomes.
• Review authors should plan at the protocol stage how the different populations, interventions, outcomes and study designs within the scope of the review will be grouped for analysis.

Cite this chapter as: McKenzie JE, Brennan SE, Ryan RE, Thomson HJ, Johnston RV, Thomas J. Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis [last updated August 2023]. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane, 2024. Available from www.training.cochrane.org/handbook .

3.1 Introduction

One of the features that distinguishes a systematic review from a narrative review is that systematic review authors should pre-specify criteria for including and excluding studies in the review (eligibility criteria, see MECIR Box 3.2.a ).

When developing the protocol, one of the first steps is to determine the elements of the review question (including the population, intervention(s), comparator(s) and outcomes, or PICO elements) and how the intervention, in the specified population, produces the expected outcomes (see Chapter 2, Section 2.5.1 and Chapter 17, Section 17.2.1 ). Eligibility criteria are based on the PICO elements of the review question plus a specification of the types of studies that have addressed these questions. The population, interventions and comparators in the review question usually translate directly into eligibility criteria for the review, though this is not always a straightforward process and requires a thoughtful approach, as this chapter shows. Outcomes usually are not part of the criteria for including studies, and a Cochrane Review would typically seek all sufficiently rigorous studies (most commonly randomized trials) of a particular comparison of interventions in a particular population of participants, irrespective of the outcomes measured or reported. It should be noted that some reviews do legitimately restrict eligibility to specific outcomes. For example, the same intervention may be studied in the same population for different purposes; or a review may specifically address the adverse effects of an intervention used for several conditions (see Chapter 19 ).

Eligibility criteria do not exist in isolation, but should be specified with the synthesis of the studies they describe in mind. This will involve making plans for how to group variants of the PICO elements for synthesis. This chapter describes the processes by which the structure of the synthesis can be mapped out at the beginning of the review, and the interplay between the review question, considerations for the analysis and their operationalization in terms of eligibility criteria. Decisions about which studies to include (and exclude), and how they will be combined in the review’s synthesis, should be documented and justified in the review protocol.

A distinction between three different stages in the review at which the PICO construct might be used is helpful for understanding the decisions that need to be made. In Chapter 2, Section 2.3 , we introduced the ideas of a review PICO (on which eligibility of studies is based), the PICO for each synthesis (defining the question that each specific synthesis aims to answer) and the PICO of the included studies (what was actually investigated in the included studies). In this chapter, we focus on the review PICO and the PICO for each synthesis as a basis for specifying which studies should be included in the review and planning its syntheses. These PICOs should relate clearly and directly to the questions or hypotheses that are posed when the review is formulated (see Chapter 2 ) and will involve specifying the population in question, and a set of comparisons between the intervention groups.

An integral part of the process of setting up the review is to specify which characteristics of the interventions (e.g. individual compounds of a drug), populations (e.g. acute and chronic conditions), outcomes (e.g. different depression measurement scales) and study designs, will be grouped together. Such decisions should be made independent of knowing which studies will be included and the methods of synthesis that will be used (e.g. meta-analysis). There may be a need to modify the comparisons and even add new ones at the review stage in light of the data that are collected. For example, important variations in the intervention may be discovered only after data are collected, or modifying the comparison may facilitate the possibility of synthesis when only one or few studies meet the comparison PICO. Planning for the latter scenario at the protocol stage may lead to less post-hoc decision making ( Chapter 2, Section 2.5.3 ) and, of course, any changes made during the conduct of the review should be recorded and documented in the final report.

3.2 Articulating the review and comparison PICO

3.2.1 defining types of participants: which people and populations.

The criteria for considering types of people included in studies in a review should be sufficiently broad to encompass the likely diversity of studies and the likely scenarios in which the interventions will be used, but sufficiently narrow to ensure that a meaningful answer can be obtained when studies are considered together; they should be specified in advance (see MECIR Box 3.2.a ). As discussed in Chapter 2, Section 2.3.1 , the degree of breadth will vary, depending on the question being asked and the analytical approach to be employed. A range of evidence may inform the choice of population characteristics to examine, including theoretical considerations, evidence from other interventions that have a similar mechanism of action, and in vitro or animal studies. Consideration should be given to whether the population characteristic is at the level of the participant (e.g. age, severity of disease) or the study (e.g. care setting, geographical location), since this has implications for grouping studies and for the method of synthesis ( Chapter 10, Section 10.11.5 ). It is often helpful to consider the types of people that are of interest in three steps.

MECIR Box 3.2.a Relevant expectations for conduct of intervention reviews

First, the diseases or conditions of interest should be defined using explicit criteria for establishing their presence (or absence). Criteria that will force the unnecessary exclusion of studies should be avoided. For example, diagnostic criteria that were developed more recently – which may be viewed as the current gold standard for diagnosing the condition of interest – will not have been used in earlier studies. Expensive or recent diagnostic tests may not be available in many countries or settings, and time-consuming tests may not be practical in routine healthcare settings.

Second, the broad population and setting of interest should be defined . This involves deciding whether a specific population group is within scope, determined by factors such as age, sex, race, educational status or the presence of a particular condition such as angina or shortness of breath. Interest may focus on a particular setting such as a community, hospital, nursing home, chronic care institution, or outpatient setting. Box 3.2.a outlines some factors to consider when developing population criteria.

Whichever criteria are used for defining the population and setting of interest, it is common to encounter studies that only partially overlap with the review’s population. For example, in a review focusing on children, a cut-point of less than 16 years might be desirable, but studies may be identified with participants aged from 12 to 18. Unless the study reports separate data from the eligible section of the population (in which case data from the eligible participants can be included in the review), review authors will need a strategy for dealing with these studies (see MECIR Box 3.2.a ). This will involve balancing concerns about reduced applicability by including participants who do not meet the eligibility criteria, against the loss of data when studies are excluded. Arbitrary rules (such as including a study if more than 80% of the participants are under 16) will not be practical if detailed information is not available from the study. A less stringent rule, such as ‘the majority of participants are under 16’ may be sufficient. Although there is a risk of review authors’ biases affecting post-hoc inclusion decisions (which is why many authors endeavour to pre-specify these rules), this may be outweighed by a common-sense strategy in which eligibility decisions keep faith with the objectives of the review rather than with arbitrary rules. Difficult decisions should be documented in the review, checked with the advisory group (if available, see Chapter 1 ), and sensitivity analyses can assess the impact of these decisions on the review’s findings (see Chapter 10, Section 10.14 and MECIR Box 3.2.b ).

Box 3.2.a Factors to consider when developing criteria for ‘Types of participants’

MECIR Box 3.2.b Relevant expectations for conduct of intervention reviews

Third, there should be consideration of whether there are population characteristics that might be expected to modify the size of the intervention effects (e.g. different severities of heart failure). Identifying subpopulations may be important for implementation of the intervention. If relevant subpopulations are identified, two courses of action are possible: limiting the scope of the review to exclude certain subpopulations; or maintaining the breadth of the review and addressing subpopulations in the analysis.

Restricting the review with respect to specific population characteristics or settings should be based on a sound rationale. It is important that Cochrane Reviews are globally relevant, so the rationale for the exclusion of studies based on population characteristics should be justified. For example, focusing a review of the effectiveness of mammographic screening on women between 40 and 50 years old may be justified based on biological plausibility, previously published systematic reviews and existing controversy. On the other hand, focusing a review on a particular subgroup of people on the basis of their age, sex or ethnicity simply because of personal interests, when there is no underlying biologic or sociological justification for doing so, should be avoided, as these reviews will be less useful to decision makers and readers of the review.

Maintaining the breadth of the review may be best when it is uncertain whether there are important differences in effects among various subgroups of people, since this allows investigation of these differences (see Chapter 10, Section 10.11.5 ). Review authors may combine the results from different subpopulations in the same synthesis, examining whether a given subdivision explains variation (heterogeneity) among the intervention effects. Alternatively, the results may be synthesized in separate comparisons representing different subpopulations. Splitting by subpopulation risks there being too few studies to yield a useful synthesis (see Table 3.2.a and Chapter 2, Section 2.3.2 ). Consideration needs to be given to the subgroup analysis method, particularly for population characteristics measured at the participant level (see Chapter 10 and Chapter 26 , Fisher et al 2017). All subgroup analyses should ideally be planned a priori and stated as a secondary objective in the protocol, and not driven by the availability of data.

In practice, it may be difficult to assign included studies to defined subpopulations because of missing information about the population characteristic, variability in how the population characteristic is measured across studies (e.g. variation in the method used to define the severity of heart failure), or because the study does not wholly fall within (or report the results separately by) the defined subpopulation. The latter issue mainly applies for participant characteristics but can also arise for settings or geographic locations where these vary within studies. Review authors should consider planning for these scenarios (see example reviews Hetrick et al 2012, Safi et al 2017; Table 3.2.b , column 3).

Table 3.2.a Examples of population attributes and characteristics

3.2.2 Defining interventions and how they will be grouped

In some reviews, predefining the intervention ( MECIR Box 3.2.c ) may be straightforward. For example, in a review of the effect of a given anticoagulant on deep vein thrombosis, the intervention can be defined precisely. A more complicated definition might be required for a multi-component intervention composed of dietary advice, training and support groups to reduce rates of obesity in a given population.

The inherent complexity present when defining an intervention often comes to light when considering how it is thought to achieve its intended effect and whether the effect is likely to differ when variants of the intervention are used. In the first example, the anticoagulant warfarin is thought to reduce blood clots by blocking an enzyme that depends on vitamin K to generate clotting factors. In the second, the behavioural intervention is thought to increase individuals’ self-efficacy in their ability to prepare healthy food. In both examples, we cannot assume that all forms of the intervention will work in the same way. When defining drug interventions, such as anticoagulants, factors such as the drug preparation, route of administration, dose, duration, and frequency should be considered. For multi-component interventions (such as interventions to reduce rates of obesity), the common or core features of the interventions must be defined, so that the review authors can clearly differentiate them from other interventions not included in the review.

MECIR Box 3.2.c Relevant expectations for conduct of intervention reviews

In general, it is useful to consider exactly what is delivered, who delivers it, how it is delivered, where it is delivered, when and how much is delivered, and whether the intervention can be adapted or tailored , and to consider this for each type of intervention included in the review (see the TIDieR checklist (Hoffmann et al 2014)). As argued in Chapter 17 , separating interventions into ‘simple’ and ‘complex’ is a false dichotomy; all interventions can be complex in some ways. The critical issue for review authors is to identify the most important factors to be considered in a specific review. Box 3.2.b outlines some factors to consider when developing broad criteria for the ‘Types of interventions’ (and comparisons).

Box 3.2.b Factors to consider when developing criteria for ‘Types of interventions’

Once interventions eligible for the review have been broadly defined, decisions should be made about how variants of the intervention will be handled in the synthesis. Differences in intervention characteristics across studies occur in all reviews. If these reflect minor differences in the form of the intervention used in practice (such as small differences in the duration or content of brief alcohol counselling interventions), then an overall synthesis can provide useful information for decision makers. Where differences in intervention characteristics are more substantial (such as delivery of brief alcohol counselling by nurses versus doctors), and are expected to have a substantial impact on the size of intervention effects, these differences should be examined in the synthesis. What constitutes an important difference requires judgement, but in general differences that alter decisions about how an intervention is implemented or whether the intervention is used or not are likely to be important. In such circumstances, review authors should consider specifying separate groups (or subgroups) to examine in their synthesis.

Clearly defined intervention groups serve two main purposes in the synthesis. First, the way in which interventions are grouped for synthesis (meta-analysis or other synthesis) is likely to influence review findings. Careful planning of intervention groups makes best use of the available data, avoids decisions that are influenced by study findings (which may introduce bias), and produces a review focused on questions relevant to decision makers. Second, the intervention groups specified in a protocol provide a standardized terminology for describing the interventions throughout the review, overcoming the varied descriptions used by study authors (e.g. where different labels are used for the same intervention, or similar labels used for different techniques) (Michie et al 2013). This standardization enables comparison and synthesis of information about intervention characteristics across studies (common characteristics and differences) and provides a consistent language for reporting that supports interpretation of review findings.

Table 3.2.b   outlines a process for planning intervention groups as a basis for/precursor to synthesis, and the decision points and considerations at each step. The table is intended to guide, rather than to be prescriptive and, although it is presented as a sequence of steps, the process is likely to be iterative, and some steps may be done concurrently or in a different sequence. The process aims to minimize data-driven approaches that can arise once review authors have knowledge of the findings of the included studies. It also includes principles for developing a flexible plan that maximizes the potential to synthesize in circumstances where there are few studies, many variants of an intervention, or where the variants are difficult to anticipate. In all stages, review authors should consider how to categorize studies whose reports contain insufficient detail.

Table 3.2.b A process for planning intervention groups for synthesis

3.2.3 Defining which comparisons will be made

When articulating the PICO for each synthesis, defining the intervention groups alone is not sufficient for complete specification of the planned syntheses. The next step is to define the comparisons that will be made between the intervention groups. Setting aside for a moment more complex analyses such as network meta-analyses, which can simultaneously compare many groups ( Chapter 11 ), standard meta-analysis ( Chapter 10 ) aims to draw conclusions about the comparative effects of two groups at a time (i.e. which of two intervention groups is more effective?). These comparisons form the basis for the syntheses that will be undertaken if data are available. Cochrane Reviews sometimes include one comparison, but most often include multiple comparisons. Three commonly identified types of comparisons include the following (Davey et al 2011).

• newer generation antidepressants versus placebo (Hetrick et al 2012); and
• vertebroplasty for osteoporotic vertebral compression fractures versus placebo (sham procedure) (Buchbinder et al 2018).
• chemotherapy or targeted therapy plus best supportive care (BSC) versus BSC for palliative treatment of esophageal and gastroesophageal-junction carcinoma (Janmaat et al 2017); and
• personalized care planning versus usual care for people with long-term conditions (Coulter et al 2015).
• early (commenced at less than two weeks of age) versus late (two weeks of age or more) parenteral zinc supplementation in term and preterm infants (Taylor et al 2017);
• high intensity versus low intensity physical activity or exercise in people with hip or knee osteoarthritis (Regnaux et al 2015);
• multimedia education versus other education for consumers about prescribed and over the counter medications (Ciciriello et al 2013).

The first two types of comparisons aim to establish the effectiveness of an intervention, while the last aims to compare the effectiveness of two interventions. However, the distinction between the placebo and control is often arbitrary, since any differences in the care provided between trials with a control arm and those with a placebo arm may be unimportant , especially where ‘usual care’ is provided to both. Therefore, placebo and control groups may be determined to be similar enough to be combined for synthesis.

In reviews including multiple intervention groups, many comparisons are possible. In some of these reviews, authors seek to synthesize evidence on the comparative effectiveness of all their included interventions, including where there may be only indirect comparison of some interventions across the included studies ( Chapter 11, Section 11.2.1 ). However, in many reviews including multiple intervention groups, a limited subset of the possible comparisons will be selected. The chosen subset of comparisons should address the most important clinical and research questions. For example, if an established intervention (or dose of an intervention) is used in practice, then the synthesis would ideally compare novel or alternative interventions to this established intervention, and not, for example, to no intervention.

3.2.3.1 Dealing with co-interventions

Planning is needed for the special case where the same supplementary intervention is delivered to both the intervention and comparator groups. A supplementary intervention is an additional intervention delivered alongside the intervention of interest, such as massage in a review examining the effects of aromatherapy (i.e. aromatherapy plus massage versus massage alone). In many cases, the supplementary intervention will be unimportant and can be ignored. In other situations, the effect of the intervention of interest may differ according to whether participants receive the supplementary therapy. For example, the effect of aromatherapy among people who receive a massage may differ from the effect of the aromatherapy given alone. This will be the case if the intervention of interest interacts with the supplementary intervention leading to larger (synergistic) or smaller (dysynergistic/antagonistic) effects than the intervention of interest alone (Squires et al 2013). While qualitative interactions are rare (where the effect of the intervention is in the opposite direction when combined with the supplementary intervention), it is possible that there will be more variation in the intervention effects (heterogeneity) when supplementary interventions are involved, and it is important to plan for this. Approaches for dealing with this in the statistical synthesis may include fitting a random-effects meta-analysis model that encompasses heterogeneity ( Chapter 10, Section 10.10.4 ), or investigating whether the intervention effect is modified by the addition of the supplementary intervention through subgroup analysis ( Chapter 10, Section 10.11.2 ).

3.2.4 Selecting, prioritizing and grouping review outcomes

3.2.4.1 selecting review outcomes.

Broad outcome domains are decided at the time of setting up the review PICO (see Chapter 2 ). Once the broad domains are agreed, further specification is required to define the domains to facilitate reporting and synthesis (i.e. the PICO for comparison) (see Chapter 2, Section 2.3 ). The process for specifying and grouping outcomes largely parallels that used for specifying intervention groups.

Reporting of outcomes should rarely determine study eligibility for a review. In particular, studies should not be excluded because they do not report results of an outcome they may have measured, or provide ‘no usable data’ ( MECIR Box 3.2.d ). This is essential to avoid bias arising from selective reporting of findings by the study authors (see Chapter 13 ). However, in some circumstances, the measurement of certain outcomes may be a study eligibility criterion. This may be the case, for example, when the review addresses the potential for an intervention to prevent a particular outcome, or when the review addresses a specific purpose of an intervention that can be used in the same population for different purposes (such as hormone replacement therapy, or aspirin).

MECIR Box 3.2.d Relevant expectations for conduct of intervention reviews

In general, systematic reviews should aim to include outcomes that are likely to be meaningful to the intended users and recipients of the reviewed evidence. This may include clinicians, patients (consumers), the general public, administrators and policy makers. Outcomes may include survival (mortality), clinical events (e.g. strokes or myocardial infarction), behavioural outcomes (e.g. changes in diet, use of services), patient-reported outcomes (e.g. symptoms, quality of life), adverse events, burdens (e.g. demands on caregivers, frequency of tests, restrictions on lifestyle) and economic outcomes (e.g. cost and resource use). It is critical that outcomes used to assess adverse effects as well as outcomes used to assess beneficial effects are among those addressed by a review (see Chapter 19 ).

Outcomes that are trivial or meaningless to decision makers should not be included in Cochrane Reviews. Inclusion of outcomes that are of little or no importance risks overwhelming and potentially misleading readers. Interim or surrogate outcomes measures, such as laboratory results or radiologic results (e.g. loss of bone mineral content as a surrogate for fractures in hormone replacement therapy), while potentially helpful in explaining effects or determining intervention integrity (see Chapter 5, Section 5.3.4.1 ), can also be misleading since they may not predict clinically important outcomes accurately. Many interventions reduce the risk for a surrogate outcome but have no effect or have harmful effects on clinically relevant outcomes, and some interventions have no effect on surrogate measures but improve clinical outcomes.

Various sources can be used to develop a list of relevant outcomes, including input from consumers and advisory groups (see Chapter 2 ), the clinical experiences of the review authors, and evidence from the literature (including qualitative research about outcomes important to those affected (see Chapter 21 )). A further driver of outcome selection is consideration of outcomes used in related reviews. Harmonization of outcomes across reviews addressing related questions facilitates broader evidence synthesis questions being addressed through the use of Overviews of reviews (see Chapter V ).

Outcomes considered to be meaningful, and therefore addressed in a review, may not have been reported in the primary studies. For example, quality of life is an important outcome, perhaps the most important outcome, for people considering whether or not to use chemotherapy for advanced cancer, even if the available studies are found to report only survival (see Chapter 18 ). A further example arises with timing of the outcome measurement, where time points determined as clinically meaningful in a review are not measured in the primary studies. Including and discussing all important outcomes in a review will highlight gaps in the primary research and encourage researchers to address these gaps in future studies.

3.2.4.2 Prioritizing review outcomes

Once a full list of relevant outcomes has been compiled for the review, authors should prioritize the outcomes and select the outcomes of most relevance to the review question. The GRADE approach to assessing the certainty of evidence (see Chapter 14 ) suggests that review authors separate outcomes into those that are ‘critical’, ‘important’ and ‘not important’ for decision making.

The critical outcomes are the essential outcomes for decision making, and are those that would form the basis of a ‘Summary of findings’ table or other summary versions of the review, such as the Abstract or Plain Language Summary. ‘Summary of findings’ tables provide key information about the amount of evidence for important comparisons and outcomes, the quality of the evidence and the magnitude of effect (see Chapter 14, Section 14.1 ). There should be no more than seven outcomes included in a ‘Summary of findings’ table, and those outcomes that will be included in summaries should be specified at the protocol stage. They should generally not include surrogate or interim outcomes. They should not be chosen on the basis of any anticipated or observed magnitude of effect, or because they are likely to have been addressed in the studies to be reviewed. Box 3.2.c summarizes the principal factors to consider when selecting and prioritizing review outcomes.

Box 3.2.c Factors to consider when selecting and prioritizing review outcomes

3.2.4.3 Defining and grouping outcomes for synthesis

Table 3.2.c outlines a process for planning for the diversity in outcome measurement that may be encountered in the studies included in a review and which can complicate, and sometimes prevent, synthesis. Research has repeatedly documented inconsistency in the outcomes measured across trials in the same clinical areas (Harrison et al 2016, Williamson et al 2017). This inconsistency occurs across all aspects of outcome measurement, including the broad domains considered, the outcomes measured, the way these outcomes are labelled and defined, and the methods and timing of measurement. For example, a review of outcome measures used in 563 studies of interventions for dementia and mild cognitive impairment found that 321 unique measurement methods were used for 1278 assessments of cognitive outcomes (Harrison et al 2016). Initiatives like COMET ( Core Outcome Measures in Effectiveness Trials ) aim to encourage standardization of outcome measurement across trials (Williamson et al 2017), but these initiatives are comparatively new and review authors will inevitably encounter diversity in outcomes across studies.

The process begins by describing the scope of each outcome domain in sufficient detail to enable outcomes from included studies to be categorized ( Table 3.2.c Step 1). This step may be straightforward in areas for which core outcome sets (or equivalent systems) exist ( Table 3.2.c Step 2). The methods and timing of outcome measurement also need to be specified, giving consideration to how differences across studies will be handled ( Table 3.2.c Steps 3 and 4). Subsequent steps consider options for dealing with studies that report multiple measures within an outcome domain ( Table 3.2.c Step 5), planning how outcome domains will be used in synthesis ( Table 3.2.c Step 6), and building in contingencies to maximize potential to synthesize ( Table 3.2.c Step 7).

Table 3.2.c A process for planning outcome groups for synthesis

3.3 Determining which study designs to include

Some study designs are more appropriate than others for answering particular questions. Authors need to consider a priori what study designs are likely to provide reliable data with which to address the objectives of their review ( MECIR Box 3.3.a ). Sections 3.3.1 and 3.3.2 cover randomized and non-randomized designs for assessing treatment effects; Chapter 17, Section 17.2.5  discusses other study designs in the context of addressing intervention complexity.

MECIR Box 3.3.a Relevant expectations for conduct of intervention reviews

3.3.1 Including randomized trials

Because Cochrane Reviews address questions about the effects of health care, they focus primarily on randomized trials and randomized trials should be included if they are feasible for the interventions of interest ( MECIR Box 3.3.b ). Randomization is the only way to prevent systematic differences between baseline characteristics of participants in different intervention groups in terms of both known and unknown (or unmeasured) confounders (see Chapter 8 ), and claims about cause and effect can be based on their findings with far more confidence than almost any other type of study. For clinical interventions, deciding who receives an intervention and who does not is influenced by many factors, including prognostic factors. Empirical evidence suggests that, on average, non-randomized studies produce effect estimates that indicate more extreme benefits of the effects of health care than randomized trials. However, the extent, and even the direction, of the bias is difficult to predict. These issues are discussed at length in Chapter 24 , which provides guidance on when it might be appropriate to include non-randomized studies in a Cochrane Review.

Practical considerations also motivate the restriction of many Cochrane Reviews to randomized trials. In recent decades there has been considerable investment internationally in establishing infrastructure to index and identify randomized trials. Cochrane has contributed to these efforts, including building up and maintaining a database of randomized trials, developing search filters to aid their identification, working with MEDLINE to improve tagging and identification of randomized trials, and using machine learning and crowdsourcing to reduce author workload in identifying randomized trials ( Chapter 4, Section 4.6.6.2 ). The same scale of organizational investment has not (yet) been matched for the identification of other types of studies. Consequently, identifying and including other types of studies may require additional efforts to identify studies and to keep the review up to date, and might increase the risk that the result of the review will be influenced by publication bias. This issue and other bias-related issues that are important to consider when defining types of studies are discussed in detail in Chapter 7 and Chapter 13 .

Specific aspects of study design and conduct should be considered when defining eligibility criteria, even if the review is restricted to randomized trials. For example, whether cluster-randomized trials ( Chapter 23, Section 23.1 ) and crossover trials ( Chapter 23, Section 23.2 ) are eligible, as well as other criteria for eligibility such as use of a placebo comparison group, evaluation of outcomes blinded to allocation sequence, or a minimum period of follow-up. There will always be a trade-off between restrictive study design criteria (which might result in the inclusion of studies that are at low risk of bias, but very few in number) and more liberal design criteria (which might result in the inclusion of more studies, but at a higher risk of bias). Furthermore, excessively broad criteria might result in the inclusion of misleading evidence. If, for example, interest focuses on whether a therapy improves survival in patients with a chronic condition, it might be inappropriate to look at studies of very short duration, except to make explicit the point that they cannot address the question of interest.

MECIR Box 3.3.b Relevant expectations for conduct of intervention reviews

3.3.2 Including non-randomized studies

The decision of whether non-randomized studies (and what type) will be included is decided alongside the formulation of the review PICO. The main drivers that may lead to the inclusion of non-randomized studies include: (i) when randomized trials are unable to address the effects of the intervention on harm and long-term outcomes or in specific populations or settings; or (ii) for interventions that cannot be randomized (e.g. policy change introduced in a single or small number of jurisdictions) (see Chapter 24 ). Cochrane, in collaboration with others, has developed guidance for review authors to support their decision about when to look for and include non-randomized studies (Schünemann et al 2013).

Non-randomized designs have the commonality of not using randomization to allocate units to comparison groups, but their different design features mean that they are variable in their susceptibility to bias. Eligibility criteria should be based on explicit study design features, and not the study labels applied by the primary researchers (e.g. case-control, cohort), which are often used inconsistently (Reeves et al 2017; see Chapter 24 ).

When non-randomized studies are included, review authors should consider how the studies will be grouped and used in the synthesis. The Cochrane Non-randomized Studies Methods Group taxonomy of design features (see Chapter 24 ) may provide a basis for grouping together studies that are expected to have similar inferential strength and for providing a consistent language for describing the study design.

Once decisions have been made about grouping study designs, planning of how these will be used in the synthesis is required. Review authors need to decide whether it is useful to synthesize results from non-randomized studies and, if so, whether results from randomized trials and non-randomized studies should be included in the same synthesis (for the purpose of examining whether study design explains heterogeneity among the intervention effects), or whether the effects should be synthesized in separate comparisons (Valentine and Thompson 2013). Decisions should be made for each of the different types of non-randomized studies under consideration. Review authors might anticipate increased heterogeneity when non-randomized studies are synthesized, and adoption of a meta-analysis model that encompasses heterogeneity is wise (Valentine and Thompson 2013) (such as a random effects model, see Chapter 10, Section 10.10.4 ). For further discussion of non-randomized studies, see Chapter 24 .

3.4 Eligibility based on publication status and language

Chapter 4 contains detailed guidance on how to identify studies from a range of sources including, but not limited to, those in peer-reviewed journals. In general, a strategy to include studies reported in all types of publication will reduce bias ( Chapter 7 ). There would need to be a compelling argument for the exclusion of studies on the basis of their publication status ( MECIR Box 3.4.a ), including unpublished studies, partially published studies, and studies published in ‘grey’ literature sources. Given the additional challenge in obtaining unpublished studies, it is possible that any unpublished studies identified in a given review may be an unrepresentative subset of all the unpublished studies in existence. However, the bias this introduces is of less concern than the bias introduced by excluding all unpublished studies, given what is known about the impact of reporting biases (see Chapter 13 on bias due to missing studies, and Chapter 4, Section 4.3 for a more detailed discussion of searching for unpublished and grey literature).

Likewise, while searching for, and analysing, studies in any language can be extremely resource-intensive, review authors should consider carefully the implications for bias (and equity, see Chapter 16 ) if they restrict eligible studies to those published in one specific language (usually English). See Chapter 4, Section 4.4.5 , for further discussion of language and other restrictions while searching.

MECIR Box 3.4.a Relevant expectations for conduct of intervention reviews

3.5 Chapter information

Authors: Joanne E McKenzie, Sue E Brennan, Rebecca E Ryan, Hilary J Thomson, Renea V Johnston, James Thomas

Acknowledgements: This chapter builds on earlier versions of the Handbook . In particular, Version 5, Chapter 5 , edited by Denise O’Connor, Sally Green and Julian Higgins.

Funding: JEM is supported by an Australian National Health and Medical Research Council (NHMRC) Career Development Fellowship (1143429). SEB and RER’s positions are supported by the NHMRC Cochrane Collaboration Funding Program. HJT is funded by the UK Medical Research Council (MC_UU_12017-13 and MC_UU_12017-15) and Scottish Government Chief Scientist Office (SPHSU13 and SPHSU15). RVJ’s position is supported by the NHMRC Cochrane Collaboration Funding Program and Cabrini Institute. JT is supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

3.6 References

Bailey JV, Murray E, Rait G, Mercer CH, Morris RW, Peacock R, Cassell J, Nazareth I. Interactive computer-based interventions for sexual health promotion. Cochrane Database of Systematic Reviews 2010; 9 : CD006483.

Bender R, Bunce C, Clarke M, Gates S, Lange S, Pace NL, Thorlund K. Attention should be given to multiplicity issues in systematic reviews. Journal of Clinical Epidemiology 2008; 61 : 857–865.

Buchbinder R, Johnston RV, Rischin KJ, Homik J, Jones CA, Golmohammadi K, Kallmes DF. Percutaneous vertebroplasty for osteoporotic vertebral compression fracture. Cochrane Database of Systematic Reviews 2018; 4 : CD006349.

Caldwell DM, Welton NJ. Approaches for synthesising complex mental health interventions in meta-analysis. Evidence-Based Mental Health 2016; 19 : 16–21.

Chamberlain C, O’Mara-Eves A, Porter J, Coleman T, Perlen S, Thomas J, McKenzie J. Psychosocial interventions for supporting women to stop smoking in pregnancy. Cochrane Database of Systematic Reviews 2017; 2 : CD001055.

Ciciriello S, Johnston RV, Osborne RH, Wicks I, deKroo T, Clerehan R, O’Neill C, Buchbinder R. Multimedia educational interventions for consumers about prescribed and over-the-counter medications. Cochrane Database of Systematic Reviews 2013; 4 : CD008416.

Cochrane Consumers & Communication Group. Outcomes of Interest to the Cochrane Consumers & Communication Group: taxonomy. http://cccrg.cochrane.org/ .

COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative. COSMIN database of systematic reviews of outcome measurement instruments. https://database.cosmin.nl/ .

Coulter A, Entwistle VA, Eccles A, Ryan S, Shepperd S, Perera R. Personalised care planning for adults with chronic or long-term health conditions. Cochrane Database of Systematic Reviews 2015; 3 : CD010523.

Davey J, Turner RM, Clarke MJ, Higgins JPT. Characteristics of meta-analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross-sectional, descriptive analysis. BMC Medical Research Methodology 2011; 11 : 160.

Desroches S, Lapointe A, Ratte S, Gravel K, Legare F, Turcotte S. Interventions to enhance adherence to dietary advice for preventing and managing chronic diseases in adults. Cochrane Database of Systematic Reviews 2013; 2 : CD008722.

Deyo RA, Dworkin SF, Amtmann D, Andersson G, Borenstein D, Carragee E, Carrino J, Chou R, Cook K, DeLitto A, Goertz C, Khalsa P, Loeser J, Mackey S, Panagis J, Rainville J, Tosteson T, Turk D, Von Korff M, Weiner DK. Report of the NIH Task Force on research standards for chronic low back pain. Journal of Pain 2014; 15 : 569–585.

Dodd S, Clarke M, Becker L, Mavergames C, Fish R, Williamson PR. A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery. Journal of Clinical Epidemiology 2018; 96 : 84–92.

Fisher DJ, Carpenter JR, Morris TP, Freeman SC, Tierney JF. Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach? BMJ 2017; 356 : j573.

Fransen M, McConnell S, Harmer AR, Van der Esch M, Simic M, Bennell KL. Exercise for osteoarthritis of the knee. Cochrane Database of Systematic Reviews 2015; 1 : CD004376.

Guise JM, Chang C, Viswanathan M, Glick S, Treadwell J, Umscheid CA. Systematic reviews of complex multicomponent health care interventions. Report No. 14-EHC003-EF . Rockville, MD: Agency for Healthcare Research and Quality; 2014.

Harrison JK, Noel-Storr AH, Demeyere N, Reynish EL, Quinn TJ. Outcomes measures in a decade of dementia and mild cognitive impairment trials. Alzheimer’s Research and Therapy 2016; 8 : 48.

Hedges LV, Tipton E, Johnson M, C. Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods 2010; 1 : 39–65.

Hetrick SE, McKenzie JE, Cox GR, Simmons MB, Merry SN. Newer generation antidepressants for depressive disorders in children and adolescents. Cochrane Database of Systematic Reviews 2012; 11 : CD004851.

Higgins JPT, López-López JA, Becker BJ, Davies SR, Dawson S, Grimshaw JM, McGuinness LA, Moore THM, Rehfuess E, Thomas J, Caldwell DM. Synthesizing quantitative evidence in systematic reviews of complex health interventions. BMJ Global Health 2019; 4 : e000858.

Hoffmann T, Glasziou P, Barbour V, Macdonald H. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014; 1687 : 1-13.

Hollands GJ, Shemilt I, Marteau TM, Jebb SA, Lewis HB, Wei Y, Higgins JPT, Ogilvie D. Portion, package or tableware size for changing selection and consumption of food, alcohol and tobacco. Cochrane Database of Systematic Reviews 2015; 9 : CD011045.

Howe TE, Shea B, Dawson LJ, Downie F, Murray A, Ross C, Harbour RT, Caldwell LM, Creed G. Exercise for preventing and treating osteoporosis in postmenopausal women. Cochrane Database of Systematic Reviews 2011; 7 : CD000333.

ICHOM. The International Consortium for Health Outcomes Measurement 2018. http://www.ichom.org/ .

IPDAS. International Patient Decision Aid Standards Collaboration (IPDAS) standards. www.ipdas.ohri.ca .

Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2012; 6 : CD000259.

Janmaat VT, Steyerberg EW, van der Gaast A, Mathijssen RH, Bruno MJ, Peppelenbosch MP, Kuipers EJ, Spaander MC. Palliative chemotherapy and targeted therapies for esophageal and gastroesophageal junction cancer. Cochrane Database of Systematic Reviews 2017; 11 : CD004063.

Kendrick D, Kumar A, Carpenter H, Zijlstra GAR, Skelton DA, Cook JR, Stevens Z, Belcher CM, Haworth D, Gawler SJ, Gage H, Masud T, Bowling A, Pearl M, Morris RW, Iliffe S, Delbaere K. Exercise for reducing fear of falling in older people living in the community. Cochrane Database of Systematic Reviews 2014; 11 : CD009848.

Kirkham JJ, Gargon E, Clarke M, Williamson PR. Can a core outcome set improve the quality of systematic reviews? A survey of the Co-ordinating Editors of Cochrane Review Groups. Trials 2013; 14 : 21.

Konstantopoulos S. Fixed effects and variance components estimation in three-level meta-analysis. Research Synthesis Methods 2011; 2 : 61–76.

Lamb SE, Becker C, Gillespie LD, Smith JL, Finnegan S, Potter R, Pfeiffer K. Reporting of complex interventions in clinical trials: development of a taxonomy to classify and describe fall-prevention interventions. Trials 2011; 12 : 125.

Lewin S, Hendry M, Chandler J, Oxman AD, Michie S, Shepperd S, Reeves BC, Tugwell P, Hannes K, Rehfuess EA, Welch V, Mckenzie JE, Burford B, Petkovic J, Anderson LM, Harris J, Noyes J. Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR). BMC Medical Research Methodology 2017; 17 : 76.

López-López JA, Page MJ, Lipsey MW, Higgins JPT. Dealing with multiplicity of effect sizes in systematic reviews and meta-analyses. Research Synthesis Methods 2018; 9 : 336–351.

Mavridis D, Salanti G. A practical introduction to multivariate meta-analysis. Statistical Methods in Medical Research 2013; 22 : 133–158.

Michie S, van Stralen M, West R. The Behaviour Change Wheel: a new method for characterising and designing behaviour change interventions. Implementation Science 2011; 6 : 42.

Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, Eccles MP, Cane J, Wood CE. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Annals of Behavioral Medicine 2013; 46 : 81–95.

Moraes VY, Lenza M, Tamaoki MJ, Faloppa F, Belloti JC. Platelet-rich therapies for musculoskeletal soft tissue injuries. Cochrane Database of Systematic Reviews 2014; 4 : CD010071.

O'Neill J, Tabish H, Welch V, Petticrew M, Pottie K, Clarke M, Evans T, Pardo Pardo J, Waters E, White H, Tugwell P. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. Journal of Clinical Epidemiology 2014; 67 : 56–64.

Pompoli A, Furukawa TA, Imai H, Tajika A, Efthimiou O, Salanti G. Psychological therapies for panic disorder with or without agoraphobia in adults: a network meta-analysis. Cochrane Database of Systematic Reviews 2016; 4 : CD011004.

Pompoli A, Furukawa TA, Efthimiou O, Imai H, Tajika A, Salanti G. Dismantling cognitive-behaviour therapy for panic disorder: a systematic review and component network meta-analysis. Psychological Medicine 2018; 48 : 1–9.

Reeves BC, Wells GA, Waddington H. Quasi-experimental study designs series-paper 5: a checklist for classifying studies evaluating the effects on health interventions – a taxonomy without labels. Journal of Clinical Epidemiology 2017; 89 : 30–42.

Regnaux J-P, Lefevre-Colau M-M, Trinquart L, Nguyen C, Boutron I, Brosseau L, Ravaud P. High-intensity versus low-intensity physical activity or exercise in people with hip or knee osteoarthritis. Cochrane Database of Systematic Reviews 2015; 10 : CD010203.

Richards SH, Anderson L, Jenkinson CE, Whalley B, Rees K, Davies P, Bennett P, Liu Z, West R, Thompson DR, Taylor RS. Psychological interventions for coronary heart disease. Cochrane Database of Systematic Reviews 2017; 4 : CD002902.

Safi S, Korang SK, Nielsen EE, Sethi NJ, Feinberg J, Gluud C, Jakobsen JC. Beta-blockers for heart failure. Cochrane Database of Systematic Reviews 2017; 12 : CD012897.

Santesso N, Carrasco-Labra A, Brignardello-Petersen R. Hip protectors for preventing hip fractures in older people. Cochrane Database of Systematic Reviews 2014; 3 : CD001255.

Shepherd E, Gomersall JC, Tieu J, Han S, Crowther CA, Middleton P. Combined diet and exercise interventions for preventing gestational diabetes mellitus. Cochrane Database of Systematic Reviews 2017; 11 : CD010443.

Squires J, Valentine J, Grimshaw J. Systematic reviews of complex interventions: framing the review question. Journal of Clinical Epidemiology 2013; 66 : 1215–1222.

Stacey D, Légaré F, Lewis K, Barry MJ, Bennett CL, Eden KB, Holmes-Rovner M, Llewellyn-Thomas H, Lyddiatt A, Thomson R, Trevena L. Decision aids for people facing health treatment or screening decisions. Cochrane Database of Systematic Reviews 2017; 4 : CD001431.

Stroke Unit Trialists Collaboration. Organised inpatient (stroke unit) care for stroke. Cochrane Database of Systematic Reviews 2013; 9 : CD000197.

Taylor AJ, Jones LJ, Osborn DA. Zinc supplementation of parenteral nutrition in newborn infants. Cochrane Database of Systematic Reviews 2017; 2 : CD012561.

Valentine JC, Thompson SG. Issues relating to confounding and meta-analysis when including non-randomized studies in systematic reviews on the effects of interventions. Research Synthesis Methods 2013; 4 : 26–35.

Vaona A, Banzi R, Kwag KH, Rigon G, Cereda D, Pecoraro V, Tramacere I, Moja L. E-learning for health professionals. Cochrane Database of Systematic Reviews 2018; 1 : CD011736.

Verheyden GSAF, Weerdesteyn V, Pickering RM, Kunkel D, Lennon S, Geurts ACH, Ashburn A. Interventions for preventing falls in people after stroke. Cochrane Database of Systematic Reviews 2013; 5 : CD008728.

Weisz JR, Kuppens S, Ng MY, Eckshtain D, Ugueto AM, Vaughn-Coaxum R, Jensen-Doss A, Hawley KM, Krumholz Marchette LS, Chu BC, Weersing VR, Fordwood SR. What five decades of research tells us about the effects of youth psychological therapy: a multilevel meta-analysis and implications for science and practice. American Psychologist 2017; 72 : 79–117.

Welch V, Petkovic J, Simeon R, Presseau J, Gagnon D, Hossain A, Pardo Pardo J, Pottie K, Rader T, Sokolovski A, Yoganathan M, Tugwell P, DesMeules M. Interactive social media interventions for health behaviour change, health outcomes, and health equity in the adult population. Cochrane Database of Systematic Reviews 2018; 2 : CD012932.

Welton NJ, Caldwell DM, Adamopoulos E, Vedhara K. Mixed treatment comparison meta-analysis of complex interventions: psychological interventions in coronary heart disease. American Journal of Epidemiology 2009; 169 : 1158–1165.

Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, Clarke M, Gargon E, Gorst S, Harman N, Kirkham JJ, McNair A, Prinsen CAC, Schmitt J, Terwee CB, Young B. The COMET Handbook: version 1.0. Trials 2017; 18 : 280.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

Evidence-Based Practice (EBP)

• The EBP Process
• Forming a Clinical Question
• Inclusion & Exclusion Criteria
• Acquiring Evidence
• Appraising the Quality of the Evidence
• Writing a Literature Review
• Finding Psychological Tests & Assessment Instruments

Selection Criteria

Inclusion and exclusion criteria are determined after formulating the research question but usually before the search is conducted (although preliminary scoping searches may need to be undertaken to determine appropriate criteria).  It may be helpful to determine the inclusion criteria and exclusion criteria for each PICO component.

Be aware that you may  introduce bias  into the final review if these are not used thoughtfully. 

Inclusion and exclusion are two sides of the same coin, so—depending on your perspective—a single database filter can be said to either include or exclude. For instance, if articles must be published within the last 3 years, that is inclusion. If articles cannot be more than 3 years old, that is exclusion. 

The most straightforward way to include or exclude results is to use database limiters (filters), usually found on the left side of the search results page.

Inclusion Criteria

Inclusion criteria are the elements of an article  that must be present  in order for it to be eligible for inclusion in a literature review. Some examples are:

• Included studies must have compared certain treatments
• Included studies must be a certain type (e.g., only Randomized Controlled Trials)
• Included studies must be located in a certain geographic area
• Included studies must have been published in the last 5 years

Exclusion Criteria

Exclusion criteria are the elements of an article that  disqualify the study from inclusion  in a literature review. Some examples are:

• Study used an observational design
• Study used a qualitative methodology
• Study was published more than 5 years ago
• Study was published in a language other than English
• << Previous: Forming a Clinical Question
• Next: Acquiring Evidence >>
• Last Updated: May 16, 2024 2:44 PM
• URL: https://libguides.umsl.edu/ebp

Systematic Reviews for Health Sciences and Medicine

• Systematic Reviews
• The research question
• Common search errors
• Search translation
• Managing results
• Inclusion and exclusion criteria
• Critical appraisal
• Updating a Review
• Resources by Review Stage

Inclusion and Exclusion Criteria

Inclusion and exclusion criteria set the boundaries for the systematic review.  They are determined after setting the research question usually before the search is conducted, however scoping searches may need to be undertaken to determine appropriate criteria.  Many different factors can be used as inclusion or exclusion criteria. Information about the inclusion and exclusion criteria is usually recorded as a paragraph or table within the methods section of the systematic review.   It may also be necessary to give the definitions, and source of the definition, used for particular concepts in the research question (e.g. adolescence, depression).  

Other inclusion/exclusion criteria can include the sample size, method of sampling or availability of a relevant comparison group in the study.  Where a single study is reported across multiple papers the findings from the papers may be merged or only the latest data may be included.

• << Previous: Managing results
• Next: Critical appraisal >>
• Last Updated: Aug 27, 2024 2:17 PM
• URL: https://unimelb.libguides.com/sysrev

• Introduction
• Preamble: Systematic Review: What it is and isn't
• Systematic Review Guidelines
• 1. Formulate a Research Question
• 2. Develop a Research Protocol
• 3. Conduct a Thorough Literature Search
• 4. Apply Inclusion and Exclusion Criteria
• 5. Perform Data Extraction/Abstraction
• 6. Conduct a Quality Appraisal of Included Studies
• 7. Complete Data Analysis and Compile Results
• 8. Interpret Results
• How to Appraise a Systematic Review
• Systematic Review Software and Tools
• Knowledge Synthesis Services This link opens in a new window

Systematic Review Overview : 4. Apply Inclusion and Exclusion Criteria

• Health Science Information Consortium
• University Health Network - New
• Systematic Review Overview

Table of Contents

• Systematic Review Software

Step 4. Apply Inclusion and Exclusion Criteria

At the beginning of large systematic reviews, researchers discuss and develop a series of inclusion and exclusion criteria to fit in with their review question and/or the brief provided by whoever is funding the project.

Systematic reviews often exclude studies if they do not conform to specific study designs, are not written in English or within a certain time frame. As a researcher, you should be cautious of any bias you might introduce into the review by adding certain inclusion or exclusion criteria. For example: limiting to studies in English may miss important studies published in other languages, leading to language bias.

All decisions to include or exclude certain studies or groups of studies should be documented in the methods section of the research proposal/protocol - this way it can be demonstrated that a systematic process has been followed.

In large systematic reviews, the inclusion/exclusion criteria are applied by at least 2 reviewers to all the studies retrieved by the literature search. A strategy to resolve any disagreements between the reviewers should be outlined in the protocol, such as bringing in a third screener.

There are two levels of the screening process. The first level of screening involves scanning the titles and abstracts of the articles; those that are clearly irrelevant can be excluded.

Full text papers are obtained for the remaining articles and the criteria are applied again for the second level of screening on the full text. Those that meet the criteria are included in the review (although sometimes if too many papers are obtained, the question and criteria are refined and the process repeated). At this stage of screening, the reason for exclusion(s) must be recorded. This process is represented by the following flow diagram ( See PRISMA Flow Diagram ).

Key Points Regarding Study Selection

• Section 1.3.2. Process for Study selection (http://www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf, actual page #35)
• Studies should be selected in an unbiased way, based on selection criteria that flow directly from the review questions, and that have been piloted to check that they can be reliably applied.
• Study selection is a staged process involving sifting through the citations located by the search, retrieving full reports of potentially relevant citations and, from their assessment, identifying those studies that fulfill the inclusion criteria.
• Parallel independent assessments should be conducted to minimize the risk of errors of judgment. If disagreements occur between reviewers, they should be resolved according to a predefined strategy using consensus and arbitration as appropriate.
• The study selection process should be documented, detailing reasons for inclusion and exclusion.

Tips to Improve Inter-Rater Reliability / Screener Selection Accuracy

While awaiting search strategy development and final citation results:

• Provide clear and explicit inclusion and exclusion criteria, with definitions and explanations where warranted.
• Conduct thorough training for all involved.
• Provide clear guidelines which should be reviewed by all prior to starting the activity.
• Provide pilot testing or beta testing of screening tools/procedures, using samples/subsets of real data (with test inter-rater reliability calculations to determine preliminary agreement or variability).
• Optional: pilot or beta test screeners in pairs: one screener with previous experience paired with a more novice screener.
• Conduct ongoing, active surveillance/auditing of activities (can see if/when going off course)
• Provide ongoing opportunities for discussion, education, and training.
• The Screening Phase for Reviews Tutorial (5 min+) This tutorial presents information on the screening process for systematic reviews or other knowledge syntheses, and contains a variety of resources including guidelines, best practices, tips, and tools for successfully preparing to complete this important research stage.
• 1. Slavin RE. Best evidence synthesis: an intelligent alternative to meta-analysis. J Clin Epidemiol. 1995 Jan;48(1):9-18.
• 2. Eysenck HJ. Meta-analysis and its problems. BMJ. 1994 Sep 24;309(6957):789-92.
• 3. Moher D, Fortin P, Jadad AR, Juni P, Klassen T, Le Lorier J, et al. Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews. Lancet. 1996 Feb 10;347(8998):363-6.
• 4. Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials. 1998 Apr;19(2):159-66.
• 5. Moher D, Pham B, Klassen T, Schultz KF, Berlin J, Jadad AR, et al. Does the language of publication of reports of randomized trials influence the estimates of intervention effectiveness reported in meta-analyses? Systematic Reviews: Evidence in Action,
• 6. PRISMA Statement. "The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram. It is an evolving document that is subject to change periodically as new evidence emerges. In fact, the PRISMA Statement is an update and expansion of the now-out dated QUORUM Statement. This website contains the current definitive version of the PRISMA Statement."
• << Previous: 3. Conduct a Thorough Literature Search
• Next: 5. Perform Data Extraction/Abstraction >>
• Last Updated: Sep 25, 2024 8:55 AM
• URL: https://guides.hsict.library.utoronto.ca/c.php?g=699108

We acknowledge this sacred land on which the University Health Network operates. For thousands of years it has been the traditional territory of the Huron-Wendat, the Haudenosaunee, and most recently, the Mississaugas of the Credit River. This territory was the subject of the Dish With One Spoon Wampum Belt Covenant, an agreement between the Haudenosaunee Confederacy and the Confederacy of the Ojibwe and allied nations to peaceably share and care for the resources around the Great Lakes. Today, the meeting place of Toronto is still the home to many Indigenous people from across Turtle Island and we are grateful to have the opportunity to work and learn on this territory

UHN Library and Information Services

Setting Inclusion and Exclusion Criteria

• First Online: 11 August 2022

Cite this chapter

• Rob Dekkers 4 ,
• Lindsey Carey 5 &
• Peter Langhorne 6  

2494 Accesses

3 Citations

After retrieving studies and sources based on the review question, see Chapter 4 , and the search strategy, see Chapter 5 , the next step is considering which sources to include and which ones to exclude.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Although the authors label the study as a ‘systematic literature review’, it actually has the characteristics of a scoping study (see Section  4.5 for the latter).

Homomorphism (Dekkers 2017 , pp. 64–5) indicates that some of the elements and relationships (structure) are not identical, but the remaining ones are sufficiently relevant for making comparisons.

Isomorphism (Dekkers 2017 , p. 64) means that elements and relationships (structure) are the same with regard to the purpose of the study. In this respect, Norbert Wiener (cited in Dekkers 2017 , p. 63) famously said that ‘The best material model of a cat is another, or preferably the same, cat.’ This means that in practice comparisons are always limited, that some degree of homomorphism is to be expected and that when making a comparison thought should be given to which elements, subjects or objects and their relationships essential to the topic the comparison is plausible.

Alper BS, Haynes RB (2016) EBHC pyramid 5.0 for accessing preappraised evidence and guidance. Evid Based Med 21(4):123–125. https://doi.org/10.1136/ebmed-2016-110447

Antony J, Escamilla JL, Caine P (2003) Lean Sigma [production and supply chain management]. Manuf Eng 82(2):40–42

Article   Google Scholar  

Atkinson P (2004) Creating and implementing lean strategies. Manag Serv 48(2):18–21, 33

Google Scholar  

Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J et al (2011) GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 64(4):401–406. https://doi.org/10.1016/j.jclinepi.2010.07.015

Bin Ali N, Usman M (2019) A critical appraisal tool for systematic literature reviews in software engineering. Inf Softw Technol 112:48–50. https://doi.org/10.1016/j.infsof.2019.04.006

Carmona C, Baxter S, Carroll C (2021) Systematic review of the methodological literature for integrating qualitative evidence syntheses into health guideline development. Res Synth Methods 12(4):491–505. https://doi.org/10.1002/jrsm.1483

Castellini G, Bruschettini M, Gianola S, Gluud C, Moja L (2018) Assessing imprecision in Cochrane systematic reviews: a comparison of GRADE and trial sequential analysis. Syst Rev 7(1):110. https://doi.org/10.1186/s13643-018-0770-1

Connor DF, Glatt SJ, Lopez ID, Jackson D, Melloni RH (2002) Psychopharmacology and aggression. I: a meta-analysis of stimulant effects on overt/covert aggression–related behaviors in ADHD. J Am Acad Child Adolesc Psychiatry 41(3):253–261. https://doi.org/10.1097/00004583-200203000-00004

Decker JS, Stannard SJ, McManus B, Wittig SMO, Sisiopiku VP, Stavrinos D (2015) The impact of billboards on driver visual behavior: a systematic literature review. Traffic Inj Prev 16:234–239. https://doi.org/10.1080/15389588.2014.936407

Dekkers R (2017) Applied systems theory, 2nd edn. Springer, Cham

Dekkers R, Kühnle H (2012) Appraising interdisciplinary contributions to theory for collaborative (manufacturing) networks: still a long way to go? J Manuf Technol Manag 23(8):1090–1128. https://doi.org/10.1108/17410381211276899

Dixon-Woods M, Sutton A, Shaw R, Miller T, Smith J, Young B et al (2007) Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. J Health Serv Res Policy 12(1):42–47. https://doi.org/10.1258/135581907779497486

Djulbegovic B, Guyatt GH (2017) Progress in evidence-based medicine: a quarter century on. The Lancet 390(10092):415–423. https://doi.org/10.1016/S0140-6736(16)31592-6

Egger M, Zellweger-Zähner T, Schneider M, Junker C, Lengeler C, Antes G (1997) Language bias in randomised controlled trials published in English and German. The Lancet 350(9074):326–329. https://doi.org/10.1016/S0140-6736(97)02419-7

Evers CW, Wu EH (2006) On generalising from single case studies: epistemological reflections. J Philos Educ 40(4):511–526. https://doi.org/10.1111/j.1467-9752.2006.00519.x

Filieri R (2013) Consumer co-creation and new product development: a case study in the food industry. Mark Intell Plan 31(1):40–53. https://doi.org/10.1108/02634501311292911

Genet N, Boerma WGW, Kringos DS, Bouman A, Francke AL, Fagerström C et al (2011) Home care in Europe: a systematic literature review. BMC Health Serv Res 11(1):207. https://doi.org/10.1186/1472-6963-11-207

Greenhalgh T (1997) How to read a paper. Getting your bearings (deciding what the paper is about). BMJ (Clin Res Ed), 315(7102):243–246. https://doi.org/10.1136/bmj.315.7102.243

Grégoire G, Derderian F, le Lorier J (1995) Selecting the language of the publications included in a meta-analysis: is there a tower of babel bias? J Clin Epidemiol 48(1):159–163. https://doi.org/10.1016/0895-4356(94)00098-B

Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P et al (2013) GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol 66(2), 151–157. https://doi.org/10.1016/j.jclinepi.2012.01.006

Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D et al (2011) GRADE guidelines 6. Rating the quality of evidence—imprecision. J Clin Epidemiol 64(12):1283–1293. https://doi.org/10.1016/j.jclinepi.2011.01.012

Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M et al (2011) GRADE guidelines: 8. Rating the quality of evidence—indirectness. J Clin Epidemiol 64(12):1303–1310. https://doi.org/10.1016/j.jclinepi.2011.04.014

Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M et al (2011) GRADE guidelines: 7. Rating the quality of evidence—inconsistency. J Clin Epidemiol 64(12):1294–1302. https://doi.org/10.1016/j.jclinepi.2011.03.017

Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J et al (2011). GRADE guidelines: 5. Rating the quality of evidence—publication bias. J Clin Epidemiol 64(12):1277–1282. https://doi.org/10.1016/j.jclinepi.2011.01.011

Guyatt GH, Oxman AD, Sultan S, Glasziou P, Akl EA, Alonso-Coello P et al (2011) GRADE guidelines: 9. Rating up the quality of evidence. J Clin Epidemiol 64(12):1311–1316. https://doi.org/10.1016/j.jclinepi.2011.06.004

Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P et al (2011) GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol 64(4):407–415. https://doi.org/10.1016/j.jclinepi.2010.07.017

Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ (2008) GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clin Res Ed) 336(7650):924–926. https://doi.org/10.1136/bmj.39489.470347.AD

Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ et al (1995) Users’ guides to the medical literature: IX. A method for grading health care recommendations. JAMA 274(22):1800–1804. https://doi.org/10.1001/jama.1995.03530220066035

Haggan M (2004) Research paper titles in literature, linguistics and science: dimensions of attraction. J Pragmat 36(2):293–317. https://doi.org/10.1016/S0378-2166(03)00090-0

Ham van I, Verhoeven AAH, Groenier KH, Groothoff JW, De Haan J (2006) Job satisfaction among general practitioners: a systematic literature review. Euro J General Pract 12(4):174–180. https://doi.org/10.1080/13814780600994376

Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Krebs Seida J, Klassen TP (2009) Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ (Clin Res Ed) 339:b4012. https://doi.org/10.1136/bmj.b4012

Hauser JR (1993) How Puritan-Bennett used the house of quality. Sloan Manag Rev 34(3):61–70

Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD et al (2011) The Cochrane collaboration’s tool for assessing risk of bias in randomised trials. BMJ (Clin Res Ed) 343:d5928. https://doi.org/10.1136/bmj.d5928

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (2019) Cochrane handbook for systematic reviews of interventions, 2nd ed. Wiley, Chichester

Higgins JPT, Thompson SG, Deeks JJ, Altman DG (2003) Measuring inconsistency in meta-analyses. BMJ (Clin Res Ed) 327(7414):557–560. https://doi.org/10.1136/bmj.327.7414.557

Holweg M (2007) The genealogy of lean production. J Oper Manag 25(2):420–437. https://doi.org/10.1016/j.jom.2006.04.001

Hooijmans CR, Rovers MM, de Vries RBM, Leenaars M, Ritskes-Hoitinga M, Langendam MW (2014) SYRCLE’s risk of bias tool for animal studies. BMC Med Res Methodol 14(1):43. https://doi.org/10.1186/1471-2288-14-43

Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJM, Gavaghan DJ, McQuay HJ (1996) Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 17(1):1–12. https://doi.org/10.1016/0197-2456(95)00134-4

Katikireddi SV, Egan M, Petticrew M (2015) How do systematic reviews incorporate risk of bias assessments into the synthesis of evidence? A methodological study. J Epidemiol Community Health 69(2):189–195. https://doi.org/10.1136/jech-2014-204711

Koukou MI (2020) End-user involvement in new product development: a comparative study between open innovation, participatory design and the ‘instrumental approach’. Doctoral Thesis, University of Glasgow, Glasgow

Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gülmezoglu M et al (2015) Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLOS Med 12(10):e1001895. https://doi.org/10.1371/journal.pmed.1001895

Majid U, Vanstone M (2018) Appraising qualitative research for evidence syntheses: a compendium of quality appraisal tools. Qual Health Res 28(13):2115–2131. https://doi.org/10.1177/1049732318785358

Margulis AV, Pladevall M, Riera-Guardia N, Varas-Lorenzo C, Hazell L, Berkman ND et al (2014) Quality assessment of observational studies in a drug-safety systematic review, comparison of two tools: the Newcastle-Ottawa scale and the RTI item bank. Clin Epidemiol 6:359–368. https://doi.org/10.2147/CLEP.S66677

Meline T (2006) Selecting studies for systematic review: inclusion and exclusion criteria. Contemp Issues Commun Sci Disorders 33:21–27. https://doi.org/10.1044/cicsd_33_S_21

Moher D, Pham KTP, Schulz KF, Berlin JA, Jadad AR, Liberati A (2000) What contributions do languages other than English make on the results of meta-analyses? J Clin Epidemiol 53(9):964–972. https://doi.org/10.1016/S0895-4356(00)00188-8

Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL et al (2010a) The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 63(7):737–745. https://doi.org/10.1016/j.jclinepi.2010.02.006

Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL et al (2010b) The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 19(4):539–549. https://doi.org/10.1007/s11136-010-9606-8

Morrison A, Polisena J, Husereau D, Moulton K, Clark M, Fiander M et al (2012) The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int J Technol Assess Health Care 28(2):138–144. https://doi.org/10.1017/S0266462312000086

Muccini H, Sharaf M, Weyns D (2016) Self-adaptation for cyber-physical systems: a systematic literature review. Paper presented at the 11th international symposium on software engineering for adaptive and self-managing systems, Austin, TX. https://doi.org/10.1145/2897053.2897069

Murad MH, Asi N, Alsawas M, Alahdab F (2016) New evidence pyramid. Evid Based Med 21(4):125–127. https://doi.org/10.1136/ebmed-2016-110401

Postman GJ, Kateman G (1992) The quality of analytical information contained within abstracts and paper on new analytical methods. Anal Chim Acta 265(1):133–155. https://doi.org/10.1016/0003-2670(92)85164-2

Purssell E (2020) Can the critical appraisal skills programme check-lists be used alongside grading of recommendations assessment, development and evaluation to improve transparency and decision-making? J Adv Nurs 76(4):1082–1089. https://doi.org/10.1111/jan.14303

Richter RR, Sebelski CA, Austin TM (2016) The quality of reporting of abstracts in physical therapy literature is suboptimal: cross-sectional, bibliographic analysis. Am J Phys Med Rehabil 95(9):673–684. https://doi.org/10.1097/phm.0000000000000467

Rosen AB, Greenberg D, Stone PW, Olchanski NV, Neumann PJ (2005) Quality of abstracts of papers reporting original cost-effectiveness analyses. Med Decis Making 25(4):424–428. https://doi.org/10.1177/0272989x05278932

Runeson P, Höst M (2008) Guidelines for conducting and reporting case study research in software engineering. Empir Softw Eng 14(2):131–164. https://doi.org/10.1007/s10664-008-9102-8

Salgado EG, Dekkers R (2018) Lean product development: nothing new under the sun? Int J Manag Rev 20(4):903–933. https://doi.org/10.1111/ijmr.12169

Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273(5):408–412. https://doi.org/10.1001/jama.1995.03520290060030

Schumpeter J (1911) Theorie der wirtschaftlichen Entwicklung. von Duncker & Humblot, Leipzig

Schumpeter JA (1934) The theory of economic development: an inquiry into profits, capital, credit, interest, and the business cycle. Harvard University Press, Cambridge, MA

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C et al (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7(1):10. https://doi.org/10.1186/1471-2288-7-10

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J et al (2017) AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Clin Res Ed) 358:j4008. https://doi.org/10.1136/bmj.j4008

Spencer L, Ritchie J, Lewis J, Dillon L (2003) Quality in qualitative evaluation: a framework for assessing research evidence. Cabinet Office, London

Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M et al (2016) ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ (Clin Res EdÜ) 355:i4919. https://doi.org/10.1136/bmj.i4919

The Joanna Briggs Institute (2017) Critical appraisal checklist 6 for qualitative research. Adelaide

Thornton A, Lee P (2000) Publication bias in meta-analysis: its causes and consequences. J Clin Epidemiol 53(2):207–216. https://doi.org/10.1016/S0895-4356(99)00161-4

Tong A, Sainsbury P, Craig J (2007) Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 19(6):349–357. https://doi.org/10.1093/intqhc/mzm042

Viswanathan M, Berkman ND (2012) Development of the RTI item bank on risk of bias and precision of observational studies. J Clin Epidemiol 65(2):163–178. https://doi.org/10.1016/j.jclinepi.2011.05.008

Walsh M, Srinathan SK, McAuley DF, Mrkobrada M, Levine O, Ribic C et al (2014) The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index. J Clin Epidemiol 67(6):622–628. https://doi.org/10.1016/j.jclinepi.2013.10.019

Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, Tugwell P (2011) The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. The Ottawa Hospital, Ottawa. Accessed from http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp

Womack JP, Jones DT (1996) Lean thinking. Simon & Schuster, New York

Womack JP, Jones DT, Roos D (1991) The machine that changed the world: the story of lean production. Free Press, New York

Yitzhaki M (1997) Variation in informativity of titles of research papers in selected humanities journals: a comparative study. Scientometrics 38(2):219–229. https://doi.org/10.1007/BF02457410

Zahra SA, Newey LR (2009) Maximizing the impact of organization science: theory-building at the intersection of disciplines and/or fields. J Manag Stud 46(6):1059–1075. https://doi.org/10.1111/j.1467-6486.2009.00848

Download references

Author information

Authors and affiliations.

University of Glasgow, Glasgow, UK

Rob Dekkers

Glasgow Caledonian University, Glasgow, UK

Lindsey Carey

Peter Langhorne

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Rob Dekkers .

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Dekkers, R., Carey, L., Langhorne, P. (2022). Setting Inclusion and Exclusion Criteria. In: Making Literature Reviews Work: A Multidisciplinary Guide to Systematic Approaches. Springer, Cham. https://doi.org/10.1007/978-3-030-90025-0_6

Download citation

DOI : https://doi.org/10.1007/978-3-030-90025-0_6

Published : 11 August 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-90024-3

Online ISBN : 978-3-030-90025-0

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Reserve a study room
• Library Account
• Undergraduate Students
• Graduate Students
• Faculty & Staff

How to Conduct a Literature Review (Health Sciences and Beyond)

• What is a Literature Review?
• Developing a Research Question

Selection Criteria

Inclusion criteria, exclusion criteria.

• Database Search
• Documenting Your Search
• Organize Key Findings
• Reference Management

You may want to think about criteria that will be used to select articles for your literature review based on your research question.  These are commonly known as  inclusion criteria  and  exclusion criteria .  Be aware that you may introduce bias into the final review if these are not used thoughtfully.

Inclusion criteria are the elements of an article that must be present in order for it to be eligible for inclusion in a literature review.  Some examples are:

• Included studies must have compared certain treatments
• Included studies must be experimental
• Included studies must have been published in the last 5 years

Exclusion criteria are the elements of an article that disqualify the study from inclusion in a literature review.  Some examples are:

• Study used an observational design
• Study used a qualitative methodology
• Study was published more than 5 years ago
• Study was published in a language other than English
• << Previous: Developing a Research Question
• Next: Databases >>
• Last Updated: Mar 15, 2024 12:22 PM
• URL: https://guides.library.vcu.edu/health-sciences-lit-review

• University of Texas Libraries

Literature Reviews

Determine inclusion and exclusion criteria.

• What is a literature review?
• Steps in the Literature Review Process
• Define your research question
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support
• Artificial Intelligence (AI) Tools

Once you have a clearly defined research question, make sure you are getting precisely the right search results from searching the databases by making decisions about these items:

• Would the most recent five years be appropriate?
• Is your research from a more historical perspective?
• Where has this type of research taken place?
• Will you confine your results to the United States?
• To English speaking countries?
• Will you translate works if needed?
• Is there a particular methodology, or population that you are focused on?
• Are there adjacent fields in which this type of research has been conducted that you would like to include?
• Is there a controversy or debate in your research field that you want to highlight
• Are you creating a historical overview? Is this background reading for your research?
• Is there new technology that can shed light on an old problem or an old technology that can be used in a new way?
• Last Updated: Aug 26, 2024 5:59 AM
• URL: https://guides.lib.utexas.edu/literaturereviews

Library guides

Our library guides bring together the essential resources in your subject area and connect you quickly and easily to information about Library Services

CityLibrary Search

Literature searching and finding evidence.

• Literature searching or literature review?
• Use the PICO or PEO frameworks

Establish your Inclusion and Exclusion criteria

• Find related search terms
• Subject Heading/MeSH Searching
• Select databases to search
• Structure your search
• Search techniques
• Search key databases
• Manage results in EBSCOhost and Ovid
• Analyse your search results
• Document your search results
• Training and support

These criteria help you decide which pieces of evidence (for example, which primary research studies) will/will not be included in your work. Using specific criteria will help make sure your final review is as unbiased, transparent and ethical as possible.

How to establish your Inclusion and Exclusion criteria

To establish your criteria you need to define each aspect of your question to clarify what you are focusing on, and consider if there are any variations you also wish to explore. This is where using frameworks like PICO help:

Example:   Alternatives to drugs for controlling headaches in children.

Using the PICO structure you clarify what aspects you are most interested in. Here are some examples to consider:

The aspects of the topic you decide to focus on are the  Inclusion  criteria.

The aspects you don't wish to include are the  Exclusion  criteria.

• << Previous: Use the PICO or PEO frameworks
• Next: Find related search terms >>
• Last Updated: Aug 23, 2024 3:26 PM
• URL: https://libguides.city.ac.uk/SHS-Litsearchguide

Scoping Reviews

• What is a Scoping Review?
• Best Practices
• Review Protocol

Eligibility Criteria

Inclusion criteria, exclusion criteria.

• Database Search Strategies
• Study Selection (Screening)
• Data Extraction
• Reference Management

Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale. 

Think about criteria that will be used to select articles for your literature review based on your research question.  These are commonly known as  inclusion criteria  and  exclusion criteria .  You may introduce bias into the final review if these are not used thoughtfully. 

According to the PRISMA-SCcR Checklist , item 6 , authors should "specify characteristics of the sources of evidence used as eligibility criteria (e.g., years considered, language, and publication status), and provide a rationale."

Inclusion criteria are the elements of an article that must be present in order for it to be eligible for inclusion in a literature review.  

For example, included studies must:

• have compared certain treatments
• be experimental or observational or both
• have been published in a certain timeframe (must have compelling reason)
• be certain publication type(s)
• have recruited a certain population

Exclusion criteria are the elements of an article that disqualify the study from inclusion in a literature review.  

For example, excluded studies: 

• used qualitative methodology
• used a certain study design (e.g, observational)
• are a certain publication type (e.g., systematic reviews)
• were published before a certain year (must have compelling reason)
• used animal models
• was published in a language other than English
• << Previous: Review Protocol
• Next: Database Search Strategies >>
• Last Updated: Sep 12, 2024 4:02 PM
• URL: https://musc.libguides.com/scopingreviews

• View  PDF
• Download full issue

Nurse Education Today

Contemporary issues guidelines for writing a systematic review.

• Previous article in issue
• Next article in issue

Cited by (0)

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 19, Issue 1
• Reviewing the literature
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Joanna Smith 1 ,
• Helen Noble 2
• 1 School of Healthcare, University of Leeds , Leeds , UK
• 2 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• Correspondence to Dr Joanna Smith , School of Healthcare, University of Leeds, Leeds LS2 9JT, UK; j.e.smith1{at}leeds.ac.uk

https://doi.org/10.1136/eb-2015-102252

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Implementing evidence into practice requires nurses to identify, critically appraise and synthesise research. This may require a comprehensive literature review: this article aims to outline the approaches and stages required and provides a working example of a published review.

Are there different approaches to undertaking a literature review?

What stages are required to undertake a literature review.

The rationale for the review should be established; consider why the review is important and relevant to patient care/safety or service delivery. For example, Noble et al 's 4 review sought to understand and make recommendations for practice and research in relation to dialysis refusal and withdrawal in patients with end-stage renal disease, an area of care previously poorly described. If appropriate, highlight relevant policies and theoretical perspectives that might guide the review. Once the key issues related to the topic, including the challenges encountered in clinical practice, have been identified formulate a clear question, and/or develop an aim and specific objectives. The type of review undertaken is influenced by the purpose of the review and resources available. However, the stages or methods used to undertake a review are similar across approaches and include:

Formulating clear inclusion and exclusion criteria, for example, patient groups, ages, conditions/treatments, sources of evidence/research designs;

Justifying data bases and years searched, and whether strategies including hand searching of journals, conference proceedings and research not indexed in data bases (grey literature) will be undertaken;

Developing search terms, the PICU (P: patient, problem or population; I: intervention; C: comparison; O: outcome) framework is a useful guide when developing search terms;

Developing search skills (eg, understanding Boolean Operators, in particular the use of AND/OR) and knowledge of how data bases index topics (eg, MeSH headings). Working with a librarian experienced in undertaking health searches is invaluable when developing a search.

Once studies are selected, the quality of the research/evidence requires evaluation. Using a quality appraisal tool, such as the Critical Appraisal Skills Programme (CASP) tools, 5 results in a structured approach to assessing the rigour of studies being reviewed. 3 Approaches to data synthesis for quantitative studies may include a meta-analysis (statistical analysis of data from multiple studies of similar designs that have addressed the same question), or findings can be reported descriptively. 6 Methods applicable for synthesising qualitative studies include meta-ethnography (themes and concepts from different studies are explored and brought together using approaches similar to qualitative data analysis methods), narrative summary, thematic analysis and content analysis. 7 Table 1 outlines the stages undertaken for a published review that summarised research about parents’ experiences of living with a child with a long-term condition. 8

• View inline

An example of rapid evidence assessment review

In summary, the type of literature review depends on the review purpose. For the novice reviewer undertaking a review can be a daunting and complex process; by following the stages outlined and being systematic a robust review is achievable. The importance of literature reviews should not be underestimated—they help summarise and make sense of an increasingly vast body of research promoting best evidence-based practice.

• ↵ Centre for Reviews and Dissemination . Guidance for undertaking reviews in health care . 3rd edn . York : CRD, York University , 2009 .
• ↵ Canadian Best Practices Portal. http://cbpp-pcpe.phac-aspc.gc.ca/interventions/selected-systematic-review-sites / ( accessed 7.8.2015 ).
• Bridges J , et al
• ↵ Critical Appraisal Skills Programme (CASP). http://www.casp-uk.net / ( accessed 7.8.2015 ).
• Dixon-Woods M ,
• Shaw R , et al
• Agarwal S ,
• Jones D , et al
• Cheater F ,

Twitter Follow Joanna Smith at @josmith175

Competing interests None declared.

Read the full text or download the PDF:

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Inclusion and exclusion criteria in research studies: definitions and why they matter

Affiliation.

• 1 Methods in Epidemiologic, Clinical, and Operations Research-MECOR-program, American Thoracic Society/Asociación Latinoamericana del Tórax, Montevideo, Uruguay.
• PMID: 29791550
• PMCID: PMC6044655
• DOI: 10.1590/s1806-37562018000000088

PubMed Disclaimer

Similar articles

• The generalizability of clinical and experimental tobacco research: Do exclusion criteria impact men and women differently? Spinella TC, Perry RN, Schlagintweit HE, Barrett SP. Spinella TC, et al. J Psychopharmacol. 2020 May;34(5):584-585. doi: 10.1177/0269881119900984. Epub 2020 Jan 22. J Psychopharmacol. 2020. PMID: 31965900 No abstract available.
• Exclusion criteria in treatment research on alcohol, tobacco and illicit drug use disorders: A review and critical analysis. Moberg CA, Humphreys K. Moberg CA, et al. Drug Alcohol Rev. 2017 May;36(3):378-388. doi: 10.1111/dar.12438. Epub 2016 Jun 21. Drug Alcohol Rev. 2017. PMID: 27324921 Review.
• The European Union Directive on Clinical Research: present status of implementation in EU member states' legislations with regard to the incompetent patient. Lemaire F, Bion J, Blanco J, Damas P, Druml C, Falke K, Kesecioglu J, Larsson A, Mancebo J, Matamis D, Pesenti A, Pimentel J, Ranieri M; ESICM Task Force on Legislation Affecting Clinical Research in the Critically Ill Patient. Lemaire F, et al. Intensive Care Med. 2005 Mar;31(3):476-9. doi: 10.1007/s00134-005-2574-8. Epub 2005 Feb 15. Intensive Care Med. 2005. PMID: 15711974 No abstract available.
• Impact of patient selection in various study designs: identifying potential bias in clinical results. Berbano EP, Baxi N. Berbano EP, et al. South Med J. 2012 Mar;105(3):149-55. doi: 10.1097/SMJ.0b013e31824b4690. South Med J. 2012. PMID: 22392211 Review.
• Standardized patient method in tuberculosis research. Grace GA, Devaleenal DB. Grace GA, et al. Natl Med J India. 2017 Jul-Aug;30(4):210-211. Natl Med J India. 2017. PMID: 29162755 No abstract available.
• Experiences of Nurses and Midwives With Indecorously Structured Duty Rosters at Selected Health Facilities in Ho, Volta Region of Ghana: A Qualitative Study. Dartey AF, Tackie V, Lotse CW, Lily D, Sagbo FM. Dartey AF, et al. SAGE Open Nurs. 2024 Aug 23;10:23779608241275323. doi: 10.1177/23779608241275323. eCollection 2024 Jan-Dec. SAGE Open Nurs. 2024. PMID: 39185503 Free PMC article.
• Efficacy of Benson's Relaxation Technique on Stress and Pain Among Patients Undergoing Maintenance Hemodialysis: A Systematic Review. Abu Maloh HIA, Soh KL, Chong SC, Ismail SIF, Soh KG, Abu Maloh DI, AbuRuz ME. Abu Maloh HIA, et al. SAGE Open Nurs. 2024 May 6;10:23779608241251663. doi: 10.1177/23779608241251663. eCollection 2024 Jan-Dec. SAGE Open Nurs. 2024. PMID: 38715771 Free PMC article. Review.
• ROUTE-T1D: A behavioral intervention to promote optimal continuous glucose monitor use among racially minoritized youth with type 1 diabetes: Design and development. Straton E, Bryant BL, Kang L, Wang C, Barber J, Perkins A, Gallant L, Marks B, Agarwal S, Majidi S, Monaghan M, Streisand R. Straton E, et al. Contemp Clin Trials. 2024 May;140:107493. doi: 10.1016/j.cct.2024.107493. Epub 2024 Mar 7. Contemp Clin Trials. 2024. PMID: 38460913 Clinical Trial.
• Long Non-coding RNA X-Inactive Specific Transcript Promotes Esophageal Squamous Cell Carcinoma Progression via the MicroRNA 34a/Zinc Finger E-box-Binding Homeobox 1 Pathway. Guo B, He M, Ma M, Tian Z, Jin J, Tian G. Guo B, et al. Dig Dis Sci. 2024 Apr;69(4):1169-1181. doi: 10.1007/s10620-024-08269-0. Epub 2024 Feb 16. Dig Dis Sci. 2024. PMID: 38366093 Free PMC article.
• Study protocol for the family empowerment program: a randomized waitlist-controlled trial to evaluate the effectiveness of online Community Reinforcement and Family Training (CRAFT) on the wellbeing of family members with a relative experiencing substance dependence and mental illness. Allan J, Snowdon N, Thapa S, Ahmed KY. Allan J, et al. BMC Psychiatry. 2024 Jan 10;24(1):43. doi: 10.1186/s12888-023-05487-0. BMC Psychiatry. 2024. PMID: 38200508 Free PMC article.
• Montes de Oca M.Menezes A.Wehrmeister FC.Lopez Varela MV.Casas A.Ugalde L Adherence to inhaled therapies of COPD patients from seven Latin American countries The LASSYC study. PLoS One. 2017;12(11):e0186777. doi: 10.1371/journal.pone.0186777. - DOI - PMC - PubMed
• Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB. Designing Clinical Research. PA: Lippincott Williams & Wilkins; 2007.
• Search in MeSH

Related information

• Cited in Books

LinkOut - more resources

Full text sources.

• Europe PubMed Central
• PubMed Central
• Scientific Electronic Library Online

Other Literature Sources

• scite Smart Citations
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Interv Pain Med
• v.1(3); 2022 Sep
• PMC11372884

Regional anesthesia and analgesia in sickle cell pain episodes: A scoping review

Review purpose.

Sickle cell disease (SCD) vaso-occlusive crises are the most common reason patients with SCD present for medical care in the US. The goal of this scoping review is to outline existing literature on regional anesthesia for sickle cell vaso-occlusive crises (VOC) and identify areas for future research.

We searched the Cochrane Central Register, Ovid-Medline and EMBASE, PubMed, and additional review sources to identify studies evaluating the benefit of regional anesthetic blocks for medication refractory vaso-occlusive crises in pediatric and adult patients.

Summary of findings

One-hundred and three articles were identified through the above search methodology. Following application of the exclusion criteria, the four pediatric case reports, one pediatric case series, and one adult case report that were found during the scoping review process were analyzed given the scarcity of available published research on nerve blocks for the treatment of SCD pain crises. Five of the 6 articles involved blocks for pain refractory to patient-controlled analgesia (PCA) despite dose escalation. One case report utilized a continuous femoral block in a patient with known morphine and new hydromorphone allergy presenting with right thigh pain. One case report recounts an epidural used for labor pain that eliminated concomitant vaso-occlusive leg pain during labor. All 6 authors achieved analgesia and a marked decrease or a total discontinuation in opioids following the block. In one case, the patient was noted to have a shorter length of stay. No studies other than those reports included were found

There is a severe dearth of evidence evaluating the benefit of regional anesthesia in SCD pain crises. Available case reports and the included case series demonstrate that regional nerve blocks are a potential tool to consider when treating refractory vaso-occlusive pain in patients with SCD. There is urgent need for future research on evaluating regional anesthesia for patients with SCD-related vaso-occlusive crisis pain.

1. Introduction

Sickle cell disease (SCD) occurs when the 6th position on the 11th chromosome, which normally codes for glutamic acid, is substituted by valine [ 1 ]. When only a single mutated allele is present, the affected individual is capable of producing greater than 50% normal beta globin chain and is effectively a clinically silent “carrier.” However, children that are homozygous for hemoglobin S are incapable of producing normal beta globin chains and instead produce abnormal sickle cell hemoglobin tetramers that are prone to polymerization during periods of physiological stress such as hypoxia, acidosis, infection, and dehydration [ 2 , 3 ]. These changes are responsible for the propagation of the hemolytic anemia and vaso-occlusive events that occur during sickle cell crises [ 4 ].

Prior research has demonstrated that the above changes results in 4 characteristic processes that underlie the pathologic sequalae of SCD: 1) Hemoglobin S polymerization results in sickling of blood cells; 2) Sickling results in vaso-occlusion that promotes adhesion between sickled red blood cells, platelets, neutrophils, and vascular endothelium; 3) Micro-vascular vaso-occlusion promotes ischemic reperfusion injury that promotes sterile inflammation that acts in a positive feedback loop to further promote increased vaso-occlusion; and 4) Sickling induces hemolysis, promotes endothelial dysfunction, decreases nitric oxide formation, and upregulates formation of hydroxyl free radical (OH∗) damage via the Fenton reaction [ 5 ]. Overall, it is this inflammatory vaso-occlusive cascade that accounts for the characteristic features of SCD including anemia, acute pain in ischemic extremities/digits, acute chest syndrome, renal impairment, auto-splenectomy, stroke, and priapism.

In the United States, this mutation disproportionately affects African Americans (1/13 carriers and 1/365 homozygous SCD) and Hispanic-Americans (1/16,300 homozygous SCD) [ 6 ]. In total, the estimated number of Americans with SCD is 100,000 [ 7 ]. Among affected individuals, population statistics indicate that people born with SCD are living longer (Median life expectancy: 28 years in 1979; 40–53 years in 2017) [ 8 , 9 ] presumably due to early medical interventions in childhood which reduce long term morbidity and mortality.

Evidence based treatment guidelines have been instrumental in reducing morbidity and mortality in SCD [ 10 ]. Epidemiological data for American pediatric and adult patients indicate that the average life expectancy for these patients has nearly doubled in the last 5 decades. Despite increased life expectancy over this period of time, the average adjusted (sex, age group, time period) annual death rate has remained stable between 1981 and 2017 (1.54 deaths/100,000 in 1981 to 1.91/100,000 in 2017) [ 9 ]. Despite reduced morbidity and mortality, SCD-related pain has remained challenging. Multiple vaso-occlusive crises (VOC) result in high hospitalization rates, patient costs, and strain on the healthcare system. For example, between 2000 and 2016 the Agency for Healthcare Research and Quality (AHRQ) compiled and analyzed data on SCD-related inpatient hospitalizations. In 2016, a total of 134,000 inpatient hospitalizations were related to SCD. Of these hospitalizations, over 75% involved a pain episode. During this time, the aggregate cost of inpatient SCD-related stays was over $811 ​M during an average 5-day hospitalization. Among those hospitalized for a sickle cell pain episode, 30-day readmission rates were markedly higher when compared to SCD patients not admitted for a pain episode (32.9% versus 21.0%) [ 11 ]. Furthermore, studies by Ho et al. on national hospital admission rates demonstrated a steady increase in admissions between 2004 and 2014 (106/100,000 to 137/100,000) [ 12 ].

2. Treatment and prevention strategies

The most common reason for hospitalization of patients with SCD is a pain episode secondary to vaso-occlusion mediated inflammation and local cellular damage. In general, patients who have had pain episodes before will have an at home oral pain regimen. As such, patients who present to the hospital are likely a smaller subset of patients with pain refractory to their normal home analgesic regimen. To address this refractory pain, rapid triage ideally within 15 ​min of hospital arrival, treatment of pain within 30 ​min with opioids (ideally intravenous (iv) morphine or hydromorphone), and fluid resuscitation if indicated have been advocated [ 13 ]. If after 3 or more doses of opioids (spaced apart by the opioid-specific time of peak effect) pain control is inadequate, patients are generally admitted and started on patient controlled/continuous (PCA) opioid analgesia therapy [ 13 ]. Despite being the mainstay of treatment for severe SCD pain, continuous and demand opioid medication side effects can preclude treatment success in some patients. Common side effects include sedation, respiratory depression with higher doses, constipation, nausea, and pruritis. In addition, after multiple hospital admissions for VOC, opioid tolerance develops, and the patient's pain becomes more difficult to control with conventional analgesics.

Other treatment approaches for pain control in SCD include hydration to correct underlying hypovolemia often present during pain episodes, ketamine in the case of refractory pain with hyperalgesia, nonsteroidal anti-inflammatory drugs (NSAIDS), ketorolac, and heat packs. Of note, caution should be used when Ketorolac is added to multimodal treatment regimens during VOC, as it can increase the risk of acute kidney injury (AKI) in a dose-dependent manner in an already at-risk patient population [ 14 ].

3. Nerve block and neuraxial analgesia in SCD pain episode

Neuraxial and peripheral nerve blocks (either single shot or continuous infusion via catheter) have infrequently been used in conjunction with the traditional treatment strategies to treat SCD pain episodes [ 15 ]. Given their favorable prospect for improved pain control, reduction of opioid consumption, decreased length of stay, and prevention of chronic pain sensitization, utilizing these techniques, particularly in patients with localized and medical refractory pain, could potentially lead to improved patient outcomes. Furthermore, while the most recent 2020 American Society of Hematology guidelines for acute and chronic pain management suggest regional anesthesia techniques for “localized pain that is refractory or not effectively treated with opioids alone,” peer reviewed quality research evaluating the benefits of these techniques are lacking [ 15 ]. As such, we planned the scoping review that follows to obtain existing literature on this topic with the following aim:

4. Study aim

To determine if peripheral nerve blocks reduce pain scores, opioid requirements, hospital length of stay, and readmission rates in SCD VOC we carried out a scoping review using the PRISMA protocol [ 16 ]. The review questions and objectives are appropriately studied using a scoping review with an inherently broader scope with more expansive inclusion criteria. This methodology was necessary given the relative dearth of published studies on peripheral nerve blocks for the treatment of SCD VOC.

5.1. Search strategy

A query of the Cochrane Central Register, Ovid-Medline and EMBASE, PubMed, and additional review sources was conducted. Search terms used during query were as follows: (“sickle cell” OR “sickle cell anemia” OR “SCD” OR “hemoglobin SS” OR “pain crisis” OR “pain episode” OR “vaso-occlusive crisis”) AND (“nerve block” OR “block” OR “Regional” OR “brachial plexus anesthesia” OR “cervical plexus block” OR “ganglion block” OR “intercostal nerve block” OR “lumbar plexus block” OR “paracervical block” OR “stellate ganglion block”).

5.2. Inclusion and exclusion criteria

Prior to conducting the scoping review, search investigators delineated the following inclusion criteria: 1) Articles will include results on peripheral nerve blocks or neuraxial blocks used for pain treatment during an active SCD pain episode; 2) Articles included should utilize either a single shot or continuous catheter-based technique for delivery of local anesthetics/pain adjuncts. 3) Articles included should be published in English, published in any year, and must be peer-reviewed and published. Important exclusion criteria included the following: 1) Articles solely using a block or continuous catheter technique for perioperative pain control (i.e. no active acute pain episode) were excluded; 2) Conference presentations and expert opinion were also excluded due to inferiority of evidence quality, heterogeneity of methodology, and uncertain generalizability.

Of the 103 original screened articles, 5 case reports and 1 case series met inclusion criteria for initial scoping review. ( Fig. 1 ). Despite the lack of sufficiently strong scientific level of evidence (Level 1/large randomized control trials (RCT), 2/small RCT or 3/cohort or case control study) [ 17 ], the 4 pediatric case reports, 1 pediatric case series, and 1 adult case report included for analysis in this scoping review were analyzed and presented as below.

PRISMA scoping review search yield, exclusion, and inclusion flow diagram.

6.1. Peri-pain episode opioid requirement modulation and hospital length of stay

Five out of 6 of the assessed articles discussed pre- and post-block opioid requirements ( Table 1 ). Hardy et al. describe the use of a lumbar epidural pain catheter for pre-operative pain control in a case of refractory priapism requiring surgical shunt creation. In their case report, a 7-year-old African American boy was admitted to the hospital with priapism, a low-flow veno-occlusive outflow obstruction of the penis that can result from sickled blood cells when caused during VOC, pleuritic chest pain, and dyspnea at rest. Prior to the procedure the boy required a 40mcg/kg/h morphine infusion for severe (10/10) refractory pain. Following the epidural catheter placement at the L3/4 interspace, and after confirmation with a test dose of 2 ​ml 0.25% bupivacaine (with additive 1:200,000 epinephrine), the patient had complete resolution of priapism-related penile pain and had partial resolution of priapism without the intended surgical shunt intervention. The epidural catheter anesthetic was 0.125% bupivacaine without opioid infused at a rate of 0.25 ​ml/kg/h until total resolution of priapism 16 ​h later [ 18 ]. Overall, no additional pain medications were needed for priapism-related pain from the time of epidural block until discharge 48 ​h after the block placement.

Table 1

Sickle cell disease (SCD) pain episode case reports that utilized regional anesthesia for refractory pain with salient features and findings. ∗VAS: Visual Analog Pain Scale; ∗Dex: Dexmedetomidine. ∗MME: Morphine Milligram Equivalents. ∗AVN: Avascular Necrosis.

The next case report assessed involved a 12-year-old African American female admitted with refractory right thigh pain (10/10) that her family had attempted to treat at home for the past 24 ​h with home oral ibuprofen and oxycodone [ 19 ]. In the context of initiating a 0.2 ​mg basal rate/0.2 ​mg demand with 15 ​min lockout hydromorphone PCA (given prior history of morphine intolerance with shortness of breath and rash) a pruritic rash erupted and she was switched to oral long acting oxycontin 10 ​mg every 12 ​h and oxycodone 5 ​mg every 4 ​h with minimal improvement of her right thigh pain.

MRI confirmed a right proximal femur diaphysis infarction, the pain team placed a femoral nerve block catheter with a continuous infusion of 0.2% ropivacaine (7 ​ml/h titrated down to 3 ​ml/h given concern for numbness in that thigh) following a single shot 8 ​ml 2% lidocaine bolus. No IV opioids were needed following block initiation and the catheter was turned off and removed on post-procedure day 4. Prior to discharge the patient was transitioned to oral oxycodone and transdermal fentanyl patch without issue.

The third case report by Wyatt and colleagues involved a 15-year-old male who presented for a vaso-occlusive pain episode involving both of his hips and thighs. Despite 101.5 ​mg of iv morphine over the first 24-h hospital period, his right hip pain persisted. In conjunction with the pain service, and after MRI confirmation of right femoral epiphyseal osteosclerosis without necrosis, the patient underwent a right sided pericapsular nerve group block (16 ​ml bupivacaine 0.25% with unspecified dose of dexmedetomidine) and femoral nerve block (8 ​ml bupivacaine 0.25% with dexmedetomidine). Post-block he reported no (0/10) pain, had full hip range of motion and was able to ambulate. The patient's pain level was maintained between 0 and 2/10 for 24 ​h post-block. Of note, he required no opioids between the time the block was performed to the time of block sensory extinction. Furthermore, he only required 11 ​mg iv morphine equivalents after that point and was discharged without adverse events 48 ​h after the block placement.

The fourth case report by Weber and colleagues described a 14-year-old male who had previously been treated for vaso-occlusive pain crises at that facility multiple times in the past [ 20 ]. On presentation he reported right ankle pain for the past day following skin exposure to snow. Despite 4 doses of 3 ​mg iv morphine at a community hospital his pain was refractory, and he was transferred to his regular tertiary center where he was admitted for pain control, advanced management, and observation. Pain medications included 620 ​mg iv acetaminophen every 6 ​h, ketorolac 15 ​mg iv every 6 ​h, and a hydromorphone PCA with no basal rate and 0.1 ​mg demand and a 10-min lockout (consumption of 4.7 ​mg hydromorphone in the first 24 ​h). Despite this multimodal approach, the pain in his right ankle continued to be 9/10. Following consultation and informed consent, the pain service placed a continuous popliteal sciatic nerve block via a lateral approach at the level of the right popliteal fossa (20 ​ml bolus 0.1% ropivacaine and continuous rate of 6 ​ml/h). Following the block, the patient's pain decreased from 9/10 to 3/10 and eventually 0/10 later that day. PCA use decreased significantly (down to 3.3 ​mg of hydromorphone over the next 24 ​h). Notably, by hospital day 4 he was transitioned to his home opioid medications, and by day 5 his perineural pain catheter was removed. Although it is unclear why he remained admitted for the next 3 days, he was discharged on hospital day 8, a full 24 ​h earlier than his multiple previous admissions for similar vaso-occlusive pain crises. Overall, pain control utilizing a continuous popliteal sciatic nerve block was superior to an oral/IV medications-only regimen in this patient.

The fifth case report by Finer and colleagues [ 21 ] recounted the presentation of a 22-year-old female who presented in labor with a concomitant sickle cell vaso-occlusive pain crisis in her left lower extremity. Per authors, she had had several left lower extremity pain crises involving her lower extremity joints that required hospitalizations with systemic narcotic treatment of unspecified dosages. The decision was made to proceed with epidural placement for labor analgesia and treatment of lower extremity pain. Following L4/5 interspace epidural placement with test dose of 3 ​ml of 0.5% bupivacaine with 1:200,000 epinephrine and 10 ​ml 0.25% bupivacaine the patient had complete resolution of labor pain and lower extremity pain. The patient delivered soon after block placement and given her lower extremity pain a 5mcg/ml fentanyl epidural infusion was started at 8 ​ml/h for 12 ​h following delivery and then removed. No recurrence of lower extremity pain was reported, and she was discharged thereafter, and authors confirmed through chart review that no additional analgesics were required prior to discharge. The patient “expressed satisfaction with the epidural technique over iv narcotics, primarily due to the lack of mental obtundation.”

The final included study, a case series by Karsenty and colleagues reported on 3 pediatric patients treated with regional continuous nerve block catheters for vaso-occlusive pain crises [ 22 ]. The first patient, a 16yo female, presented with left upper extremity pain resulting from vaco-occlusive-related avascular necrosis (AVN) of the humeral head. Despite up-titration of opioids and after the addition of subanesthetic ketamine infusion her pain was refractory and under minimal sedation (60 ​mg ketamine and 2 ​mg midazolam) she underwent a left supraclavicular nerve block (loading dose 40 ​mg ropivacaine with continuous perineural catheter infusion of ropivacaine at 0.1 ​mg/kg/h) with resolution of pain and total elimination of required narcotics within 24 ​h. The remaining 2 patients, a 13yo male and 11yo male, both underwent interscalene nerve blocks with continuous nerve block catheter therapy following development of upper-extremity vaso-occlusive crisis pain. Both interscalene blocks were placed following development of medication side-effects with up-titration and resultant need to discontinue or decrease medications (hallucinations with ketamine initiation in one patient and respiratory depression requiring oxygen in the other patient). Overall, all three patients had reduction in pain scores to 0/10 following block placement (post-block day 0), did not require opioids by 24 ​h post-block, and were discharged within 48 ​h of initiating continuous peripheral nerve block catheters.

6.2. Re-admission rates

None of the six included studies discussed readmission rates/events following nerve block for a vaso-occlusive pain episode.

7. Discussion

From as early as the first year of life patients with SCD experience pain directly arising from their disease pathophysiology [ 23 ]. As patients progress through childhood and adolescence painful crises often reflect an acute on chronic pain state [ 24 , 25 ]. Notably, SCD pain episodes are the most common reason people with SCD present for medical care in the US [ 26 ].

To aid providers in delivering best-practice pain management care for patients in SCD pain episodes, the American Society of Hematology published guidelines for the acute and chronic management of pain in patient with SCD [ 27 ]. Among the included recommendations, expeditious assessment and treatment of pain within 60 ​min of arrival to the emergency department, tailored escalation of opioid therapy when indicated based on patient's home regimen, a short course of nonsteroidal anti-inflammatory drugs pending no contraindications, use of subanesthetic (analgesic) ketamine infusion as an adjunct when pain in refractory to opioids alone, and use of regional anesthetic treatment modalities when pain is localized and refractory to treatment with opioids alone.

As suggested by the above studies, regional techniques for localized pain relief in patients during SCD pain episodes serve as a potential treatment modality in the case of refractory pain. The mechanism of analgesia is two-fold. First, regional nerve blocks directly block afferent nerve conduction through binding of voltage gated sodium channels in the inner pore of plasma membranes, thus inhibiting membrane depolarization and conduction [ 28 ]. Second, local anesthetic-mediated regional vasodilation, or sympathectomy-related vasodilation in the case of epidurals, reduces regional blood flow impedance and we hypothesize that the interplay of these two mechanisms are chiefly responsible for block-related analgesia.

Despite progressively larger doses of opioids, investigators from 4 of the 6 studies were unable to adequately treat their pediatric patient's pain with IV medications alone ( Table 1 ). Furthermore, the progressive escalation of opioids carries the risk of increasing tolerance, hyperalgesia, and adverse effects (respiratory depression, nausea, constipation, pruritis, etc.) that are associated with larger doses of opioids. Thus, nerve block techniques, when paired with appropriate patient selection, help circumvent suboptimal pain treatment during opioid refractory pain episodes.

The fourth case report by Vuong et al. underscores the importance of patient variability and tailoring SCD pain crisis treatment regimens in the case of this 12-year-old girl who was intolerant of both morphine and hydromorphone PCAs. In patients with true opioid allergies where alternative pharmacologic treatments are inadequate, nerve block techniques can be an invaluable tool in the armament of the pediatric pain team.

Thoughtful patient selection is important in maximizing treatment success. Patients who suffer from distant multifocal pain, or a more global pain picture, are unlikely to reap the extent of analgesia attained in patients who have pain in a single limb or a localized area. In patients who have localized pain that is perceived to be amenable to a nerve block/continuous nerve block catheter, consent must be obtained from the parents or guardians in the case of non-emancipated minors less than 18 years old following an informed consent discussion on the risks, benefits, and alternative treatment modalities available. Patient cooperation during the block procedure is also important given the fact that most blocks will be performed under local anesthesia alone.

To the author's knowledge, no large robust clinical trials on nerve block and pain control in SCD patients during VOC currently exist. Although the authors acknowledge that the current mainstay treatment for moderate to severe pain during sickle cell episode is opioid-based analgesia, included articles may suggest utility of regional/neuraxial techniques with regard to potentially improving patient pain control, reducing opioid requirements, and decreasing the length of hospital stay. As highlighted in the case reports, these modalities were effective in select cases of refractory pain where progressively larger doses of PCA opioids associated with increased risk of adverse medication effects (apnea, constipation, nausea/vomiting, pruritis) were used. Thus, more research is urgently needed in this realm, especially when considering the impact the opioid epidemic has had on the US population.

8. Bias, limitations, and areas for future study

As noted above, we undertook the current scoping review with the goal of determining whether peripheral nerve blocks or neuraxial anesthetics are effective adjuncts in treating pain during SCD pain crises. Given the lack of robust randomized control trials/cohort studies/other larger multi-center studies, the investigators needed to rely on lower quality evidence studies (case reports and case series) that are more prone to confounding, involve lower degrees of external validity, and are unable to establish a cause-and-effect relationship between treatments and outcomes [ 29 ]. As such, the potential for publication bias is certainly possible and should be considered when examining the case reports discussed.

Another limitation of the study relates to regional anesthesia as a treatment modality. Namely, not all hospitals have anesthesiologist or other providers trained in pain medicine or regional anesthesia. If there are no available providers trained, credentialed, and willing to provide the nerve block at the receiving facility, then this treatment adjunct is not an option without patient transfer to a qualifying facility. In addition, an absolute contraindication for a nerve block is lack of patient consent (or parental/guardian consent in the case of a minor). If relevant caregivers are unwilling to consent to the procedure for a nerve block after the informed consent process, then this treatment approach is not a viable adjunct for refractory pain.

Although the above limitations exist, the evidence extracted from the 6 included studies provides preliminary evidence to facilitate creation of larger controlled studies (proof of concept). Furthermore, given the formerly discussed rise in hospitalization rates for SCD acute painful episodes novel approaches aimed at cost containment and reduced length of hospital stays has never been timelier. With that said, in centers that do relatively few regional blocks for SCD pain crises, larger controlled studies with adequate statistical power would likely require research collaboration among several larger tertiary care centers to achieve adequate study size enrollment.

9. Conclusion

Neuraxial and peripheral nerve blocks are an understudied treatment approach for SCD acute pain crises. Targeted blocks based on the anatomical location of discomfort have the potential to reduce the pain experienced during hospitalization as well as emergency room work-up and management. Overall, future research will be necessary to increase the confidence in this treatment modality for acute pain episode management in SCD. With that said the current study sheds light on the potential benefit that blocks may provide beneficial for patients with localized medication-refractory SCD-related pain.

The authors have no sources of funding to declare for this manuscript.

Financial support sources

Note, the authors have no acknowledgements, conflicts of interest, or funding sources to disclose.

Declaration of competing interest

The authors declare no conflicts of interest.

Acknowledgements

Redirect Notice

Nih policy and guidelines on the inclusion of women and minorities as subjects in clinical research.

This notice updates the NIH policy on the inclusion of women and minorities as subjects in clinical research. It supercedes the 1994 Federal Register notice ( /grants/guide/notice-files/not94-100.html ) and the August 2000 notice in the NIH Guide to Grants and Contracts ( /grants/guide/notice-files/NOT-OD-00-048.html ). It incorporates the definition of clinical research as reported in the 1997 Report of the NIH Director's Panel on Clinical research. Also, this notice provides additional guidance on reporting analyses of sex/gender and racial/ethnic differences in intervention effects for NIH-defined Phase III clinical trials. The guidelines ensure that all NIH-funded clinical research will be carried out in a manner sufficient to elicit information about individuals of both sexes/genders and diverse racial and ethnic groups and, particularly in NIH-defined Phase III clinical trials, to examine differential effects on such groups. Since a primary aim of research is to provide scientific evidence leading to a change in health policy or standard of care, it is imperative to determine whether the intervention or therapy being studied affects women or men or members of minority groups and their subpopulations differently.

In June 2001, NIH adopted the definition of clinical research as: (1) Patient-oriented research. Research conducted with human subjects (or on material of human origin such as tissues, specimens and cognitive phenomena) for which an investigator (or colleague) directly interacts with human subjects. Excluded from this definition are in vitro studies that utilize human tissues that cannot be linked to a living individual. Patient-oriented research includes: (a) mechanisms of human disease, (b) therapeutic interventions, (c) clinical trials, and (d) development of new technologies; (2) Epidemiologic and behavioral studies; and (3) Outcomes research and health services research http://www.nih.gov/news/crp/97report/execsum.htm.

I. Legislative Background

The NIH Revitalization Act of 1993, PL 103-43, signed into law on June 10, 1993, directed the NIH to establish guidelines for inclusion of women and minorities in clinical research. The statute states that:

In conducting or supporting clinical research for the purposes of this title, the Director of NIH shall ... ensure that (a) women are included as subjects in each project of such research; and (b) members of minority groups are included in such research. 492B(a)(1) The statute further directed the NIH to establish guidelines to specify: (a) the circumstances under which the inclusion of women and minorities as subjects in projects of clinical research is inappropriate …; (b) the manner in which clinical trials are required to be designed and carried out; and (c) the operation of outreach programs, 492B(d)(1)

The statute defines "clinical research" to include "clinical trials" and states that:

In the case of any clinical trial in which women or members of minority groups will be included as subjects, the Director of NIH shall ensure that the trial is designed and carried out in a manner sufficient to provide for valid analysis of whether the variables being studied in the trial affect women or members of minority groups, as the case may be, differently than other subjects in the trial. 492B(c)

Specifically addressing the issue of minority groups, the statute states that:

The term "minority group" includes subpopulations of minority groups. The Director of NIH shall, through the guidelines established...define the terms "minority group" and "subpopulation" for the purposes of the preceding sentence. 492B(g)(2)

The statute speaks specifically to outreach and states that:

The Director of NIH, in consultation with the Director of the Office of Research on Women's Health and the Director of the Office of Research on Minority Health, shall conduct or support outreach programs for the recruitment of women and members of minority groups as subjects in the projects of clinical research. 492B(a)(2)

The statute includes a specific provision pertaining to the cost of clinical research and, in particular clinical trials.

(A)(i) In the case of a clinical trial, the guidelines shall provide that the costs of such inclusion in the trial is (sic) not a permissible consideration in determining whether such inclusion is inappropriate. 492B(d)(2)

(ii) In the case of other projects of clinical research, the guidelines shall provide that the costs of such inclusion in the project is (sic) not a permissible consideration in determining whether such inclusion is inappropriate unless the data regarding women or members of minority groups, respectively, that would be obtained in such project (in the event that such inclusion were required) have been or are being obtained through other means that provide data of comparable quality. 492B(d)(2)

Exceptions to the requirement for inclusion of women and minorities are stated in the statute, as follows:

The requirements established regarding women and members of minority groups shall not apply to the project of clinical research if the inclusion, as subjects in the project, of women and members of minority groups, respectively-

(1) is inappropriate with respect to the health of the subjects; (2) is inappropriate with respect to the purpose of the research; or (3) is inappropriate under such other circumstances as the Director of NIH may designate. 492B(b)

(B) In the case of a clinical trial, the guidelines may provide that such inclusion in the trial is not required if there is substantial scientific data demonstrating that there is no significant difference between-

(i) the effects that the variables to be studied in the trial have on women or members of minority groups, respectively; and

(ii) the effects that the variables have on the individuals who would serve as subjects in the trial in the event that such inclusion were not required. 492B(d)(2)

A. Inclusion of Women and Minorities as Subjects in Clinical Research

It is the policy of NIH that women and members of minority groups and their subpopulations must be included in all NIH-funded clinical research, unless a clear and compelling rationale and justification establishes to the satisfaction of the relevant Institute/Center Director that inclusion is inappropriate with respect to the health of the subjects or the purpose of the research. Exclusion under other circumstances may be made by the Director, NIH, upon the recommendation of an Institute/Center Director based on a compelling rationale and justification. Cost is not an acceptable reason for exclusion except when the study would duplicate data from other sources. Women of childbearing potential should not be routinely excluded from participation in clinical research. This policy applies to research subjects of all ages in all NIH-supported clinical research studies.

The inclusion of women and members of minority groups and their subpopulations must be addressed in developing a research design or contract proposal appropriate to the scientific objectives of the study/contract. The research plan/proposal should describe the composition of the proposed study population in terms of sex/gender and racial/ethnic group, and provide a rationale for selection of such subjects. Such a plan/proposal should contain a description of the proposed outreach programs for recruiting women and minorities as participants.

B. NIH-defined Phase III Clinical Trials: Planning, Conducting, and Reporting of Analyses for Sex/Gender and Race/Ethnicity Differences.

When an NIH-defined Phase III clinical trial is proposed, evidence must be reviewed to show whether or not clinically important sex/gender and race/ethnicity differences in the intervention effect are to be expected. This evidence may include, but is not limited to, data derived from prior animal studies, clinical observations, metabolic studies, genetic studies, pharmacology studies, and observational, natural history, epidemiology and other relevant studies.

Investigators must consider the following when planning, conducting, analyzing, and reporting an NIH-Defined Phase III clinical trial. Based on prior studies, one of the three situations below will apply: 1. Prior Studies Support the Existence of Significant Differences If the data from prior studies strongly support the existence of significant differences of clinical or public health importance in intervention effect based on sex/gender, racial/ethnic, and relevant subpopulation comparisons, the primary question(s) to be addressed by the proposed NIH-defined Phase III clinical trial and the design of that trial must specifically accommodate this. For example, if men and women are thought to respond differently to an intervention, then the Phase III clinical trial must be designed to answer two separate primary questions, one for men and the other for women, with adequate sample size for each. The Research Plan (for grant applications) or Proposal (for contract solicitations) must include a description of plans to conduct analyses to detect significant differences in intervention effect ( see DEFINITIONS - Significant Difference ) by sex/gender, racial/ethnic groups, and relevant subpopulations, if applicable. The final protocol(s) approved by the Institutional Review Board (IRB) must include these plans for analysis. The award will require that for each funded protocol, investigators must report in their annual Progress Report cumulative subject accrual and progress in conducting analyses for sex/gender and race/ethnicity differences. If final analyses of sex/gender and race/ethnicity are not available at the time of the Final Progress Report or Competing Continuation for the grant, a justification and plan ensuring completion and reporting of the analyses are required. If final analyses are required as part of the contract, these analyses must be included as part of the deliverables. These requirements will be cited in the terms and conditions of all awards for grants, cooperative agreements and contracts supporting NIH-defined Phase III clinical trials. Inclusion of the results of sex/gender, race/ethnicity and relevant subpopulations analyses is strongly encouraged in all publication submissions. If these analyses reveal no differences, a brief statement to that effect, indicating the groups and/or subgroups analyzed, will suffice. 2. Prior Studies Support No Significant Differences If the data from prior studies strongly support no significant differences of clinical or public health importance in intervention effect based on sex/gender, racial/ethnic and/or relevant subpopulation comparisons, then sex/gender and race/ethnicity will not be required as subject selection criteria. However, the inclusion and analysis of sex/gender and/or racial/ethnic subgroups is still strongly encouraged. 3. Prior Studies Neither Support nor Negate Significant Differences If the data from prior studies neither strongly support nor strongly negate the existence of significant differences of clinical or public health importance in intervention effect based on sex/gender, racial/ethnic, and relevant subpopulation comparisons, then the NIH-defined Phase III clinical trial will be required to include sufficient and appropriate entry of sex/gender and racial/ethnic participants, so that valid analysis of the intervention effects can be performed. However, the trial will not be required to provide high statistical power for these comparisons. The Research Plan (for grant applications) or Proposal (for contract solicitations) must include a description of plans to conduct valid analysis ( see DEFINITIONS - Valid Analysis ) by sex/gender, racial/ethnic groups, and relevant subpopulations, if applicable. The final protocol(s) approved by the Institutional Review Board (IRB) must include these plans for analysis. The award will require that for each funded protocol, investigators must report in their annual Progress Report cumulative subject accrual and progress in conducting analyses for sex/gender and race/ethnicity differences. If final analyses of sex/gender and race/ethnicity are not available at the time of the Final Progress Report or Competing Continuation for the grant, a justification and plan ensuring completion and reporting of the analyses are required. If final analyses are required as part of the contract, these analyses must be included as part of the deliverables. These requirements will be cited in the terms and conditions of all awards for grants, cooperative agreements and contracts supporting NIH-defined Phase III clinical trials. Inclusion of the results of sex/gender, race/ethnicity and relevant subpopulations analyses is strongly encouraged in all publication submissions. If these analyses reveal no differences, a brief statement to that effect, indicating the groups and/or subgroups analyzed, will suffice.

For all three situations, cost is not an acceptable reason for exclusion of women and minorities from clinical trials.

III. Roles and Responsibilities

While this policy applies to all applicants/offerors for NIH-supported clinical research, certain individuals and groups have special roles and responsibilities with regard to its implementation. 1. NIH Staff The NIH staff provide educational opportunities for the extramural and intramural communities concerning this policy; monitor its implementation during the development, review, award and conduct of research; and manage the NIH research portfolio to comply with the policy. 2. Principal Investigators Principal investigators should assess the theoretical and/or scientific linkages between sex/gender, race/ethnicity, and their topic of study. Following this assessment, the principal investigator and the applicant/offeror institution will address the policy in each application and proposal, providing the required information on inclusion of women and minorities and their subpopulations in clinical research projects, and any required justifications for exceptions to the policy. For foreign awards and domestic awards with a foreign component, the NIH policy on inclusion of women and minority groups in research is the same as that for research conducted in the U.S. If there is scientific rationale for examining subpopulation group differences within the foreign population, investigators should consider designing their studies to accommodate these differences. Investigators and their staff(s) are urged to develop appropriate and culturally sensitive outreach programs and activities commensurate with the goals of the study or objectives of the contract. The objective should be to actively recruit and retain the most diverse study population consistent with the purposes of the research project. Indeed, the purpose should be to establish a relationship between the investigator(s) and staff(s) and populations and community(ies) of interest such that mutual benefit is derived for participants in the study. Investigator(s) should take precautionary measures to ensure that ethical issues are considered, such that there is minimal possibility of coercion or undue influence in the incentives or rewards offered in recruiting into or retaining participants in studies. To assist investigators and potential study participants, NIH staff have prepared educational materials, including a notebook titled the, "NIH Outreach Notebook On the Inclusion of Women and Minorities in Biomedical and Behavioral Research." The notebook as well as the Frequently Asked Questions document, are located at the following URL: /grants/funding/women_min/women_min.htm 3. Institutional Review Boards (IRBs) It is the responsibility of the IRBs to address the ethical issues as outlined in Section IV(2) for Principal Investigators. As the IRBs implement the regulation for the protection of human subjects as described in Title 45 CFR Part 46, "Protection of Human Subjects", http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html they must also attend to the guidelines for the inclusion of women and minorities and their subpopulations in clinical research. They should take into account the Food and Drug Administration's "Guidelines for the Study and Evaluation of Gender Differences in the Clinical Evaluation of Drugs," Vol. 58 Federal Register 39406 http://www.fda.gov/cder/guidance/old036fn.pdf. 4. Peer Review Groups In conducting peer review for scientific and technical merit, appropriately constituted initial review groups (including study sections), technical evaluation groups, and intramural review panels are instructed, as follows:

• to evaluate the proposed plan for the inclusion of minorities and both genders for appropriate representation or to evaluate the proposed justification when representation is limited or absent,
• to evaluate the proposed exclusion of minorities and women on the basis that a requirement for inclusion is inappropriate with respect to the health of the subjects,
• to evaluate the proposed exclusion of minorities and women on the basis that a requirement for inclusion is inappropriate with respect to the purpose of the research,
• to determine whether the design of clinical trials is adequate to measure differences when warranted,
• to evaluate the plans for valid analysis for NIH-defined Phase III clinical trials,
• to evaluate the plans for recruitment/outreach for study participants, and
• to include these criteria as part of the scientific assessment and evaluation.

The review instructions for grants are available on line at the following URL: /grants/peer/hs_review_inst.pdf For contracts, the contracting officer will provide instructions for contract reviewers. Further information on instructions for contracts may be obtained at the following URL: http://oa.od.nih.gov/oamp/index.html. Or contact: National Institutes of Health Division of Acquisition Policy and Evaluation Office of Acquisition Management and Policy 6100 Executive Boulevard, Room 6C01 Phone: 301-496-6014 Fax: 301- 402-1199 5. NIH Advisory Councils In addition to other responsibilities for review of projects where the peer review groups have raised questions about the appropriate inclusion of women and minorities, the Advisory Council/Board of each Institute/Center shall prepare biennial reports, for inclusion in the overall NIH Director's biennial report, describing the manner in which the Institute/Center has complied with the provisions of the statute. 6. Institute/Center Directors Institute/Center Directors and their staff shall ensure compliance with the policy. 7. NIH Director

The NIH Director may approve, on a case-by-case basis, the exclusion of projects, as recommended by the Institute/Center Director, that may be inappropriate to include within the requirements of these guidelines on the basis of circumstances other than the health of the subjects, the purpose of the research, or costs.

IV. Definitions

Throughout the section of the statute pertaining to the inclusion of women and minorities, terms are used which require definition for the purpose of implementing these guidelines. These terms, drawn directly from the statute, are defined below. A. Clinical Research Clinical research is defined as: (1) Patient-oriented research. Research conducted with human subjects (or on material of human origin such as tissues, specimens and cognitive phenomena) for which an investigator (or colleague) directly interacts with human subjects. Excluded from this definition are in vitro studies that utilize human tissues that cannot be linked to a living individual. Patient-oriented research includes: (a) mechanisms of human disease, (b) therapeutic interventions, (c) clinical trials, and (d) development of new technologies, (2) Epidemiologic and behavioral studies, (3) Outcomes research and health services research. http://www.nih.gov/news/crp/97report/execsum.htm B. NIH-defined Clinical Trial For the purpose of these guidelines, an NIH-defined "clinical trial" is a broadly based prospective Phase III clinical investigation, usually involving several hundred or more human subjects, for the purpose of evaluating an experimental intervention in comparison with a standard or control intervention or comparing two or more existing treatments. Often the aim of such investigation is to provide evidence leading to a scientific basis for consideration of a change in health policy or standard of care. The definition includes pharmacologic, non-pharmacologic, and behavioral interventions given for disease prevention, prophylaxis, diagnosis, or therapy. Community trials and other population-based intervention trials are also included. C. Valid Analysis   The term "valid analysis" means an unbiased assessment. Such an assessment will, on average, yield the correct estimate of the difference in outcomes between two groups of subjects. Valid analysis can and should be conducted for both small and large studies. A valid analysis does not need to have a high statistical power for detecting a stated effect. The principal requirements for ensuring a valid analysis of the question of interest are:

• allocation of study participants of both sexes/genders (males and females) and different racial/ethnic groups to the intervention and control groups by an unbiased process such as randomization,
• unbiased evaluation of the outcome(s) of study participants, and
• use of unbiased statistical analyses and proper methods of inference to estimate and compare the intervention effects among the sex/gender and racial/ethnic groups.

D. Significant Difference   For purposes of this policy, a "significant difference" is a difference that is of clinical or public health importance, based on substantial scientific data. This definition differs from the commonly used "statistically significant difference," which refers to the event that, for a given set of data, the statistical test for a difference between the effects in two groups achieves statistical significance. Statistical significance depends upon the amount of information in the data set. With a very large amount of information, one could find a statistically significant, but clinically small difference that is of very little clinical importance. Conversely, with less information one could find a large difference of potential importance that is not statistically significant. E. Racial and Ethnic Categories   1. Minority Groups A minority group is a readily identifiable subset of the U.S. population that is distinguished by racial, ethnic, and/or cultural heritage. The Office of Management and Budget (OMB) Directive No. 15 http://www.whitehouse.gov/omb/fedreg/ombdir15.html defines minimum standards for maintaining, collecting and presenting data on race and ethnicity for all Federal reporting. NIH is required to use these definitions to allow comparisons to other federal databases, especially the census and national health databases. The categories in this classification are social-political constructs and should not be interpreted as anthropological in nature. When an investigator is planning data collection on race and ethnicity, these categories shall be used. The collection of greater detail is encouraged. However, more detailed items should be designed in a way that they can be aggregated into these required categories. Using respondent self-report or self-identification to collect an individual's data on ethnicity and race, investigators should use two separate questions with ethnicity information collected first followed by the option to select more than one racial designation. Respondents shall be offered the opportunity to select more than one racial designation. When data are collected separately, provision shall be made to report the number of respondents in each racial category who are Hispanic or Latino.  

The following definitions apply for ethnic categories.

Hispanic or Latino - a person of Cuban, Mexican, Puerto Rican, South or Central American, or other Spanish culture or origin, regardless of race. The term "Spanish origin" can also be used in addition to "Hispanic or Latino." Not Hispanic or Latino The following definitions apply for racial categories. American Indian or Alaska Native - a person having origins in any of the original peoples of North, Central, or South America, and who maintains tribal affiliations or community attachment. Asian - a person having origins in any of the original peoples of the Far East, Southeast Asia, or the Indian subcontinent including, for example, Cambodia, China, India, Japan, Korea, Malaysia, Pakistan, the Philippine Islands, Thailand, and Vietnam. (Note: Individuals from the Philippine Islands have been recorded as Pacific Islanders in previous data collection strategies.) Black or African American - a person having origins in any of the black racial groups of Africa. Terms such as "Haitian" or "Negro" can be used in addition to "Black or African American." Native Hawaiian or Other Pacific Islander - a person having origins in any of the original peoples of Hawaii, Guam, Samoa, or other Pacific Islands. 2. Majority Group White - a person having origins in any of the original peoples of Europe, the Middle East, or North Africa. NIH recognizes the diversity of the U.S. population and that changing demographics are reflected in the changing racial and ethnic composition of the population. The terms "minority groups" and "minority subpopulations" are meant to be inclusive, rather than exclusive, of differing racial and ethnic categories. 3. Subpopulations Each racial and ethnic group contains subpopulations that are delimited by geographic origins, national origins and/or cultural differences. It is recognized that there are different ways of defining and reporting racial and ethnic subpopulation data. The subpopulation to which an individual is assigned depends on self-reporting of specific origins and/or cultural heritage. Attention to subpopulations also applies to individuals who self identify with more than one race or ethnicity. Researchers should be cognizant of the possibility that these racial/ethnic combinations may have biomedical, behavioral, and/or social-cultural implications related to the scientific question under study. F. Outreach Strategies   These are outreach efforts by investigators and their staff(s) to appropriately recruit and retain populations of interest into research studies. Such efforts should represent a thoughtful and culturally sensitive plan of outreach and generally include involvement of other individuals and organizations relevant to the populations and communities of interest, e.g., family, religious organizations, community leaders and informal gatekeepers, and public and private institutions and organizations. The objective is to establish appropriate lines of communication and cooperation to build mutual trust and cooperation such that both the study and the participants benefit from such collaboration.  

V. NIH Contacts for More Information

The following senior extramural staff from the NIH Institutes and Centers may be contacted for further information about the policy and relevant Institute/Center programs: Dr. Paulette Gray National Cancer Institute 6116 Executive Boulevard, Suite 8001 Bethesda, MD 20892-8327 Telephone: (301) 496-5147 Email: [email protected] Dr. Lore Anne McNicol National Eye Institute Executive Plaza South 6120 Executive Boulevard, Room 350 Rockville, MD 20892 Telephone: (301) 496-5301 Email: [email protected] Ms. Sharry Palagi National Heart, Lung and Blood Institute Building 31 31 Center Drive, Room 5A-07 Bethesda, MD 20892 Telephone: (301) 402-3424 Email: [email protected] Dr. Miriam Kelty National Institute on Aging Gateway Building 7201 Wisconsin Avenue, Room 2C218 Bethesda, MD 20892 Telephone: (301) 496-9322 Email: [email protected] Dr. Eleanor Hanna National Institute on Alcohol Abuse and Alcoholism Willco Building 6000 Executive Boulevard, Suite 514 Rockville, MD 20892 Telephone: (301) 594-6231 Email: [email protected] Dr. John McGowan National Institute of Allergy and Infectious Diseases 6700 B Rockledge 6700 Rockledge Drive Bethesda, MD 20817 Telephone: (301) 496-7291 Email: [email protected] Dr. Julia Freeman National Institute of Arthritis and Musculoskeletal and Skin Diseases Natcher Building Building 45, Room 5AS19F Bethesda, MD 20892 Telephone: (301) 594-4543 Email: [email protected] Dr. Susan Streufert National Institute of Child Health and Human Development 6100 Executive Boulevard Building 61EB, Room 4A05 Bethesda, MD 20892 Telephone: (301) 435-6856 Email: [email protected] Dr. Julie Gulya National Institute on Deafness and Other Communication Disorders Executive Plaza South 6120 Executive Boulevard, Room 400D-7 Rockville, MD 20892 Telephone: (301) 435-4085 Email: [email protected] Dr. Norman S. Braveman National Institute on Dental and Craniofacial Research Natcher Building Building 45, Room 4AN24C Bethesda, MD 20892 Telephone: (301) 594-2089 Email: [email protected] Dr. Robert Hammond National Institute of Diabetes and Digestive and Kidney Diseases 2 Democracy Boulevard, Room 715 Bethesda, MD 20892 Telephone: (301) 594-8834 Email: [email protected] Dr. Teresa Levitin National Institute on Drug Abuse Neuroscience Building 6001 Executive Boulevard, Room 3158 Bethesda, MD 20852 Telephone: (301) 443-2755 Email: [email protected] Dr. Anne P. Sassaman National Institute of Environmental Health Sciences P.O. Box 12233, MD EC-30 Research Triangle Park, NC 27709 Telephone: (919) 541-7723 Email: [email protected] Mr. Gary Marlowe National Institute of General Medical Sciences Natcher Building Building 45, Room 2AN12L Bethesda, MD 20892 Telephone: (301) 594-5143 Email: [email protected] Dr. Richard Nakamura National Institute of Mental Health Neuroscience Building 6001 Executive Boulevard, Room 8235 Bethesda, MD 20852 Telephone: (301) 443-3675 Email: [email protected] Ms. Lynn Morin National Institute of Neurological Disorders and Stroke Neuroscience Building 6001 Executive Boulevard, Room 2152 Bethesda, MD 20892 Telephone: (301) 496-3102 Email: [email protected] Dr. Mark Guyer National Human Genome Research Institute Building 31 31 Center Drive, Room B2B07 Bethesda, MD 20892 Telephone: (301) 496-7531 Email: [email protected] Dr. Carole Hudgings National Institute of Nursing Research Natcher Building 45 Center Drive, Room 3AN-12 Bethesda, MD 20892 Telephone: (301) 594-5976 Email: [email protected] Dr. Christine Goertz National Center for Complementary and Alternative Medicine Building 31 31 Center Drive, Room 5B-58 Telephone: (301) 402-1030 Email: [email protected] Dr. Geoffrey Cheung National Center for Research Resources Rockledge Centre I 6705 Rockledge Dr, Rm 6118 Bethesda, MD 20817 Telephone: (301) 435-0768 Email: [email protected] Dr. Kenneth Bridbord Fogarty International Center Building 31 31 Center Drive, Room B2C39 Bethesda, MD 20892 Telephone: (301) 496-2516 Email: [email protected] Dr. Joan T. Harmon National Institute of Biomedical Imaging and Bioengineering Room 697 6707 Democracy Boulevard Bethesda, MD 20892 Telephone: (301) 594-8813 Email: [email protected]

Dr. Eric Bailey National Center for Minority Health and Health Disparities 2 Democracy Boulevard, Suite 800 Bethesda, MD 20817 Telephone: (301) 402-1366 Email: [email protected]

Rumination Across Depression, Anxiety, And Eating Disorders: A Meta-Analytic Review

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

Learn about our Editorial Process

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Rumination is a repetitive thought process focused on symptoms, causes, and consequences of distress. It’s a common feature in disorders like depression, anxiety, and eating disorders, potentially exacerbating symptoms and prolonging negative mood states. Studying rumination is crucial because it may be a transdiagnostic factor underlying multiple psychological disorders. Understanding its role can inform more effective assessment, diagnosis, and treatment approaches, potentially leading to improved outcomes across various mental health conditions.

• The primary methods of this meta-analysis included comparing rumination levels between patients with depression, anxiety, and eating disorders and healthy controls, as well as examining correlations between psychopathology severity and rumination.
• Factors like total rumination, brooding, and reflection were found to be significantly higher in patients with major depressive disorder (MDD), generalized anxiety disorder ( GAD ), social anxiety disorder ( SAD ), and anorexia nervosa ( AN ) compared to healthy controls.
• Significant medium to large correlations were found between psychopathology severity and rumination for MDD, GAD, SAD, and AN.
• This research has certain limitations such as the small number of studies available for some diagnostic groups, potential publication bias, and reliance on self-report measures of rumination.
• Rumination appears to be a transdiagnostic factor across depression, anxiety, and eating disorders, with implications for assessment, diagnosis, and treatment approaches.

This meta-analysis aimed to examine rumination across depression, anxiety, and eating disorders in adults. The rationale for conducting this study was based on several key factors:

• Previous research has identified rumination as a potential transdiagnostic process that may underlie multiple psychological disorders (McLaughlin & Nolen-Hoeksema, 2011; Nolen-Hoeksema & Watkins, 2011). However, most studies have focused on rumination in specific disorders rather than comparing across diagnostic categories.
• While past meta-analyses have investigated rumination in depression and anxiety disorders (Aldao et al., 2010; Olatunji et al., 2013), or in eating disorders alone (Smith et al., 2018), there was a need for an updated and comprehensive review examining rumination across all three diagnostic categories.
• Previous reviews often combined different anxiety disorders into a single group, potentially obscuring differences between specific diagnoses like GAD, SAD, and panic disorder . This study aimed to examine rumination in these anxiety disorders separately.
• There was a lack of research on the specific components of rumination (brooding and reflection) across different disorders. This meta-analysis sought to address this gap by analyzing these subtypes when possible.
• The most recent comprehensive review of rumination across depression, anxiety, and eating disorders (Aldao et al., 2010) only included studies up to 2008. An updated meta-analysis was needed to incorporate more recent research.
• Previous reviews lacked quality assessments of included studies, limiting the ability to evaluate the strength of the evidence. This study aimed to conduct a formal quality appraisal.
• There was a need to examine potential moderators of the relationship between rumination and psychopathology, such as age, gender, and clinical factors, which had shown mixed results in past research.

By addressing these gaps in the literature, this meta-analysis aimed to provide a more comprehensive and up-to-date understanding of rumination as a transdiagnostic process across depression, anxiety, and eating disorders.

This meta-analytic review adhered to the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines. The project was registered on PROSPERO on July 31, 2020.

Search strategy and terms:

The initial search was conducted on July 29, 2020, using Medline, Embase, and PsychInfo databases.

Grey literature in the form of doctoral dissertations was searched on ProQuest Theses and Dissertations Global on August 4, 2020. A follow-up search was performed on all databases on December 1, 2021.

The search was limited to articles published after July 2008.

Medical Subject Headings (MeSH) terms for each disorder (depressive disorder, anxiety disorder, and eating disorder) were searched in combination with truncated terms for relevant specific disorders within the diagnostic categories set out in the DSM-5. The MeSH term “rumination” and associated truncated terms were also searched.

The full search strategies for each database are provided in the online supplementary materials.

Inclusion and exclusion criteria:

Inclusion criteria:

• Published and unpublished empirical articles and doctoral dissertations written in English
• At least one measure of rumination
• Participants aged 18-65 years
• Primary diagnosis of depression, anxiety, or eating disorder confirmed through clinical interview or provided by a health professional
• Cross-sectional and interventional studies with baseline measures of rumination, brooding, or reflection

Exclusion criteria:

• Participants younger than 18 or older than 65
• Self-report symptoms meeting clinical threshold without a formal diagnosis
• Primary diagnosis other than depression, anxiety, or eating disorder
• Sample comprised entirely of participants in remission
• Comorbid medical conditions
• Measures of repetitive negative thinking without a rumination subscale
• Only ecological momentary assessment (EMA) of rumination without baseline trait measure

Statistical measures:

Data analysis was conducted using the R package meta. Random effects modeling was used for the main analyses.

Standardized mean differences were used for group difference-based analyses, while r values (converted to z scores for analysis and back-transformed to r values for interpretation) were used for correlational analyses.

Heterogeneity was evaluated using Cochrane’s Q and quantified using I2. Metaregression was used to follow up on significant heterogeneity effects where data permitted.

Publication bias was assessed using funnel plots and Egger’s test.

Group Differences in Rumination:

• Total rumination was significantly higher in MDD, GAD, SAD, and AN groups compared to healthy controls. No significant difference was found for panic disorder (PD).
• Brooding was significantly higher in MDD, GAD, and SAD groups compared to healthy controls.
• Reflection was significantly higher in MDD and SAD groups compared to healthy controls.
• The largest differences in effect sizes for brooding were observed in the SAD group, followed by MDD and GAD.
• For reflection, the largest differences were found in the MDD group, followed by SAD.

Correlations Between Psychopathology and Rumination:

• Significant medium to large correlations were found between symptom severity and total rumination in MDD, GAD, SAD, and AN groups.
• Strong positive correlations were observed between depressive symptoms and brooding in MDD, and between anxiety symptoms and brooding in SAD.
• A smaller positive correlation was found between depressive symptoms and reflection in MDD.
• Nonsignificant correlations were observed between anxiety symptoms and brooding in GAD and PD, and between anxiety symptoms and reflection in SAD.

Moderation Effects:

• Gender was the only significant moderator, with stronger associations between depressive symptoms and brooding observed in samples with a higher proportion of males in the MDD group.
• Age, presence of comorbidities, and medication use were not significant moderators for group differences or correlations in the MDD group.

Publication Bias:

• Egger’s test showed publication bias for MDD group comparisons with healthy controls on total rumination and brooding.
• No significant publication bias was found for other comparisons and correlations, but this may be due to the small number of studies in some analyses.

Quality Appraisal:

• Studies generally reported hypotheses, aims, objectives, main outcomes, and patient characteristics well.
• External validity and internal validity regarding bias were generally sufficient.
• Issues were identified with reporting of recruitment periods, adjustment for confounding factors, and statistical power.

This meta-analysis provides several key insights into the role of rumination across depression, anxiety, and eating disorders:

• Transdiagnostic nature of rumination: The findings support the conceptualization of rumination as a transdiagnostic process, with elevated levels observed across MDD, GAD, SAD, and AN compared to healthy controls. This aligns with and extends previous theoretical models, such as the response-styles theory (Nolen-Hoeksema, 1991) and the transdiagnostic model (Nolen-Hoeksema & Watkins, 2011).
• Disorder-specific patterns: While rumination was elevated across multiple disorders, the strength of associations varied. For example, the relationship between rumination and psychopathology was strongest for MDD and SAD, followed by GAD and AN. This suggests that while rumination is indeed transdiagnostic, its role may be more pronounced in certain disorders.
• Importance of examining specific anxiety disorders: By analyzing GAD, SAD, and PD separately, the study revealed important differences that may have been obscured in previous research combining anxiety disorders. For instance, rumination was not significantly elevated in PD compared to controls, contrasting with findings for GAD and SAD.
• Differential roles of brooding and reflection: The analysis of rumination subtypes provided novel insights. Brooding showed stronger associations with psychopathology than reflection, particularly in SAD. This extends previous findings and suggests that certain components of rumination may be more closely linked to specific disorders.
• Gender as a potential moderator: The finding that associations between depressive symptoms and brooding were stronger in samples with more males challenges some previous research and highlights the need for further investigation into gender differences in rumination.
• Limited research in eating disorders: The study highlighted a significant gap in research on rumination in eating disorders, particularly for specific diagnoses like bulimia nervosa and binge eating disorder.

These findings extend previous research by providing a more nuanced understanding of rumination across multiple disorders and highlighting areas for future investigation.

Further research is needed to:

• Examine rumination in larger samples of patients with specific anxiety and eating disorder diagnoses.
• Investigate the causal relationships between rumination and psychopathology using longitudinal designs.
• Explore the efficacy of rumination-focused treatments across different disorders.
• Examine potential neurobiological underpinnings of rumination as a transdiagnostic process.
• Investigate how rumination interacts with other transdiagnostic factors (e.g., perfectionism, intolerance of uncertainty) across disorders.

The study had several methodological strengths:

• Comprehensive scope: The meta-analysis examined rumination across multiple disorders, providing a broader perspective than previous reviews focused on single diagnostic categories.
• Specific diagnostic groups: By analyzing specific anxiety disorders separately, the study revealed important differences that might have been overlooked in previous research.
• Analysis of rumination subtypes: The inclusion of brooding and reflection as separate constructs provided more nuanced insights into the role of rumination across disorders.
• Quality assessment: The study included a formal quality appraisal of included studies, which was lacking in previous reviews.
• Up-to-date literature: The analysis included studies published up to December 2021, providing a current synthesis of the field.
• Adherence to guidelines: The study followed PRISMA guidelines and was pre-registered, enhancing its methodological rigor and transparency.
• Examination of moderators: The analysis attempted to investigate potential moderators of the relationship between rumination and psychopathology, addressing an important gap in the literature.

Limitations

Several limitations should be considered when interpreting the results:

• Small sample sizes: For some diagnostic groups (e.g., PD, AN) and analyses of brooding and reflection, the number of available studies was small, limiting the reliability and generalizability of findings.
• Reliance on self-report measures: The assessment of rumination was based on self-report questionnaires, which may be subject to bias and may not fully capture the complexity of ruminative processes.
• Cross-sectional design : Most included studies were cross-sectional, limiting the ability to draw causal conclusions about the relationship between rumination and psychopathology.
• Publication bias: Significant publication bias was detected for some analyses, particularly those involving MDD, which may have inflated effect sizes.
• Limited data on moderators: Insufficient data were available to thoroughly examine potential moderators across all diagnostic groups, restricting analyses to the MDD group for most moderators.
• Exclusion of remitted patients: By focusing only on current diagnoses, the study may have missed important information about the role of rumination in the course of disorders.
• Age restrictions: The exclusion of studies with participants under 18 or over 65 limits the generalizability of findings to these age groups.
• Lack of non-Western samples: Most included studies were conducted in Western countries, potentially limiting the cultural generalizability of the findings.

These limitations highlight the need for further research with larger, more diverse samples, longitudinal designs, and alternative methods of assessing rumination.

Implications

The findings of this meta-analysis have several important implications for clinical psychology practice and research:

• Assessment and diagnosis: The strong associations between rumination and multiple disorders suggest that assessing rumination could be valuable in the diagnostic process. Clinicians should consider incorporating measures of rumination, particularly brooding, when evaluating patients with depressive, anxiety, and eating disorders.
• Treatment approaches: The transdiagnostic nature of rumination supports the development and use of treatment approaches that specifically target ruminative processes across disorders. Rumination-focused cognitive-behavioral therapy (RFCBT) and other interventions targeting repetitive negative thinking may be beneficial for multiple conditions.
• Personalized interventions: The variation in the strength of associations between rumination and different disorders suggests that the emphasis on rumination in treatment may need to be tailored to specific diagnoses. For example, interventions for SAD and MDD may need to focus more heavily on reducing rumination compared to those for PD.
• Focus on brooding: The stronger associations found for brooding compared to reflection suggest that treatments should particularly target this maladaptive component of rumination.
• Gender considerations: The finding that associations between depressive symptoms and brooding were stronger in samples with more males highlights the need for clinicians to be aware of potential gender differences in ruminative processes.
• Transdiagnostic research: The results support the value of transdiagnostic approaches in psychopathology research and suggest that future studies should continue to examine processes like rumination across multiple disorders.
• Eating disorder focus: The limited research available on rumination in eating disorders, particularly for specific diagnoses like bulimia nervosa and binge eating disorder, indicates a need for more studies in this area.
• Longitudinal research: To better understand the causal relationships between rumination and psychopathology, more longitudinal studies are needed. This could inform prevention efforts by identifying whether rumination precedes the onset of disorders.
• Measurement development: The limitations of self-report measures suggest a need for developing and validating alternative methods of assessing rumination, such as behavioral tasks or ecological momentary assessment.
• Neurobiological research: The consistent finding of elevated rumination across multiple disorders suggests that investigating the neural correlates of rumination as a transdiagnostic process could be a fruitful area for future research.

These implications underscore the importance of considering rumination in both clinical practice and research across a range of psychological disorders.

By targeting rumination as a transdiagnostic process, there is potential to improve assessment, treatment, and prevention efforts for multiple conditions.

Primary reference

Rickerby, N., Krug, I., Fuller-Tyszkiewicz, M., Forte, E., Davenport, R., Chayadi, E., & Kiropoulos, L. (2024). Rumination across depression, anxiety, and eating disorders in adults: A meta-analytic review.  Clinical Psychology: Science and Practice, 31 (2), 251–268.  https://doi.org/10.1037/cps0000110

Other references

Aldao, A., Nolen-Hoeksema, S., & Schweizer, S. (2010). Emotion-regulation strategies across psychopathology: A meta-analytic review.  Clinical psychology review ,  30 (2), 217-237. https://doi.org/10.1016/j.cpr.2009.11.004

McLaughlin, K. A., & Nolen-Hoeksema, S. (2011). Rumination as a transdiagnostic factor in depression and anxiety.  Behaviour research and therapy ,  49 (3), 186-193. https://doi.org/10.1016/j.brat.2010.12.006

Nolen-Hoeksema, S. (1991). Responses to depression and their effects on the duration of depressive episodes.  Journal of abnormal psychology ,  100 (4), 569. https://doi.org/10.1037/0021-843X.100.4.569

Nolen-Hoeksema, S., & Watkins, E. R. (2011). A heuristic for developing transdiagnostic models of psychopathology: Explaining multifinality and divergent trajectories.  Perspectives on psychological science ,  6 (6), 589-609. https://doi.org/10.1177/1745691611419672

Olatunji, B. O., Naragon-Gainey, K., & Wolitzky-Taylor, K. B. (2013). Specificity of rumination in anxiety and depression: A multimodal meta‐analysis.  Clinical Psychology: Science and Practice ,  20 (3), 225. https://doi.org/10.1037/h0101719

Smith, K. E., Mason, T. B., & Lavender, J. M. (2018). Rumination and eating disorder psychopathology: A meta-analysis.  Clinical psychology review ,  61 , 9-23. https://doi.org/10.1016/j.cpr.2018.03.004

Keep Learning

• How might the finding that rumination is elevated across multiple disorders inform the development of transdiagnostic treatment approaches?
• What are the potential implications of the stronger associations found for brooding compared to reflection? How might this influence cognitive-behavioral interventions for depression and anxiety?
• Given the limitations of self-report measures, what alternative methods could be developed to assess rumination more objectively or ecologically?
• How might the observed gender differences in the relationship between depressive symptoms and brooding inform our understanding of gender disparities in depression rates?
• Considering the limited research on rumination in eating disorders, what specific research questions should be prioritized to advance our understanding in this area?
• How might the neuroscience of rumination as a transdiagnostic process inform our understanding of the shared neural mechanisms underlying different psychological disorders?
• Given the cross-sectional nature of most included studies, how could future longitudinal research better elucidate the causal relationships between rumination and psychopathology?
• How might cultural factors influence ruminative processes across different disorders? How can future research address the limited representation of non-Western samples in this field?