• Flashes Safe Seven
  • FlashLine Login
  • Faculty & Staff Phone Directory
  • Emeriti or Retiree
  • All Departments
  • Maps & Directions

Kent State University Home

  • Building Guide
  • Departments
  • Directions & Parking
  • Faculty & Staff
  • Give to University Libraries
  • Library Instructional Spaces
  • Mission & Vision
  • Newsletters
  • Circulation
  • Course Reserves / Core Textbooks
  • Equipment for Checkout
  • Interlibrary Loan
  • Library Instruction
  • Library Tutorials
  • My Library Account
  • Open Access Kent State
  • Research Support Services
  • Statistical Consulting
  • Student Multimedia Studio
  • Citation Tools
  • Databases A-to-Z
  • Databases By Subject
  • Digital Collections
  • Discovery@Kent State
  • Government Information
  • Journal Finder
  • Library Guides
  • Connect from Off-Campus
  • Library Workshops
  • Subject Librarians Directory
  • Suggestions/Feedback
  • Writing Commons
  • Academic Integrity
  • Jobs for Students
  • International Students
  • Meet with a Librarian
  • Study Spaces
  • University Libraries Student Scholarship
  • Affordable Course Materials
  • Copyright Services
  • Selection Manager
  • Suggest a Purchase

Library Locations at the Kent Campus

  • Architecture Library
  • Fashion Library
  • Map Library
  • Performing Arts Library
  • Special Collections and Archives

Regional Campus Libraries

  • East Liverpool
  • College of Podiatric Medicine

systematic literature review methodology ppt

  • Kent State University
  • Systematic Methods for Literature Reviews
  • Workshop Handout & PPT

Systematic Methods for Literature Reviews: Workshop Handout & PPT

  • Systematic Review Methods (SRM)
  • SRM - Library research models
  • SRM - General steps
  • SRM - Checklists
  • SRM - Manage
  • SRM - Books/Articles & More
  • SRM - Examples & related

Workshop Handout and PPT

  • SRM Handout
  • SRM Workshop PPT
  • << Previous: SRM - General/What/Why
  • Next: SRM - Library research models >>
  • Last Updated: Sep 1, 2021 1:38 PM
  • URL: https://libguides.library.kent.edu/systematicreviews

Street Address

Mailing address, quick links.

  • How Are We Doing?
  • Student Jobs

Information

  • Accessibility
  • Emergency Information
  • For Our Alumni
  • For the Media
  • Jobs & Employment
  • Life at KSU
  • Privacy Statement
  • Technology Support
  • Website Feedback

University of Maryland Libraries Logo

Systematic Review

  • Library Help
  • What is a Systematic Review (SR)?

Steps of a Systematic Review

  • Framing a Research Question
  • Developing a Search Strategy
  • Searching the Literature
  • Managing the Process
  • Meta-analysis
  • Publishing your Systematic Review

Forms and templates

Logos of MS Word and MS Excel

Image: David Parmenter's Shop

  • PICO Template
  • Inclusion/Exclusion Criteria
  • Database Search Log
  • Review Matrix
  • Cochrane Tool for Assessing Risk of Bias in Included Studies

   • PRISMA Flow Diagram  - Record the numbers of retrieved references and included/excluded studies. You can use the Create Flow Diagram tool to automate the process.

   •  PRISMA Checklist - Checklist of items to include when reporting a systematic review or meta-analysis

PRISMA 2020 and PRISMA-S: Common Questions on Tracking Records and the Flow Diagram

  • PROSPERO Template
  • Manuscript Template
  • Steps of SR (text)
  • Steps of SR (visual)
  • Steps of SR (PIECES)

Adapted from  A Guide to Conducting Systematic Reviews: Steps in a Systematic Review by Cornell University Library

Source: Cochrane Consumers and Communications  (infographics are free to use and licensed under Creative Commons )

Check the following visual resources titled " What Are Systematic Reviews?"

  • Video  with closed captions available
  • Animated Storyboard
  • << Previous: What is a Systematic Review (SR)?
  • Next: Framing a Research Question >>
  • Last Updated: Jan 26, 2024 4:35 PM
  • URL: https://lib.guides.umd.edu/SR

U.S. flag

An official website of the Department of Health & Human Services

AHRQ: Agency for Healthcare Research and Quality

Effective Health Care Program

  • EHC en Español

Main Navigation Menu

Training Modules for the Systematic Reviews Methods Guide

Page contents.

Due to greater interest in comparative effectiveness research and a desire to expand capacity to conduct Comparative Effectiveness Reviews, AHRQ has produced training modules to familiarize new investigators with the methodological framework and guidance outlined in AHRQ's Methods Guide for Effectiveness and Comparative Effectiveness Reviews .

The training modules below can be used by faculty engaged in educating investigators and clinicians interested in systematic review methods.

These presentations are available in PowerPoint format only. People using assistive technology may not be able to fully access information in these files. For additional assistance, please contact us .

Presentations*

The ahrq training modules for the systematic reviews methods guide: an introduction.

  • Presentation slides (PPT; 5.8 MB)

Topic Refinement

  • Presentation slides (PPT; 2.2 MB)

Analytic Frameworks

  • Presentation slides (PPT; 1.8 MB)

Study Eligibility Criteria

  • Presentation slides (PPT; 599 kB)

Searching for Relevant Studies

When to select observational studies as evidence.

  • Presentation slides (PPT; 993 kB)
  • Quiz slides (PPT; 1,7 MB)

Prioritization and Selection of Harms

  • Prioritization and Selection of Harms for Inclusion in Systematic Reviews (PDF, 251 KB)

Data Extraction

  • Presentation slides (PPT; 3.3 MB)

Assessing the Quality of Individual Studies

  • Presentation slides (PPT; 2.1 MB)

Assessing Applicability

Presentation of findings.

  • Presentation slides (PPT; 3.0 MB)

Quantitative Synthesis

  • Learning Objectives and Recommendations (PDF, 344 MB)
  • Chapter 1. Decision To Combine Trials (PDF, 385 MB)
  • Chapter 2. Presenting Different Effects for Different Data Types (PDF, 225 MB)
  • Chapter 3. Choice of Statistical Model for Combining Studies (PDF, 168 MB)
  • Chapter 4. Quantifying, Testing, and Exploring Statistical Heterogeneity (PDF, 376 MB)
  • Chapter 5. Network Meta-Analysis (PDF, 256 MB)
  • Assessing the Risk of Bias in Systematic Reviews of Health Care Interventions (PDF, 296 KB)

Strength of Evidence

Reporting the review.

  • Presentation slides (PPT; 2.7 MB)

*Note: Users of some versions of Microsoft Internet Explorer may discover that the slides appear in their browser window when they select the link above. In that case, the presentation in the browser window may be saved by selecting "Save As" under the browser "File" menu. The file can then be opened and viewed in Microsoft PowerPoint.

Project Timeline

Methods guide for effectiveness and comparative effectiveness reviews.

This product is part of a curated collection. Check out other items in this collection.

Methods Guide – Chapter | Sep 19, 2022 Inclusion of Nonrandomized Studies of Interventions in Systematic Reviews of Intervention Effectiveness: An Update

Methods Guide – Chapter | Feb 23, 2018 Quantitative Synthesis—An Update

Methods Guide – Chapter | Oct 5, 2009 Comparing Medical Interventions: AHRQ and the Effective Health Care Program

Methods Guide – Chapter | Oct 5, 2009 Principles in Developing and Applying Guidance for Comparing Medical Interventions

Methods Guide – Chapter | Oct 5, 2009 Identifying, Selecting and Refining Topics for Comparative Effectiveness Systematic Reviews

Methods Guide – Chapter | Nov 19, 2012 Developing and Selecting Topic Nominations for Systematic Reviews

Methods Guide – Chapter | Jan 24, 2013 The Refinement of Topics for Systematic Reviews: Lessons and Recommendations From the Effective Health Care Program

Methods Guide – Chapter | Jan 5, 2011 Finding Evidence for Comparing Medical Interventions

Methods Guide – Chapter | Nov 18, 2013 Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions: AHRQ and the Effective Health Care Program

Methods Guide – Chapter | Feb 20, 2013 Avoiding Bias in Selecting Studies

Methods Guide – Chapter | Jun 14, 2010 Selecting Observational Studies for Comparing Medical Interventions

Methods Guide – Chapter | Dec 13, 2017 Assessing the Risk of Bias in Systematic Reviews of Health Care Interventions

Methods Guide – Chapter | Mar 8, 2012 Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions

Methods Guide – Chapter | Dec 30, 2010 Assessing the Applicability of Studies When Comparing Medical Interventions

Methods Guide – Chapter | Feb 2, 2018 Prioritization and Selection of Harms for Inclusion in Systematic Reviews

Methods Guide – Chapter | Oct 5, 2009 Assessing Harms when Comparing Medical Interventions

Research Report | Dec 29, 2016 Updating Quantitative Synthesis

Methods Guide – Chapter | Nov 18, 2013 Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update

Methods Guide – Chapter | Oct 5, 2009 Using Existing Systematic Reviews to Replace de Novo Processes in CERs

Methods Guide – Chapter | Feb 26, 2015 Integrating Bodies of Evidence: Existing Systematic Reviews and Primary Studies

Methods Guide – Chapter | Jul 27, 2011 Updating Comparative Effectiveness Reviews: Current Efforts in AHRQ’s Effective Health Care Program

Methods Guide – Chapter | Oct 18, 2016 Guidance for the Conduct and Reporting of Modeling and Simulation Studies in the Context of Health Technology Assessment

View Collection

You Might Also Like

  • Research Report Nov 09, 2022 Analysis of Requirements for Coverage With Evidence Development (CED) – Topic Refinement
  • Series Overview Nov 04, 2021 AHRQ EPC Program Helps Health Systems Use Evidence
  • Abstract Oct 14, 2021 Outcome Measure Harmonization and Data Infrastructure for Patient-Centered Outcomes Research in Depression
  • Series Overview Aug 01, 2019 AHRQ EPC Pilot Projects Summary: Improving Health Systems' Access to High Quality Evidence
  • Research Protocol Jun 12, 2019 Impact of Community Health Worker Certification on Workforce and Service Delivery for Asthma and Other Selected Chronic Diseases

Internet Citation: Presentation: Training Modules for the Systematic Reviews Methods Guide. Content last reviewed July 2022. Effective Health Care Program, Agency for Healthcare Research and Quality, Rockville, MD. https://effectivehealthcare.ahrq.gov/products/cer-methods-guide/presentations

Select to copy citation

How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses

Affiliations.

  • 1 Behavioural Science Centre, Stirling Management School, University of Stirling, Stirling FK9 4LA, United Kingdom; email: [email protected].
  • 2 Department of Psychological and Behavioural Science, London School of Economics and Political Science, London WC2A 2AE, United Kingdom.
  • 3 Department of Statistics, Northwestern University, Evanston, Illinois 60208, USA; email: [email protected].
  • PMID: 30089228
  • DOI: 10.1146/annurev-psych-010418-102803

Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question. The best reviews synthesize studies to draw broad theoretical conclusions about what a literature means, linking theory to evidence and evidence to theory. This guide describes how to plan, conduct, organize, and present a systematic review of quantitative (meta-analysis) or qualitative (narrative review, meta-synthesis) information. We outline core standards and principles and describe commonly encountered problems. Although this guide targets psychological scientists, its high level of abstraction makes it potentially relevant to any subject area or discipline. We argue that systematic reviews are a key methodology for clarifying whether and how research findings replicate and for explaining possible inconsistencies, and we call for researchers to conduct systematic reviews to help elucidate whether there is a replication crisis.

Keywords: evidence; guide; meta-analysis; meta-synthesis; narrative; systematic review; theory.

  • Guidelines as Topic
  • Meta-Analysis as Topic*
  • Publication Bias
  • Review Literature as Topic
  • Systematic Reviews as Topic*

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

  • Formulate a research question
  • Develop a protocol
  • Search for all relevant studies
  • Apply the selection criteria
  • Extract the data
  • Synthesize the data
  • Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism. Run a free check.

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

  • A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
  • If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
  • Access to databases and journal archives. Often, your educational institution provides you with access.
  • Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
  • Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

  • They minimize research bias by considering all available evidence and evaluating each study for bias.
  • Their methods are transparent , so they can be scrutinized by others.
  • They’re thorough : they summarize all available evidence.
  • They can be replicated and updated by others.

Systematic reviews also have a few cons .

  • They’re time-consuming .
  • They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

  • Allow you to more effectively communicate your research to other researchers and practitioners
  • Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

  • Population(s) or problem(s)
  • Intervention(s)
  • Comparison(s)

You can rearrange these four components to write your research question:

  • What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

  • Type of study design(s)
  • The population of patients with eczema
  • The intervention of probiotics
  • In comparison to no treatment, placebo , or non-probiotic treatment
  • The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
  • Randomized control trials, a type of study design

Their research question was:

  • What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

  • Background information : Provide the context of the research question, including why it’s important.
  • Research objective (s) : Rephrase your research question as an objective.
  • Selection criteria: State how you’ll decide which studies to include or exclude from your review.
  • Search strategy: Discuss your plan for finding studies.
  • Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

  • Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
  • Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
  • Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
  • Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

  • Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
  • Handsearch: Conference proceedings and reference lists of articles
  • Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
  • Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

  • Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
  • Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

  • Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
  • Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

  • Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
  • Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

  • Abstract : A summary of the review
  • Introduction : Including the rationale and objectives
  • Methods : Including the selection criteria, search method, data extraction method, and synthesis method
  • Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
  • Discussion : Including interpretation of the results and limitations of the review
  • Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

  • Student’s  t -distribution
  • Normal distribution
  • Null and Alternative Hypotheses
  • Chi square tests
  • Confidence interval
  • Quartiles & Quantiles
  • Cluster sampling
  • Stratified sampling
  • Data cleansing
  • Reproducibility vs Replicability
  • Peer review
  • Prospective cohort study

Research bias

  • Implicit bias
  • Cognitive bias
  • Placebo effect
  • Hawthorne effect
  • Hindsight bias
  • Affect heuristic
  • Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved February 22, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, what is your plagiarism score.

U.S. flag

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Cover of Handbook of eHealth Evaluation: An Evidence-based Approach

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

  • formulating the research question(s) and objective(s),
  • searching the extant literature,
  • screening for inclusion,
  • assessing the quality of primary studies,
  • extracting data, and
  • analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

  • Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
  • Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
  • Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
  • Analyzing data using quantitative or qualitative methods.
  • Presenting results in summary of findings tables.
  • Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Table 9.1. Typology of Literature Reviews (adapted from Paré et al., 2015).

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

  • Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
  • Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
  • Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
  • Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
  • A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
  • Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
  • Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
  • Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
  • Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
  • Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
  • Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
  • Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
  • Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
  • Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
  • Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
  • Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
  • Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
  • Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
  • Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
  • Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
  • Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
  • Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
  • Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
  • Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
  • Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
  • Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
  • King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
  • Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
  • Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
  • Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
  • Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
  • Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
  • Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
  • Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
  • Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
  • Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
  • Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
  • Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
  • Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
  • Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
  • Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
  • Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
  • Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
  • Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
  • Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
  • Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
  • Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
  • Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
  • Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
  • Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
  • Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
  • Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
  • Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
  • Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
  • Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
  • Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
  • Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
  • Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
  • Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
  • Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
  • Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

  • Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
  • PDF version of this title (4.5M)
  • Disable Glossary Links

In this Page

  • Introduction
  • Overview of the Literature Review Process and Steps
  • Types of Review Articles and Brief Illustrations
  • Concluding Remarks

Related information

  • PMC PubMed Central citations
  • PubMed Links to PubMed

Recent Activity

  • Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

statistics

  • Locations and Hours
  • UCLA Library
  • Research Guides
  • Biomedical Library Guides

Systematic Reviews

  • Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

  • Planning Your Systematic Review
  • Database Searching
  • Creating the Search
  • Search Filters & Hedges
  • Grey Literature
  • Managing & Appraising Results
  • Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x

  • << Previous: Home
  • Next: Planning Your Systematic Review >>
  • Last Updated: Feb 20, 2024 9:33 AM
  • URL: https://guides.library.ucla.edu/systematicreviews

research methodology 5 systematic literature review slr

Research Methodology 5. Systematic Literature Review (SLR)

Nov 09, 2019

2.24k likes | 3.67k Views

Research Methodology 5. Systematic Literature Review (SLR). Romi Satria Wahon o [email protected] http ://romisatriawahono.net/rm WA/SMS : +6281586220090. Romi Satria Wahono. SD Sompok Semarang (1987) SMPN 8 Semarang (1990) SMA Taruna Nusantara Magelang (1993)

Share Presentation

  • software technology
  • systematic literature
  • literature review
  • systematic literature reviews
  • research interests software engineering

billya

Presentation Transcript

Research Methodology5. Systematic Literature Review (SLR) Romi Satria [email protected]://romisatriawahono.net/rmWA/SMS: +6281586220090

Romi Satria Wahono • SD Sompok Semarang (1987) • SMPN 8 Semarang (1990) • SMA Taruna NusantaraMagelang (1993) • B.Eng, M.Eng and Ph.Din Software Engineering fromSaitama University Japan (1994-2004)Universiti Teknikal Malaysia Melaka (2014) • Research Interests: Software Engineering,Machine Learning • Founder danKoordinatorIlmuKomputer.Com • Peneliti LIPI (2004-2007) • Founder dan CEO PT Brainmatics Cipta Informatika

Course Outline

5. Systematic Literature Review (SLR) 5.1 Pengantar SLR 5.2 Tahapan Planning 5.3 Tahapan Conducting 5.4 Tahapan Reporting

5.1 Pengantar SLR

Literature Review • Literature Review is a critical and in depth evaluation of previous research (Shuttleworth, 2009)(https://explorable.com/what-is-a-literature-review) • A summary and synopsis of a particular area of research, allowing anybody reading the paper to establish the reasons for pursuing a particular research • A good Literature Review evaluates quality and findings of previous research

ManfaatMereviewLiteratur • Memperdalampengetahuantentangbidang yang diteliti (Textbooks) • Mengetahuihasilpenelitian yangberhubungan dan yang sudah pernahdilaksanakan (Related Research) (Paper) • Mengetahuiperkembanganilmupadabidang yang kitapilih (state-of-the-art) (Paper) • Memperjelasmasalahpenelitian (Paper)

Literature Review Methods • Typesand Methods of Literature Review: • Traditional Review • Systematic Literature Review or Systematic Review • Systematic Mapping Study (Scoping Study) • Tertiary Study • SLR is now well established review method in the field of software engineering (Kitchenham & Charters, Guidelines in performing Systematic Literature Reviews in Software Engineering, EBSE Technical Report version 2.3, 2007)

1. Traditional Review • Provides an overview of the research findings on particular topics • Advantages: produce insightful, valid syntheses of the research literature if conducted by the expert • Disadvantages: vulnerable to unintentional and intentional bias in the selection, interpretation and organization of content • Examples: • Liao et al., Intrusion Detection System: A Comprehensive Review, Journal of Network and Computer Applications, 36(2013) • Galar et al., A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based Approaches, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 42, No. 4, July 2012 • CagatayCatal, Software fault prediction: A literature review and current trends, Expert Systems with Applications 38 (2011)

2. Systematic Mapping Study • Suitable for a very broad topic • Identify clusters of evidence (making classification) • Direct the focus of future SLRs • To identify areas for future primary studies • Examples: • Neto et al., A systematic mapping study of software product lines testing, Information and Software Technology Vol. 53, Issue 5, May 2011 • Elberzhager et al., Reducing test effort: A systematic mapping study on existing approaches, Information and Software Technology 54 (2012)

3. Systematic Literature Review (SLR) • The purpose of a systematic literature reviews is to provide as complete a list as possible of all the published studies relating to a particular subject area • A process of identifying, assessing, and interpreting all available research evidence, to provide answers for a particular research question • A form of secondary study that uses a well-defined methodology • SLRs are well established in other disciplines, particularly medicine. They integrate an individual clinical expertise and facilitate access to the outcomes of the research (Kitchenham & Charters, Guidelines in performing Systematic Literature Reviews in Software Engineering, EBSE Technical Report version 2.3, 2007)

3. Systematic Literature Review (SLR) Examples of SLR: • Hall et al., A Systematic Literature Review on Fault Prediction Performance in Software Engineering, IEEE Transaction on Software Engineering, Vol. 38, No. 6, 2012 • Romi Satria Wahono, A Systematic Literature Review of Software Defect Prediction: Research Trends, Datasets, Methods and Frameworks, Journal of Software Engineering, Vol. 1, No. 1, April 2015 • Matthias Galster, Danny Weyns, Dan Tofan, BartoszMichalik, and Paris Avgeriou, Variability in Software Systems: A Systematic Literature Review, IEEE Transactions on Software Engineering, Vol 40, No 3, 2014

4. Tertiary study • Is a SLR of SLRs • To answer a more wider question • Uses the same method as in SLR • Potentially less resource intensive • Examples: • Kitchenham et al., Systematic literature reviews in software engineering – A tertiary study, Information and Software Technology 52 (2010) • Cruzes et al., Research synthesis in software engineering: A tertiary study, Information and Software Technology 53 (2011)

Tahapan SLR 1. Formulate the Review’sResearchQuestion 2. Develop the Review’sProtocol 5.1 PLANNING 1. Identify the RelevantLiterature 2. Perform Selection of PrimaryStudies 3. Perform DataExtraction 4. Assess Studies’ Quality 5. Conduct Synthesis of Evidence 5.2 CONDUCTING 1. Write Up the SLR Paper 2. Choose the Right Journal 5.3 REPORTING

5.1Tahapan Planning Formulate the Review’s Research Question Develop the Review’s Protocol

Formulate the Review’s Research Question • Features of good question: • The RQ is meaningful and important to practitioners and researchers. • The RQ will lead to changes in current software engineering practice or to increase confidence in the value of current practice • The RQ will identify discrepancies between commonly held beliefs and the reality • RQ can be derived primarily based on researcher’s interest • An SLR for PhD thesis should identify existing basis for the research work and where it fits in the current body of knowledge

The Research Question (RQ) • Is the most important part in any SLR • Is not necessarily the same as questions addressed in your research • Is used to guide the search process • Is used to guide the extraction process • Data analysis (synthesis of evidence) is expected to answer your SLR’s RQ

RQ and PICOC The formulation of RQs about effectiveness of a treatment should focus on 5 elements known as PICOC: • Population (P)- the target group for the investigation (e.g. people, software etc.) • Intervention (I) - specifies the investigation aspects or issues of interest to the researchers • Comparison (C)– aspect of the investigation with which the intervention is being compared to • Outcomes (O)– the effect of the intervention • Context (C)– the setting or environment of the investigation (Petticrew et al., Systematic Reviews in the Social Sciences: A Practical Guide, Blackwell Publishing, 2006)

Example of PICOC (Kitchenham et al., 2007) Kitchenham et al., A Systematic Review of Cross- vs. Within-Company Cost Estimation Studies, IEEE Transactions on Software Engineering, 33 (5), 2007

Example of PICOC (Wahono, 2015) Romi Satria Wahono, A Systematic Literature Review of Software Defect Prediction: Research Trends, Datasets, Methods and Frameworks, Journal of Software Engineering, Vol. 1, No. 1, pp. 1-16, April 2015

Example of RQs (Kitchenham, 2007) Kitchenham et al., A Systematic Review of Cross- vs. Within-Company Cost Estimation Studies, IEEE Transactions on Software Engineering, 33 (5), 2007 • RQ1: What evidence is there that cross-company estimation models are not significantly different from within-company estimation models for predicting effort for software/Web projects? • RQ2: What characteristics of the study data sets and the data analysis methods used in the study affect the outcome of within- and cross-company effort estimation accuracy studies? • RQ3: Which experimental procedure is most appropriate for studies comparing within- and cross-company estimation models?

Example of RQs (Davis et al., 2006) Davis et al., Effectiveness of Requirements Elicitation Techniques: Empirical Results Derived from a Systematic Review, 14th IEEE Requirements Engineering Conference, 2006 • RQ: What elicitation technique is most efficient in a particular setting?

Example of RQs (Radjenovic et al., 2013) Radjenovic et al., Software fault prediction metrics: A systematic literature review, Information and Software Technology, Vol. 8, No. 55, pp. 1397-1418, 2013 • RQ1: Which software metrics for fault prediction exist in literature? • RQ2: What data sets are used for evaluating metrics?

Example of RQ (Wahono, 2015) Romi Satria Wahono, A Systematic Literature Review of Software Defect Prediction: Research Trends, Datasets, Methods and Frameworks, Journal of Software Engineering, Vol. 1, No. 1, pp. 1-16, April 2015

2. Develop the Review’s Protocol • A plan that specifies the basic review procedures (method) • Components of a protocol: • Background • Research Questions • Search terms • Selection criteria • Quality checklist and procedures • Data extraction strategy • Data synthesis strategy

5.2Tahapan Conducting Identify the Relevant Literature Perform Selection of Primary Studies Perform Data Extraction Assess Studies’ Quality Conduct Synthesis of Evidence

1. Identifying Relevant Literature • Involves a comprehensive and exhaustive searching of studies to be included in the review • Define a search strategy • Search strategies are usually iterative and benefit from: • Preliminary searches (to identify existing review and volume of studies) • Trial searches (combination of terms from RQ) • Check the search results against list of known studies • Consult the experts in the field

Approach to Construct Search String • Derive major terms used in the review questions based on the PICOC • List the keywords mentioned in the article • Search for synonyms and alternative words • Use the boolean OR to incorporate alternative synonyms • Use the boolean AND to link major terms

Example of Search String (Kitchenham et al., 2007) • Kitchenham et al. (2007) used their structured questions to construct search strings for use with electronic databases: • Population: software OR application OR product OR Web OR WWW OR Internet OR World-Wide Web OR project OR development • Intervention: cross company OR cross organisation OR cross organization OR multiple-organizational OR multiple-organisational model OR modeling OR modelling effort OR cost OR resource estimation OR prediction OR assessment • Contrast: within-organisation OR within-organization OR within-organizational OR within-organisational OR single company OR single organisation • Outcome: Accuracy OR Mean Magnitude Relative Error • The search strings were constructed by linking the four OR lists using the Boolean AND

Example of Search String (Wahono, 2015) Search String: (software OR applicati* OR systems ) AND (fault* OR defect* OR quality OR error-prone) AND (predict* OR prone* OR probability OR assess* OR detect* OR estimat* OR classificat*) Romi Satria Wahono, A Systematic Literature Review of Software Defect Prediction: Research Trends, Datasets, Methods and Frameworks, Journal of Software Engineering, Vol. 1, No. 1, pp. 1-16, April 2015

Example of Search String (Salleh et al., 2011) • The complete search term initially used : (student* OR undergraduate*) AND (pair programming OR pair-programming) AND ((experiment* OR measurement OR evaluation OR assessment) AND (effective* OR efficient OR successful) • A very limited number of results retrieved when using the complete string, thus a much simpler string was derived. • Subject librarian suggested to revise the search string: “pair programming” OR “pair-programming”

Sources of Evidence • Digital libraries • Reference lists from relevant primary studies and review articles • Journals (including company journals such as the IBM Journal of Research and Development), grey literature (i.e. technical reports, work in progress) • Conference proceedings • Research registers • The Internet (google) • Direct contact specific researcher(s)

Studies SelectionStrategy(Wahono, 2015) • Publication Year: • 2000-2013 • Publication Type: • Journal • Conference Proceedings • Search String: softwareAND(fault* OR defect* OR quality OR error-prone) AND(predict* OR prone* OR probability OR assess* OR detect* ORestimat* ORclassificat*) • Selected Studies: • 71

Sources of Evidence (Kitchenham et al., 2007) • The search strings were used on 6 digital libraries: • INSPEC , El Compendex, Science Direct, Web of Science, IEEExplore, ACM Digital library • Search specific journals and conf. proceedings: • Empirical Software Engineering (J) • Information and Software Technology (J) • Software Process Improvement and Practice (J) • Management Science (J) • International Software Metrics Symposium (C) • International Conference on Software Engineering (C) • Manual search: • Evaluation and Assessment in Software Engineering (C) • Check references of each relevant article • Contact researchers

Managing Bibliography • Use relevant Bibliographic package to manage large number of references • E.g. Mendeley, EndNote, Zotero, JabRef Reference Manager etc.

Documenting the Search • The process of conducting SLR must be transparent and replicable • The review should be documented in sufficient detail • The search should be documented and changes noted • Unfiltered search results should be saved for possible reanalysis

2. Selection of Studies • Primary studies need to be assessed for their actual relevance • Set the criteria for including or excluding studies (decided earlier during protocol development, can be refined later) • Inclusion & exclusion criteria should be based on RQ • Selection process should be piloted • Study selection is a multistage process

Selection of Studies(Kitchenham et al., 2007) • Kitchenham et al. (2007) used the following inclusion criteria: • Any study that compared predictions of cross-company models with within-company models based on analysis of single company project data. • They used the following exclusion criteria: • Studies where projects were only collected from a small number of different sources (e.g. 2 or 3 companies) • Studies where models derived from a within-company data set were compared with predictions from a general cost estimation model.

Selection of Studies (Wahono, 2015)

Selection of Studies (Salleh et al., 2011) • Inclusion criteria: • to include any empirical studies of PP that involved highereducation students as the population of interest. • Exclusion criteria: • Papers presenting unsubstantiated claims made by the author(s), for which no evidence was available. • Papers about Agile/XP describing development practices other than PP, such as test-first programming, refactoring etc. • Papers that only described tools (software or hardware) that could support the PP practice. • Papers not written in English. • Papers involving students but outside higher education

3. Assessing Studies’ Quality • To provide more detailed Inclusion/Exclusion criteria • To check whether quality differences provide an explanation for differences in study results • As a means of weighting the importance of individual studies when results are being synthesized • To guide the interpretation of findings and determine the strength of inferences • To guide recommendations for further research

Assessing Studies’ Quality • Quality relates to the extent to which the study minimizes bias and maximizes internal and external validity(Khan et al. 2001) • Quality Concepts Definition (Kitchenham & Charter, 2007)

Assessing Studies’ Quality • Assessing quality of studies: • Methodology or design of the study • Analysis of studies’ findings • Quality checklist or instrument need to be designed to facilitate quality assessment • Most quality checklists include questions aimed at assessing the extent to which articles have addressed bias and validity

Study Quality Assessment (Salleh et al., 2011)

Study Quality Assessment(Kitchenham et al., 2007) Kitchenham et al. (2007) constructed a quality questionnaire based on 5 issues affecting the quality of the study: • Is the data analysis process appropriate? • Did studies carry out a sensitivity or residual analysis? • Were accuracy statistics based on the raw data scale? • How good was the study comparison method? • The size of the within-company data set(e.g < 10 projects considered poor quality)

4. Data Extraction • Involve reading the full text article • Data extracted from primary studies should be recorded using data extraction form • The form should be designed and piloted when the protocol is defined • Collect all the information that can be used to answer the RQ and the study’s quality criteria • Both quality checklist and review data can be included in the same form • In case of duplicates publications (reporting the same data), refer the most complete one • For validation, a set of papers should be reviewed by 2 or more researchers. Compare results and resolve any conflicts

5. Synthesis of Evidence • Involves collating and summarizing the results of the included primary studies • Key objectives of data synthesis(Cruzes & Dyba, 2011): • to analyze and evaluate multiple studies • to select appropriate methods for integrating or providing new interpretive explanations about them • Synthesis can be: • Descriptive (narrative/non-quantitative) • Quantitative (e.g. meta-analysis) (Cruzes et al., Research Synthesis in Software Engineering: A tertiary study, Information and Software Technology, 53(5), 2011)

Descriptive Synthesis (Narrative) “An approach to the synthesis of findings from multiple studies that relies primarily on the use of words and text to summarize and explain the findings of the synthesis. It adopts a textual approach to the process of synthesis to ‘tell the story’ of the findings from the included studies.” (Popay et al. 2006) • Use tables to tabulate information extracted from included studies (e.g. population, number of included studies, study quality etc.) • Tables should be structured to highlight similarity or differences of study outcomes • Were the findings consistent (homogeneous) or inconsistent?

Quantitative Synthesis (Meta-Analysis) • Meta-analysis can be used to aggregate results or to pool data from different studies • The outcome of a meta-analysis is an average effect size with an indication of how variable that effect size is between studies • Meta-analysis involves three main steps: 1. Decide which studies to be included in the meta-analysis 2. Estimate an effect size for each individual study 3. Combine the effect sizes from the individual studies to estimate and test the combined effect • Results of the meta-analysis can be presented in a forest plot

5.3Tahapan Reporting Write Up the SLR Paper Choose the Right Journal

  • More by User

Literature Review and Research Problems

Literature Review and Research Problems

Literature Review and Research Problems. 9810009m Lisa 9810010m Angela. Outline. Major Reasons for Doing Literature Reviews The Search Process (Step 1 ~ 9 ) Critical Analysis of Literature Reviews Publication Bias Variable Quality in the Primary Research Studies

615 views • 39 slides

Research &amp; Literature Review

Research &amp; Literature Review

Research &amp; Literature Review. Prepared by: Kemal Akkaya &amp; Nezih Pala Presented by: Ahmed Ibrahim ([email protected]) Electrical &amp; Computer Engineering Florida International University. Overview of the Research Process. Topic 1. 2.

1.13k views • 21 slides

Systematic Review

Systematic Review

Systematic Review. An Introduction. Review. The general term for all attempts to synthesize the results and conclusions of two or more publications on a given topic. A review may or may not be systematic. Overview=Systematic Review.

2.16k views • 47 slides

LITERATURE REVIEW IN ACADEMIC RESEARCH

LITERATURE REVIEW IN ACADEMIC RESEARCH

LITERATURE REVIEW IN ACADEMIC RESEARCH. BEING A WORKSHOP PRESENTATION FOR CAPACITY BUILDING T O PG STUDENTS By Dr. Muhammad I. Bello Department of Geography Faculty of Social Sciences UNIVERSITY OF MAIDUGURI March 28 th , 2019. PURPOSEFUL READING OR PURPOSEFUL WASTING TIME.

451 views • 25 slides

MATH 5393 MATHEMATICS EDUCATION LITERATURE RESEARCH METHODOLOGY

MATH 5393 MATHEMATICS EDUCATION LITERATURE RESEARCH METHODOLOGY

MATH 5393:LITERATURE REVIEW AND RESEARCH METHODOLOGYReading, analyzing, and synthesizing mathematicseducation research literature for the purpose of informing teaching practice. Includes a study of qualitative research with a focus on the components of a research study (research question(s), li

832 views • 33 slides

Research Problem and Literature Review

Research Problem and Literature Review

Research Problem and Literature Review. Outline. Learn how to define a research problem in CS field. Learn how to conduct a L iterature Review. How to Find Research Problems.

570 views • 46 slides

A Systematic Literature Review for Software Sustainability Measures

A Systematic Literature Review for Software Sustainability Measures

A Systematic Literature Review for Software Sustainability Measures . We are quite sure sustainability is related to quality We need to define , formally, sustainability (similarly to quality definition)

343 views • 7 slides

A Systematic Review of the Literature

A Systematic Review of the Literature

A Systematic Review of the Literature. CHILDREN, INCARCERATED MOTHERS AND THE CHILD WELFARE SYSTEM CALIFORNIA STATE UNIVERSITY, LONG BEACH SCHOOL OF SOCIAL WORK ANGELIQUE FLORES, MAY 2013. Introduction.

497 views • 10 slides

Systematic Literature Review of the latest relevant literature on LWS

Systematic Literature Review of the latest relevant literature on LWS

235 views • 1 slides

Research Problem and Literature Review

Research Problem and Literature Review . Outline . Learn how to define a research problem in CS field. Learn how to conduct a L iterature Review. How to Find Research Problems.

590 views • 46 slides

Systematic Review

Interventions for preventing unintended pregnancies among adolescents. Systematic Review. NU517 Clinical Scholarship for Evidence Based Practice Cornelia C. Campbell (Roline) 22 September 2010. Clinical problem. Purpose of this systematic review. Unintended pregnancy among adolescents

487 views • 30 slides

Research question and relevance Literature review and hypotheses Methodology and data

Research question and relevance Literature review and hypotheses Methodology and data

Private Equity Investors and Going Privates in Continental Europe Bastian Hinterramskogler (TU Munich) ( joint with Ann-Kristin Achleitner (TU Munich) and André Betzer (University of Bonn)) EFM Symposium Corporate Governance and Control Cambridge, 10th April 2009. Agenda.

297 views • 19 slides

Systematic Review of the Literature: A Novel Research Approach

Systematic Review of the Literature: A Novel Research Approach

Systematic Review of the Literature: A Novel Research Approach. Objectives:. By the end of the session, the student will be able to: 1. State the definition of systematic review 2. State the reasons for doing a systematic review 3. State when a systematic review is needed

581 views • 31 slides

SYSTEMATIC LITERATURE SEARCHING

SYSTEMATIC LITERATURE SEARCHING

SYSTEMATIC LITERATURE SEARCHING. Sheila Fisken University of Edinburgh. The literature searching process “ is not an exact science but an art.” Samuel Butler Try to think of the process as a ‘journey not a destination’ Hearst 1999. Steps in the Review Process

496 views • 30 slides

Find Best Tips to Write Systematic Literature Review

Find Best Tips to Write Systematic Literature Review

Are you looking for help for literature review? If yes then visit Geoffandfrancis.co.uk and get best ideas for your literature review. Our main aim is to show your reader (your tutor) that you have read, and have a best grasp of, the main published work concerning a particular topic or question in your field. Know more details:- http://www.geoffandfrancis.co.uk

739 views • 6 slides

SYSTEMATIC LITERATURE SEARCHING

459 views • 30 slides

Systematic Literature  Searching:  Preparing a Campbell Review Protocol

Systematic Literature Searching: Preparing a Campbell Review Protocol

Systematic Literature Searching: Preparing a Campbell Review Protocol. Philip Curry Leyla De Amicis Robbie Gilligan. Overview. Background to review and introduction to systematic reviewing (Philip)

328 views • 30 slides

Motivation  Literature review  Research gap Research questions Research framework Methodology

Motivation Literature review Research gap Research questions Research framework Methodology

Role of Traditional Chinese Philosophies and New Product Development under Circular Economy Considerations on Performance: Empirical Evidence from Chinese Private Manufacturing Enterprises Lin WU Lecturer in Business Management Nachiappan Subramanian Tinghua Shen Angappa Gunasekaran

239 views • 18 slides

An Introduction to writing a Systematic review of literature - Scientific research support

An Introduction to writing a Systematic review of literature - Scientific research support

A systematic review is a type of literature review that uses a systematic method to collect Secondary data in which there is a comprehensive search for relevant studies on a particular topic, and those identified are then evaluated and synthesized according to a predetermined and explicit method This framework especially is essential for early career researchers and medical students to enhance their writing knowledge on the systematic review of the literature. Why Pubrica: When you order our services, we promise you the following u2013 Plagiarism free, always on Time, outstanding customer support, written to Standard, Unlimited Revisions support and High-quality Subject Matter Experts. Learn More: https://bit.ly/38jAbCT Contact us: Web: https://pubrica.com/ Blog: https://pubrica.com/academy/ Email: [email protected] WhatsApp : 91 9884350006 United Kingdom : 44-1143520021

123 views • 10 slides

SlidePlayer

  • My presentations

Auth with social network:

Download presentation

We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!

Presentation is loading. Please wait.

Systematic Literature Review: Step by step process

Published by Adeel Tariq Modified over 3 years ago

Similar presentations

Presentation on theme: "Systematic Literature Review: Step by step process"— Presentation transcript:

Systematic Literature Review: Step by step process

Critical Reading Strategies: Overview of Research Process

systematic literature review methodology ppt

Protocol Development.

systematic literature review methodology ppt

Student Learning Development, TCD1 Systematic Approaches to Literature Reviewing Dr Tamara O’Connor Student Learning Development Trinity College Dublin.

systematic literature review methodology ppt

Systematic Approaches to Literature Reviewing

systematic literature review methodology ppt

Mapping Studies – Why and How Andy Burn. Resources The idea of employing evidence-based practices in software engineering was proposed in (Kitchenham.

systematic literature review methodology ppt

8. Evidence-based management Step 3: Critical appraisal of studies

systematic literature review methodology ppt

Systematic Reviews & Meta-Analysis

systematic literature review methodology ppt

Systematic Approaches to Literature Reviewing Dr. Mark Matthews Student Learning Development Systematic Approaches to Literature Reviews.

systematic literature review methodology ppt

1 The Literature Review March 2007 (3). 2 The Literature Review The review of the literature is defined as a broad, comprehensive, in- depth, systematic,

systematic literature review methodology ppt

How to Critically Review an Article

systematic literature review methodology ppt

Systematic Approaches to Literature Reviewing Presenter: Caroline Forsyth Adapted from presentation by Dr. Derek Richards & Dr Mark Mathews.

systematic literature review methodology ppt

WRITING A REVIEW ARTICLE STRUCTURE AND STYLE OF A REVIEW ARTICLE Saleem Saaed Qader MBChB, MD, MSc, MPH, PhD, SBGS Consultant General Surgeon, Lecturer.

systematic literature review methodology ppt

Reading Scientific Papers Shimae Soheilipour

systematic literature review methodology ppt

EMPRICAL RESEARCH REPORTS

systematic literature review methodology ppt

Systematic Approaches to Literature Reviewing. The Literature Review ? “Literature reviews …… introduce a topic, summarise the main issues and provide.

systematic literature review methodology ppt

Literature Review Evaluating Existing Research

systematic literature review methodology ppt

Systematic Approaches to Literature Reviewing Dr. Derek Richards derek.richards [at] tcd.ie.

systematic literature review methodology ppt

Systematic Reviews.

systematic literature review methodology ppt

Literature review Osama A Samarkandi, PhD, RN BSc, GMD, BSN, MSN, NIAC EMS 423; EMS Research and Evidence Based Practice.

About project

© 2024 SlidePlayer.com Inc. All rights reserved.

IMAGES

  1. PPT

    systematic literature review methodology ppt

  2. Systematic Literature Review Methodology

    systematic literature review methodology ppt

  3. PPT

    systematic literature review methodology ppt

  4. The Systematic Review Process

    systematic literature review methodology ppt

  5. PPT

    systematic literature review methodology ppt

  6. PPT

    systematic literature review methodology ppt

VIDEO

  1. Systematic literature review

  2. Research Methods: Writing a Literature Review

  3. SYSTEMATIC AND LITERATURE REVIEWS

  4. What is Literature Review?

  5. Part 03: Literature Review (Research Methods and Methodology) By Dr. Walter

  6. Systematic Literature Review, by Prof. Ranjit Singh, IIIT Allahabad

COMMENTS

  1. PDF Introduction to Systematic Review and Meta-Analysis:

    PowerPoint Presentation Introduction to Systematic Review and Meta-Analysis: A Health Care Perspective Sally C. Morton Department of Biostatistics University of Pittsburgh Methods for Research Synthesis: A Cross-Disciplinary Approach, October 2013 Cross-Disciplinary Communication From the Healthcare Systematic Review World Terminology

  2. Workshop Handout & PPT

    Systematic Review Methods (SRM) SRM - General/What/Why. Workshop Handout & PPT ; SRM - Library research models; SRM - General steps; SRM - Checklists; SRM - Manage

  3. PowerPoint Presentation

    A systematic review is an appraisal and synthesis of primary research papers using a rigorous and clearly documented methodology in both the search strategy and the selection of studies. This minimises bias in the results. The clear documentation of the process and the decisions made allow the review to be reproduced and updated.

  4. PDF Systematic Literature Reviews: an Introduction

    Systematic literature reviews (SRs) are a way of synthesising scientific evidence to answer a particular research question in a way that is transparent and reproducible, while seeking to include all published evidence on the topic and appraising the quality of this evidence.

  5. Steps of a Systematic Review

    Steps of SR (visual) Steps of SR (PIECES) Image by TraceyChandler Steps to conducting a systematic review Quick overview of the process: Steps and resources from the UMB HSHSL Guide. YouTube video (26 min) Another detailed guide on how to conduct and write a systematic review from RMIT University

  6. PPT Boston University Medical Campus

    ÐÏ à¡± á> þÿ T X þÿÿÿ± ° ¯ Z ­ V U ...

  7. How-to conduct a systematic literature review: A quick guide for

    A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure . An SLR updates the reader with current literature about a subject .

  8. Training Modules for the Systematic Reviews Methods Guide

    The AHRQ Training Modules for the Systematic Reviews Methods Guide: An Introduction Presentation slides (PPT; 5.8 MB) Topic Refinement Presentation slides (PPT; 2.2 MB) Analytic Frameworks Presentation slides (PPT; 1.8 MB) Study Eligibility Criteria Presentation slides (PPT; 599 kB) Searching for Relevant Studies Presentation slides (PPT; 2.2 MB)

  9. How to Do a Systematic Review: A Best Practice Guide for ...

    Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of e …

  10. Guidance on Conducting a Systematic Literature Review

    This article is organized as follows: The next section presents the methodology adopted by this research, followed by a section that discusses the typology of literature reviews and provides empirical examples; the subsequent section summarizes the process of literature review; and the last section concludes the paper with suggestions on how to improve the quality and rigor of literature ...

  11. An overview of methodological approaches in systematic reviews

    1. INTRODUCTION. Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses (MA) when appropriate, is considered the "gold standard" of methods for synthesizing evidence related to a topic of interest. 2 The central strength of an SR is the transparency of the methods used to systematically search ...

  12. Easy guide to conducting a systematic review

    Systematic reviews are considered to provide the highest level of evidence on the hierarchy of evidence pyramid. Systematic reviews are conducted following rigorous research methodology. To minimise bias, systematic reviews utilise a predefined search strategy to identify and appraise all available published literature on a specific topic.

  13. Method for conducting systematic literature review and meta-analysis

    Method details: the six basic steps Protocol - SLR methodology step 1. The need for a research protocol for SLR is for the consideration of transparency, transferability, and replicability of the work, which are the characteristics that make a literature review systematic [12].This helps to minimize the bias by conducting exhaustive literature searches.

  14. Systematic Review

    A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review

  15. Chapter 9 Methods for Literature Reviews

    9.3. Types of Review Articles and Brief Illustrations. EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic.

  16. Systematic reviews: Brief overview of methods, limitations, and

    CONCLUSION. Siddaway 16 noted that, "The best reviews synthesize studies to draw broad theoretical conclusions about what the literature means, linking theory to evidence and evidence to theory" (p. 747). To that end, high quality systematic reviews are explicit, rigorous, and reproducible. It is these three criteria that should guide authors seeking to write a systematic review or editors ...

  17. Research Guides: Systematic Reviews: Types of Literature Reviews

    Rapid review. Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research. Completeness of searching determined by time constraints. Time-limited formal quality assessment. Typically narrative and tabular.

  18. Systematic Literature Review

    8. Systematic Literature Review • To investigate all available evidence that supports or refutes a particular "topic of interest". • To summarize the existing evidence concerning a treatment or technology, e.g. to summarize the empirical evidence of the benefits and limitations of a specific agile method.

  19. How to Undertake an Impactful Literature Review: Understanding Review

    Important aspects of a systematic literature review (SLR) include a structured method for conducting the study and significant transparency of the approaches used for summarizing the literature (Hiebl, 2023).The inspection of existing scientific literature is a valuable tool for (a) developing best practices and (b) resolving issues or controversies over a single study (Gupta et al., 2018).

  20. Systematic literature review: An introduction

    3. LEVELS OF LITERATURE REVIEW • Narrative review • Traditional expert review; usually subjective in nature • Systematic review • A review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research, and to collect and analyze data from the studies that are included in the review • Meta-analysis ...

  21. Systematic review and meta analysis

    1. Systematic review and Meta-Analysis 3. What are they? • A systematic review answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria. • A meta-analysis is the use of statistical methods to summaries the results of these studies 5.

  22. How to Undertake an Impactful Literature Review: Understanding Review

    The systematic literature review (SLR) is one of the important review methodologies which is increasingly becoming popular to synthesize literature in any discipline in general and management in particular. In this article, we explain the SLR methodology and provide guidelines for performing and documenting these studies.

  23. Research Methodology 5. Systematic Literature Review (SLR)

    Systematic Literature Review (SLR) PowerPoint Presentation - ID:8998457 Presentation 1 / 56 Download Presentation >> Research Methodology 5. Systematic Literature Review (SLR) Nov 09, 2019 2.24k likes | 3.64k Views Research Methodology 5. Systematic Literature Review (SLR).

  24. Systematic Literature Review: Step by step process

    Download presentation Presentation on theme: "Systematic Literature Review: Step by step process"— Presentation transcript: 1 Systematic Literature Review: Step by step process Adeel Tariq Ph.D. Candidate School of Management Asian Institute of Technology 2 Outline: Outline Why literature review? How to get started

  25. Promoting sustainable consumption among university students: a

    Purpose This study aims to identify and analyze research that promotes sustainable consumption among university students, understand what areas of consumption were addressed, how consumption was intended to become more sustainable and what results were reported. Design/methodology/approach A systematic literature review was conducted using the Scopus and Web of Science databases and the ...

  26. Systematic review

    4. 4 Literature review The general term for all attempts to synthesize the results and conclusions of two or more publications on a given topic. A review may or may not be systematic. 5. Narrative review Traditional expert review; usually subjective in nature Systematic review A review of a clearly formulated question that uses systematic and ...

  27. Exploring the role of social capital in flood risk reduction: Insights

    Flooding is a widespread and destructive natural hazard that can have serious consequences for communities. It is particularly dangerous for vulnerable populations and can threaten lives and property. The potential of social capital to mitigate flood risks has been discussed in disaster literature. This study aims to conduct a systematic literature review comprising bibliometric and thematic ...

  28. A systematic literature review of knowledge graph ...

    In this paper, a systematic literature review (SLR) has been conducted to comprehensively examine KG construction methodologies and their applications across five key domains in education. In each examined study, we highlight the specific KG functionalities, knowledge extraction techniques, knowledge base characteristics, resource requirements ...

  29. The Unified Theory of Acceptance and Use of Technology (UTAUT) in

    However, for the other four aspects—occupation and country/region of the study sample, data collection and analysis methods and tools, theoretical models used, and results of UTAUT internal hypothesis testing—the methodology used by Williams et al. (2015) in their UTAUT review in the general field was adopted. This methodological choice is ...