literature review in google scholar

Literature Review: Google Scholar

Sample Searches
Examples of Published Literature Reviews
Researching Your Topic
Subject Searching
Google Scholar
Track Your Work
Citation Managers This link opens in a new window
Citation Guides This link opens in a new window
Tips on Writing Your Literature Review This link opens in a new window
Research Help

Ask a Librarian

Chat with a Librarian

Lisle: (630) 829-6057 Mesa: (480) 878-7514 Toll Free: (877) 575-6050 Email: [email protected]

Book a Research Consultation Library Hours

Google Scholar Library Links

To see links to BenU Library subscription content in your Google Scholar search results:

Go to Google Scholar > Settings > Library Links
Search " Benedictine "
Check the boxes
Click Save and you're done!
Google Scholar Library Links Tutorial This tutorial will guide you step-by-step through the quick setup process.

Finding Academic Literature

8 Winning hacks to use Google Scholar for your research paper

<< Previous: Subject Searching
Next: Track Your Work >>
Last Updated: Apr 25, 2024 3:34 PM
URL: https://researchguides.ben.edu/lit-review

Kindlon Hall 5700 College Rd. Lisle, IL 60532 (630) 829-6050

Gillett Hall 225 E. Main St. Mesa, AZ 85201 (480) 878-7514

Literature Review & Research Skills Guide: Use Google Scholar

Introduction
What is a Literature Review?
Your research topic
Topic Analysis
Developing a Search Strategy
Preliminary Reading
Use Primo Search
Use Google Scholar
Use Journal Databases
Sage Research Methods
Managing your literature review
Evaluation & Critical Appraisal
Referencing

Using Google Scholar

Google Scholar allows you to locate resources such as articles, theses and books.

Unlike Primo Search, which is set to search the Library's holdings only, Google Scholar searches beyond Charles Sturt University Library and will include resources that are not available to you.

Set up Library Links to access the Library's online resources using these instructions .

Google Scholar

Google Scholar is a search engine for scholarly information. It makes available information records that its 'robots' find or that an author, university repository, or journal publisher have chosen to list. It is useful because it searches across many resources and returns many resource types, including journal articles and book chapters, though it is important to note that results are not always from academic-quality sources and it will return both results you do have access to via Charles Sturt Library, and those you do not.

The video to the right provides you with an overview of Google Scholar, including how to change the settings so that it shows you which search results are held in Charles Sturt Library's collection:

Citation Searching

You can see who has cited a resource from the results page in Google Scholar. Once you've located a source you like, you can view who has cited that resource by clicking on the 'Cited by x' link beneath the item record.

You will then see all of the resources that have cited the original item. Keep in mind that this does not include every resource that has ever cited the original item - it only includes those resources that are indexed in Google Scholar.

In the Google Scholar Search box at the top of the page, enter some keywords that you identified in your topic analysis

Use the left hand menu to limit by date.
Have you set your library links? Select the Find it at CSU links to access articles in our collection.
You may notice that there is a mix of topics, including health, education etc. Add more keywords to your search to narrow it down.

Search Google Scholar

Using Google effectively

Citation searching

<< Previous: Use Primo Search
Next: Use Journal Databases >>
Last Updated: Aug 1, 2024 11:34 AM
URL: https://libguides.csu.edu.au/education-research

Charles Sturt University is an Australian University, TEQSA Provider Identification: PRV12018. CRICOS Provider: 00005F.

+91-0000000000
[email protected]
Research Gate

Koshal Research Support

—Information | Knowledge | Wisdom—

How To Conduct A Literature Review Using Google Scholar Step By Step Guide

A free academic search engine known as Google Scholar (GS) is sometimes referred to as the academic counterpart of Google. Instead of searching all of the online content that has been indexed, it searches publisher repositories, academic databases, or scholarly websites.

Typically, a smaller portion of the pool gets searched by Google in this manner. Even though everything is done automatically, the majority of search results come from trustworthy academic sources. In contrast to highly moderated subscription-based academic databases like Scopus and Web of Science, Google is also less selective about what it includes in search results, therefore it is important to determine the authority of the sources linked through Google Scholar on your own.

When conducting thorough searches, such as systematic reviews, Google Scholar (and Web of Science ) shouldn’t be used as a stand-alone resource for locating evidence and finding: Title searches rather than full-text searches in Google Scholar get much more results for ancient literature.

When you are going to complete a literature review , it’s essential to collect research information from many domains and eras, and google scholar is helpful in that regard.

Now, there are a few different reasons why I prefer using Google Scholar while conducting a literature review study.

The first is that Google does a reasonably good job of discovering all of the relevant work that is available online and indexing journals, conference papers, and research databases.

The second is that Google provides us with a few useful tools that we may use to browse the literature and find new information as we search.

Tools for Automatic Referencing
Free Online Citation Generator Tools

In order to show you how to use Google Scholar, allow me to give you an example. Consider that our literature review is looking for information on social eye observation in robotics.

Step By Step Guide of using Google Scholar for Literature search

Step 1: Type “ Google Schola r” into the search bar and click and find the google scholar dashboard

How to Conduct a Literature Review Using Google Scholar Step by Step Guide

Note: The first thing to keep in mind is that you may type “google scholar” in the search field and use the same Google tactics as you would when conducting a regular Google search.

Step 2: Now you can search any topic of your interest on the search bar of google scholar For example, you can Search Fish farming by typing the word in the search bar and clicking the enter button.

Note: Add the word(s) in question to the search tab and press Enter to look for the exact phrase. And it’s clear that Google provides us with a lengthy list of results with the same outcomes. Additionally, it provides us with the paper’s title, the authors who wrote it (although it can be difficult to see on other platforms), where it was published, and the year (see above screenshot) So feel free to click on any of these sites immediately and find the articles details.

Despite these details, Google Scholar Dashboard will also indicate on the right-hand side of this page whether the article is directly available as a PDF (see above screenshot). If so, it will also let you know which website is hosting the PDF. This is due to the fact that PDFs aren’t always hosted by the journal or conference that published the work; instead, writers may upload the PDFs on their own personal pages, for instance.

If you want to find older works that may have been published a little while ago as well as more recent publications that really show the recent R&D in the field of the subject selected.

Step 3: Google scholar allows us to search for articles based on their publication date and years and if you wish to restrict our search, you can search articles starting in the year (as shown below ) 2022, we will only see paper those that have been released in the selected years and this aids in limiting the scope to the most recent work.

As shown above, we are obtaining all 2022 academic research papers (shown in the above screenshot) that have been published in that particular field, complete with source information If you wish to look for research within specific timeframes, you may also build a custom range of your search.

Now, if you read the references in a certain paper, you can locate pertinent articles that were released earlier than that paper. But google scholar offers us a convenient way to locate publications citation and cite scores (see inside the red circle in below screenshot)

Note: If you are enjoying a specific research article and also wanted to see whether there was any additional research that was interested in it and had cited the publication. To view a list of publications that have mentioned the article we are most interested in, click the “cited by” (as shown in the red circle of below screenshot) link and see results and use these papers for your literature review.

Step 4: Google Scholar also offers the ability to automatically generate references, and you can easily obtain several citation types or styles by simply checking the cite button and finding the results as displayed below screenshot.

As mentioned above, you can find references for citations in several citation formats. To use them, just copy and paste the references into your written document’s references area.

Citation Writing Pattern
How to Increase Your Citation

You can see that you may acquire the citations in a variety of different standardized formats by clicking the cite link that is located beneath each article as shown in the picture above.

In fact, if you read down to the bottom, you may also find the citation in BibTeX style (see in the red circle below screenshot) a text format that you can copy and paste directly into your bib file, which is particularly useful for students studying computer science field.

You can also use one more useful feature of Google Scholar is the ability to conduct a fresh Google search inside the articles that are quoting the article we’re most interested in by selecting “search within citing articles” from the drop-down menu. And by doing so, you can significantly hone in on your area of interest.

Step 5: Google scholar dashboard also provides us with the beneficial feature of occasionally being able to click on individual author names to access that author’s research profile.

As an illustration, by clicking on my name (Koshal Kumar) here, you can find a Google Scholar profile which I set up with a picture of myself and my affiliation and citation I received till date (see below red circle) and Google automatically indexes all of the articles online that are published by me.

In this way, you can find the literature you want, and google scholar is also automatically ordered by how frequently they have been cited, but you can manually sort by recency by clicking the year or by author name and titles of the paper. Additionally, it might occasionally be helpful to research recent works by a particular author while building your literature review, and finally, Google provides us with a convenient way to cite, allowing us to determine how to cite specific articles.

I do recommend checking the citation over, because oftentimes Google automatically generates it, and there may be some weirdness or errors in the way it has handled the title or the journal-title. So that was a whirlwind tour of how to use Google Scholar to do a literature review.

This is all about this article, and we sincerely hope that these steps and advice of using google scholar in the literature review will be useful to you as you are going to search for literature review online in your journey of research and you are familiar with the online literature review process. KressUp is an online learning platform that occasionally publishes new articles; stay connected to more updates

Please share and subscribe to our website so that it can help as many people as possible. You can also write to us at [email protected] for a free consultation if you’re looking for further E-content or research support.

If you find this article useful, don’t forget to share it!

What is A literature Review and Types of Literature Review
How to write A literature Review paper II Literature Review Paper structure
4 Easy Steps To Writing A Literature Review Paper
Important Purpose of Literature Review in Research

One thought on “How To Conduct A Literature Review Using Google Scholar Step By Step Guide”

Pingback: What Are Citation Metrics And Why Are They Important | Koshal Research Support

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed Google Scholar

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article Google Scholar

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article PubMed Google Scholar

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Research management

The grassroots organizations continuing the fight for Ukrainian science

Career Feature 11 SEP 24

How a struggling biotech company became a university ‘spin-in’

Career Q&A 10 SEP 24

The human costs of the research-assessment culture

Career Feature 09 SEP 24

Why I’m committed to breaking the bias in large language models

Career Guide 04 SEP 24

Binning out-of-date chemicals? Somebody think about the carbon!

Correspondence 27 AUG 24

Publishing nightmare: a researcher’s quest to keep his own work from being plagiarized

News 04 SEP 24

Intellectual property and data privacy: the hidden risks of AI

Faculty Position

The Institute of Cellular and Organismic Biology (ICOB), Academia Sinica, Taiwan, is seeking candidates to fill multiple tenure-track faculty position

Taipei (TW)

Institute of Cellular and Organismic Biology, Academia Sinica

Postdoctoral Associate

Houston, Texas (US)

Baylor College of Medicine (BCM)

Associate or Senior Editor, Nature Energy

Job Title: Associate or Senior Editor, Nature Energy Location: New York, Jersey City, Philadelphia or London — Hybrid Working Application Deadline:...

New York City, New York (US)

Springer Nature Ltd

Open Rank Tenure-track/Tenured Faculty Positions in the Department of Genetic and Genomic Sciences

Seeking expertise in areas including Cancer Genetics; Artificial Intelligence; Drug Development & Clinical Trials; Functional Genomics; & Gene Editing

Mount Sinai Department of Genetics and Genomic Sciences

Assistant Professor (Tenure Track) of Robotics

The Department of Mechanical and Process Engineering (D-MAVT, www.mavt.ethz.ch) at ETH Zurich invites applications for the above-mentioned position.

Zurich city

Quick links

Explore articles by subject
Guide to authors
Editorial policies

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base

Methodology

How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

Search for relevant literature
Evaluate sources
Identify themes, debates, and gaps
Outline the structure
Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

Quick Run-through
Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

Demonstrate your familiarity with the topic and its scholarly context
Develop a theoretical framework and methodology for your research
Position your work in relation to other researchers and theorists
Show how your research addresses a gap or contributes to a debate
Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
Body image, self-perception, self-esteem, mental health
Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

Your university’s library catalogue
Google Scholar
Project Muse (humanities and social sciences)
Medline (life sciences and biomedicine)
EconLit (economics)
Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

What question or problem is the author addressing?
What are the key concepts and how are they defined?
What are the key theories, models, and methods?
Does the research use established frameworks or take an innovative approach?
What are the results and conclusions of the study?
How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
Themes: what questions or concepts recur across the literature?
Debates, conflicts and contradictions: where do sources disagree?
Pivotal publications: are there any influential theories or studies that changed the direction of the field?
Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

Most research has focused on young women.
There is an increasing interest in the visual aspects of social media.
But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

Look at what results have emerged in qualitative versus quantitative research
Discuss how the topic has been approached by empirical versus theoretical scholarship
Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
Critically evaluate: mention the strengths and weaknesses of your sources
Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

Sampling methods
Simple random sampling
Stratified sampling
Cluster sampling
Likert scales
Reproducibility

Statistics

Null hypothesis
Statistical power
Probability distribution
Effect size
Poisson distribution

Research bias

Optimism bias
Cognitive bias
Implicit bias
Hawthorne effect
Anchoring bias
Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

To familiarize yourself with the current state of knowledge on your topic
To ensure that you’re not just repeating what others have already done
To identify gaps in knowledge and unresolved problems that your research can address
To develop your theoretical framework and methodology
To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other academic texts , with an introduction , a main body, and a conclusion .

An annotated bibliography is a list of source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a paper .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved September 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Literature Reviews and Annotated Bibliographies

Initial databases for a literature review.

What is a Literature Review?

How to Write a Literature Review?
Graduate Research and the Literature Review
What is an Annotated Bibliography?
How to Evaluate Sources?
Citation & Avoiding Plagiarism

The databases listed here are interdisciplinary and suitable for most disciplines. For databases specific to your discipline see our Research Guides

Academic Search Ultimate Includes some full text

A great place to start to search for magazine and journal articles on almost all topics. Tip : Check "peer reviewed" box to limit your search to scholarly journals.

Dissertations and Theses (1861+) Indexes dissertations accepted for doctoral degrees by accredited North American educational institutions and over 200 other institutions. Also covers masters theses since 1962. Starting in the early to mid-1900's, the full text is included for an increasingly comprehensive number of dissertations and theses.

Google Scholar Enables you to search specifically for scholarly literature, including peer-reviewed papers, theses, books, preprints, abstracts and technical reports from all broad areas of research. Use Google Scholar to find articles from a widevariety of academic publishers, professional societies, preprint repositories and universities, as well as scholarly articles available across the web

Humanities and Social Science Retrospective Bibliographic database that provides citations to articles in a wide range of English language journals in the humanities and social sciences for the period 1907-1984.

JSTOR Includes full text Includes long runs of backfiles of scholarly journals. Subjects covered include Anthropology, Asian Studies, Ecology, Economics, Education, Finance, History, Mathematics, Philosophy, Political Science, Population Studies, and Sociology.

Periodical Archives Online- (1770-1995) Includes full text; Full text archive of hundreds of periodicals in the humanities and social sciences from their first issues to 1995 Allows date-limited searching. Periodical Index Online, 1665 - 1995

"A literature review is an account of what has been published on a topic by accredited scholars and researchers. Occasionally you will be asked to write one as a separate assignment, ..., but more often it is part of the introduction to an essay, research report, or thesis. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries."

--Written by Dena Taylor, Health Sciences Writing Centre and available at http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review (Accessed August 8th, 2011)

Writing the Literature Review sites :

Literature Reviews: UNC - Chapel Hill

Write a Literature Review: UC-Santa Cruz

Writing a Literature Review: Perdue OWL

Methods Map: Literature Review

What are the goals of creating a Literature Review?

To develop a new theory
To evaluate a theory or theories
To survey what’s known about a topic
Identify a problem in a field of research
Provide a historical overview of the development of a topic

Type of Literature Reviews:

Mature and/or established topic: Topic is well-known and the purpose of this type of review is to analyze and synthesize this accumulated body of research.
Emerging Topic: The purpose of this type of review to identify understudy or new emerging research area.
<< Previous: Home
Next: How to Write a Literature Review? >>
Last updated: Jan 8, 2024 2:52 PM
URL: https://libguides.asu.edu/LiteratureReviews

The ASU Library acknowledges the twenty-three Native Nations that have inhabited this land for centuries. Arizona State University's four campuses are located in the Salt River Valley on ancestral territories of Indigenous peoples, including the Akimel O’odham (Pima) and Pee Posh (Maricopa) Indian Communities, whose care and keeping of these lands allows us to be here today. ASU Library acknowledges the sovereignty of these nations and seeks to foster an environment of success and possibility for Native American students and patrons. We are advocates for the incorporation of Indigenous knowledge systems and research methodologies within contemporary library practice. ASU Library welcomes members of the Akimel O’odham and Pee Posh, and all Native nations to the Library.

Repeatedly ranked #1 in innovation (ASU ahead of MIT and Stanford), sustainability (ASU ahead of Stanford and UC Berkeley), and global impact (ASU ahead of MIT and Penn State)

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
J Grad Med Educ
v.8(3); 2016 Jul

The Literature Review: A Foundation for High-Quality Medical Education Research

a These are subscription resources. Researchers should check with their librarian to determine their access rights.

Despite a surge in published scholarship in medical education 1 and rapid growth in journals that publish educational research, manuscript acceptance rates continue to fall. 2 Failure to conduct a thorough, accurate, and up-to-date literature review identifying an important problem and placing the study in context is consistently identified as one of the top reasons for rejection. 3 , 4 The purpose of this editorial is to provide a road map and practical recommendations for planning a literature review. By understanding the goals of a literature review and following a few basic processes, authors can enhance both the quality of their educational research and the likelihood of publication in the Journal of Graduate Medical Education ( JGME ) and in other journals.

The Literature Review Defined

In medical education, no organization has articulated a formal definition of a literature review for a research paper; thus, a literature review can take a number of forms. Depending on the type of article, target journal, and specific topic, these forms will vary in methodology, rigor, and depth. Several organizations have published guidelines for conducting an intensive literature search intended for formal systematic reviews, both broadly (eg, PRISMA) 5 and within medical education, 6 and there are excellent commentaries to guide authors of systematic reviews. 7 , 8

A literature review forms the basis for high-quality medical education research and helps maximize relevance, originality, generalizability, and impact.
A literature review provides context, informs methodology, maximizes innovation, avoids duplicative research, and ensures that professional standards are met.
Literature reviews take time, are iterative, and should continue throughout the research process.
Researchers should maximize the use of human resources (librarians, colleagues), search tools (databases/search engines), and existing literature (related articles).
Keeping organized is critical.

Such work is outside the scope of this article, which focuses on literature reviews to inform reports of original medical education research. We define such a literature review as a synthetic review and summary of what is known and unknown regarding the topic of a scholarly body of work, including the current work's place within the existing knowledge . While this type of literature review may not require the intensive search processes mandated by systematic reviews, it merits a thoughtful and rigorous approach.

Purpose and Importance of the Literature Review

An understanding of the current literature is critical for all phases of a research study. Lingard 9 recently invoked the “journal-as-conversation” metaphor as a way of understanding how one's research fits into the larger medical education conversation. As she described it: “Imagine yourself joining a conversation at a social event. After you hang about eavesdropping to get the drift of what's being said (the conversational equivalent of the literature review), you join the conversation with a contribution that signals your shared interest in the topic, your knowledge of what's already been said, and your intention.” 9

The literature review helps any researcher “join the conversation” by providing context, informing methodology, identifying innovation, minimizing duplicative research, and ensuring that professional standards are met. Understanding the current literature also promotes scholarship, as proposed by Boyer, 10 by contributing to 5 of the 6 standards by which scholarly work should be evaluated. 11 Specifically, the review helps the researcher (1) articulate clear goals, (2) show evidence of adequate preparation, (3) select appropriate methods, (4) communicate relevant results, and (5) engage in reflective critique.

Failure to conduct a high-quality literature review is associated with several problems identified in the medical education literature, including studies that are repetitive, not grounded in theory, methodologically weak, and fail to expand knowledge beyond a single setting. 12 Indeed, medical education scholars complain that many studies repeat work already published and contribute little new knowledge—a likely cause of which is failure to conduct a proper literature review. 3 , 4

Likewise, studies that lack theoretical grounding or a conceptual framework make study design and interpretation difficult. 13 When theory is used in medical education studies, it is often invoked at a superficial level. As Norman 14 noted, when theory is used appropriately, it helps articulate variables that might be linked together and why, and it allows the researcher to make hypotheses and define a study's context and scope. Ultimately, a proper literature review is a first critical step toward identifying relevant conceptual frameworks.

Another problem is that many medical education studies are methodologically weak. 12 Good research requires trained investigators who can articulate relevant research questions, operationally define variables of interest, and choose the best method for specific research questions. Conducting a proper literature review helps both novice and experienced researchers select rigorous research methodologies.

Finally, many studies in medical education are “one-offs,” that is, single studies undertaken because the opportunity presented itself locally. Such studies frequently are not oriented toward progressive knowledge building and generalization to other settings. A firm grasp of the literature can encourage a programmatic approach to research.

Approaching the Literature Review

Considering these issues, journals have a responsibility to demand from authors a thoughtful synthesis of their study's position within the field, and it is the authors' responsibility to provide such a synthesis, based on a literature review. The aforementioned purposes of the literature review mandate that the review occurs throughout all phases of a study, from conception and design, to implementation and analysis, to manuscript preparation and submission.

Planning the literature review requires understanding of journal requirements, which vary greatly by journal ( table 1 ). Authors are advised to take note of common problems with reporting results of the literature review. Table 2 lists the most common problems that we have encountered as authors, reviewers, and editors.

Sample of Journals' Author Instructions for Literature Reviews Conducted as Part of Original Research Article a

An external file that holds a picture, illustration, etc.
Object name is i1949-8357-8-3-297-t01.jpg

Common Problem Areas for Reporting Literature Reviews in the Context of Scholarly Articles

An external file that holds a picture, illustration, etc.
Object name is i1949-8357-8-3-297-t02.jpg

Locating and Organizing the Literature

Three resources may facilitate identifying relevant literature: human resources, search tools, and related literature. As the process requires time, it is important to begin searching for literature early in the process (ie, the study design phase). Identifying and understanding relevant studies will increase the likelihood of designing a relevant, adaptable, generalizable, and novel study that is based on educational or learning theory and can maximize impact.

Human Resources

A medical librarian can help translate research interests into an effective search strategy, familiarize researchers with available information resources, provide information on organizing information, and introduce strategies for keeping current with emerging research. Often, librarians are also aware of research across their institutions and may be able to connect researchers with similar interests. Reaching out to colleagues for suggestions may help researchers quickly locate resources that would not otherwise be on their radar.

During this process, researchers will likely identify other researchers writing on aspects of their topic. Researchers should consider searching for the publications of these relevant researchers (see table 3 for search strategies). Additionally, institutional websites may include curriculum vitae of such relevant faculty with access to their entire publication record, including difficult to locate publications, such as book chapters, dissertations, and technical reports.

Strategies for Finding Related Researcher Publications in Databases and Search Engines

An external file that holds a picture, illustration, etc.
Object name is i1949-8357-8-3-297-t03.jpg

Search Tools and Related Literature

Researchers will locate the majority of needed information using databases and search engines. Excellent resources are available to guide researchers in the mechanics of literature searches. 15 , 16

Because medical education research draws on a variety of disciplines, researchers should include search tools with coverage beyond medicine (eg, psychology, nursing, education, and anthropology) and that cover several publication types, such as reports, standards, conference abstracts, and book chapters (see the box for several information resources). Many search tools include options for viewing citations of selected articles. Examining cited references provides additional articles for review and a sense of the influence of the selected article on its field.

Box Information Resources

Web of Science a
Education Resource Information Center (ERIC)
Cumulative Index of Nursing & Allied Health (CINAHL) a
Google Scholar

Once relevant articles are located, it is useful to mine those articles for additional citations. One strategy is to examine references of key articles, especially review articles, for relevant citations.

Getting Organized

As the aforementioned resources will likely provide a tremendous amount of information, organization is crucial. Researchers should determine which details are most important to their study (eg, participants, setting, methods, and outcomes) and generate a strategy for keeping those details organized and accessible. Increasingly, researchers utilize digital tools, such as Evernote, to capture such information, which enables accessibility across digital workspaces and search capabilities. Use of citation managers can also be helpful as they store citations and, in some cases, can generate bibliographies ( table 4 ).

Citation Managers

An external file that holds a picture, illustration, etc.
Object name is i1949-8357-8-3-297-t04.jpg

Knowing When to Say When

Researchers often ask how to know when they have located enough citations. Unfortunately, there is no magic or ideal number of citations to collect. One strategy for checking coverage of the literature is to inspect references of relevant articles. As researchers review references they will start noticing a repetition of the same articles with few new articles appearing. This can indicate that the researcher has covered the literature base on a particular topic.

Putting It All Together

In preparing to write a research paper, it is important to consider which citations to include and how they will inform the introduction and discussion sections. The “Instructions to Authors” for the targeted journal will often provide guidance on structuring the literature review (or introduction) and the number of total citations permitted for each article category. Reviewing articles of similar type published in the targeted journal can also provide guidance regarding structure and average lengths of the introduction and discussion sections.

When selecting references for the introduction consider those that illustrate core background theoretical and methodological concepts, as well as recent relevant studies. The introduction should be brief and present references not as a laundry list or narrative of available literature, but rather as a synthesized summary to provide context for the current study and to identify the gap in the literature that the study intends to fill. For the discussion, citations should be thoughtfully selected to compare and contrast the present study's findings with the current literature and to indicate how the present study moves the field forward.

To facilitate writing a literature review, journals are increasingly providing helpful features to guide authors. For example, the resources available through JGME include several articles on writing. 17 The journal Perspectives on Medical Education recently launched “The Writer's Craft,” which is intended to help medical educators improve their writing. Additionally, many institutions have writing centers that provide web-based materials on writing a literature review, and some even have writing coaches.

The literature review is a vital part of medical education research and should occur throughout the research process to help researchers design a strong study and effectively communicate study results and importance. To achieve these goals, researchers are advised to plan and execute the literature review carefully. The guidance in this editorial provides considerations and recommendations that may improve the quality of literature reviews.

Health Professions Education for Doctoral Students: Reviewing the Literature

Articles on Your topic
Reviewing the Literature
Writing Your Thesis
Writing & Citing
Publishing Your Research

What is a Literature Review?

“A review of the literature consists of reading, analyzing, and writing a synthesis of scholarly materials about a specific topic. When the review is of scientific literature, the focus is on the hypotheses, the scientific methods, the strengths and weaknesses of the study, the results, and the authors’ interpretations and conclusions. A review of the scientific literature is fundamental to understanding the accumulated knowledge about the topic being reviewed."

Garrard, J. (2017). Health sciences literature review made easy: The matrix method . Burlington, MA: Jones and Bartlett Learning. p. 4.

Search Strategies

For a review of the literature, you want to conduct a comprehensive search to identify as many sources as possible on your topic. Be sure to take notes on your search process, otherwise it's easy to lose track of what you've already tried.

Database Searching

Search health sciences and other disciplinary databases (i.e. CINAHL, MEDLINE, ERIC) for keywords related to your topic. (See the Articles on Your Topic page of this guide for tips and resources).

Having trouble? Make an appointment with a librarian to discuss databases, keywords, and search strategies.

Citation Searching

When you find a relevant article, use it to identify other potentially relevant articles:

Look at its references to identify other relevant articles that you haven't found.

Use Google Scholar or Scopus to identify more recent articles that have cited that one.

Structure of a Literature Review

When you're writing a literature review, you won't just summarize all of the articles that you've found on your topic. Instead, you'll synthesize the evidence. This means that you'll look at the state of the evidence as a whole, telling your reader what it means when you put it all together. Rather than reviewing your sources one at a time, you'll organize your lit review by concept, chronology, methodology, etc. -- whatever makes the most sense for your topic.

example of a lit review that goes source-by-source

The Matrix Method

What is the Matrix Method?

The Matrix Method is an organizational system that guides you through the literature review process from start to finish, outlined in the book Health Sciences Literature Review Made Easy by Judith Garrard (see citation below). It is “intended to teach you how to read a paper, organize the review using the Matrix Method, analyze it, and create a narrative synthesis” (p. 31).

Matrix Method Handout This handout provides a general introduction to the Matrix Method. Please refer to Garrard’s book for further instructions on using the Matrix Method.

How can I access Garrard’s book?

Garrard, J. (2017). Health sciences literature review made easy: The matrix method . Burlington, MA: Jones and Bartlett Learning.

<< Previous: Articles on Your topic
Next: Writing Your Thesis >>
Last Updated: Jul 8, 2024 10:59 AM
URL: https://simmons.libguides.com/hped

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching

Affiliations.

1 MISTRA EviEM, Royal Swedish Academy of Sciences, Stockholm, Sweden.
2 Centre for Environmental Policy, Imperial College, London, United Kingdom; Department for Environmental, Food and Rural Affairs, London, United Kingdom.
3 Department for Environmental, Food and Rural Affairs, London, United Kingdom; Department for Civil and Environmental Engineering, Imperial College, London, United Kingdom.
4 Department for Environmental, Food and Rural Affairs, London, United Kingdom; Environment Agency, London, United Kingdom.
PMID: 26379270
PMCID: PMC4574933
DOI: 10.1371/journal.pone.0138237

Google Scholar (GS), a commonly used web-based academic search engine, catalogues between 2 and 100 million records of both academic and grey literature (articles not formally published by commercial academic publishers). Google Scholar collates results from across the internet and is free to use. As a result it has received considerable attention as a method for searching for literature, particularly in searches for grey literature, as required by systematic reviews. The reliance on GS as a standalone resource has been greatly debated, however, and its efficacy in grey literature searching has not yet been investigated. Using systematic review case studies from environmental science, we investigated the utility of GS in systematic reviews and in searches for grey literature. Our findings show that GS results contain moderate amounts of grey literature, with the majority found on average at page 80. We also found that, when searched for specifically, the majority of literature identified using Web of Science was also found using GS. However, our findings showed moderate/poor overlap in results when similar search strings were used in Web of Science and GS (10-67%), and that GS missed some important literature in five of six case studies. Furthermore, a general GS search failed to find any grey literature from a case study that involved manual searching of organisations' websites. If used in systematic reviews for grey literature, we recommend that searches of article titles focus on the first 200 to 300 results. We conclude that whilst Google Scholar can find much grey literature and specific, known studies, it should not be used alone for systematic review searches. Rather, it forms a powerful addition to other traditional search methods. In addition, we advocate the use of tools to transparently document and catalogue GS search results to maintain high levels of transparency and the ability to be updated, critical to systematic reviews.

PubMed Disclaimer

Conflict of interest statement

Competing Interests: The authors have declared that no competing interests exist.

Fig 1. Proportion of total a) full…

Fig 1. Proportion of total a) full text and b) title Google Scholar search results…

Publication types

Search in MeSH

Related information

Grants and funding, linkout - more resources, full text sources.

Europe PubMed Central
PubMed Central
Public Library of Science
Spiral, Imperial College Digital Repository

Other Literature Sources

scite Smart Citations

Research Materials

NCI CPTC Antibody Characterization Program

Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Start your free trial

Arrange a trial for your organisation and discover why FSTA is the leading database for reliable research on the sciences of food and health.

REQUEST A FREE TRIAL

Research Skills Blog

Google Scholar: a review of literature examining its effectiveness as a search tool

By Dr Helena Korjonen on 08-Nov-2021 12:15:10

By Helena Korjonen, PhD Postdoctoral Researcher, Sustainable Food Practices, University of Luxembourg

Google Scholar (GS) is the top search engine used by those who are looking for scholarly content 1 . There are many reasons for this, not least that it feels familiar 2 , 3 . However, a review of the literature reveals that there are significant limitations to its effectiveness.

Lack of curation

The i nternet is full of ‘scholarly’ content, information, and data in various formats and of various quality. As John Naisbitt said , “We are drowning in information , but starv ed for knowledge”

When Google Scholar trawls the i nternet to seek out scholarly content, it does not assess it. It does not try to make sense of it for researchers in a logical indexed structure. And, unlike many trusted curated tools 4 , Google Scholar does not have transparent indexing guideline s that defin e what they collect, and why they collect it and make it available.

Recent research shows that Google Scholar is “unsuitable as a primary review for resources” 5 . Other research argues that Google Scholar has flaws and lacks technological features to enable systematic searching. As a result, it fails to adhere to several FAIR principles (Findable, Accessible, Interoperable and Reusable) and cannot be classed as a professional searching tool 6 .

Google Scholar is not a curated controlled database. Content is harvested automatically.

Whilst there are some advantages to this, for example in the retrieval of grey literature 7 , there are also significant disadvantages.

Search results rankings

Research has shown that Google Scholar results can be dated. This presents an issue when most people don’t go beyond the first page of results when searching for papers.

The citation count of an article is a major factor in the Google Scholar results ranking 8 , 9 . This benefits publications from high Impact Factor sources, which are cited or linked to extensively around the Internet.

Unfortunately, new papers have what’s called a citation lag, and older papers are cited more often 10 . Newly published papers may be retrieved by GS, but they will be further down the results list and you may not see them.

P aper s can also be manipulated into a higher rank than it ‘deserves’ by placing words in titles or abstracts to increase recall or manipulate citing data 3 .

Should you need to include multi-lingual items in your search, there is a definite bias in Google Scholar towards English-language content. Documents published in languages other than English are relegated to positions that make them virtually invisible 8 .

Replicating your search

It is nearly impossible to replicate a search in GS, to retrieve the same references again. For this reason, it’s recommended to carefully document a search done in Google Scholar for full transparency 6 . That can be extremely time consuming.

References are often dropped unknowingly from GS 5 , 11 , 12 and you won’t even know the ones that might be missing 1 .

Using your results

In Google Scholar, once you have the results, it does not allow you to see beyond 1,000 records or to download more than 20 references to your bibliographic tool at a time. Most curated databases allow you to run your search, save it, download the results to your tool and revisit it.

How to search more effectively

IFIS has published a guide called Best Practice for Literature Searching . This provides guidance on where to search; how to formulate research questions and evaluate results. Click here to access the guide

(Image Credit: Kaitlyn Baker and Solen Feyissa at Unsplash )

Do you know which search tool to use and when? It is a common mistake to use search tools designed for information access for the task of information discovery, but doing this means you are doing your searching backwards! Find out more in our Guide for Effective Literature Searching

Gardner T, Inger S. How readers discover content: Trends in reader behaviour from 2005 to 2021. Renew Consultants. 2021; 17. Available from: https://renewconsultants.com/wp-content/uploads/2021/07/How-Readers-Discover-Content-2021.pdf
Hemminger BM , Lu D , Vaughan KTL , Adams SJ . Information seeking behavior of academic scientists . Journal of the American Society for Information Science and Technology . 2007 ; 58(14): 2205 - 2225. Available from: https://doi.org/10.1002/asi.20686
Georgas H . Google vs. the library: Student preferences and perceptions when doing research using Google and a federated search tool . portal: Libraries and the Academy . 2013 ; 13(2): 165 - 185.Available from: https://doi.org/10.1353/pla.2013.0011
Halevi G, Moed H, Bar-Ilan J. "Suitability of Google Scholar as a source of scientific information and as a source of data for scientific evaluation—Review of the literature. ." Journal of Informetrics. 2017; 11(3): 823-834. Available from: https://doi.org/10.1016/j.joi.2017.06.005
Gusenbauer M, Haddaway NR. "Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources." Research Synthesis Methods 2020; 11(2): 181-217. Available from: https://doi.org/10.1002/jrsm.1378
Boeker M, Vach W, Motschall E. Google Scholar as replacement for systematic literature searches: Good relative recall and precision are not enough. BMC Medical Research Methodology. 2013; 13(1):1-2. Available from: https://doi.org/10.1186/1471-2288-13-131
Haddaway NR, Collins AM, Coughlin D, Kirk S. The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PloS One. 2015; 17;10(9): e0138237. Available from: https://doi.org/10.1371/journal.pone.0138237
Beel J, Gipp B. Google Scholar's ranking algorithm: The impact of citation counts (an empirical study). Proceedings of the 2009 3rd International Conference on Research Challenges in Information Science, RCIS 2009; 439–446. Available from: http://doi.org/10.1109/RCIS.2009.5089308
Rovira C, Codina L, Guerrero-Solé F, Lopezosa C. Ranking by relevance and citation counts, a comparative study: Google Scholar, Microsoft Academic, WoS and Scopus. Future Internet. 2019 Sep;11(9):202. Available from: https://doi.org/10.3390/fi11090202
Khare R, Leaman R, Lu Z. Accessing biomedical literature in the current information landscape. Biomedical Literature Mining. 2014; 11-31. Available from: https://doi.org/10.1007/978-1-4939-0709-0_2
Nicholas D, Boukacem‐Zeghmouri C, Rodríguez‐Bravo B, Xu J, Watkinson A, Abrizah A, Herman E, Świgoń M. Where and how early career researchers find scholarly information. Learned Publishing. 2017; 30(1): 19-29. Available from: https://doi.org/10.1002/leap.1087 .
Jamali HR, Asadi S. Google and the scholar: The role of Google in scientists' information‐seeking behaviour. Online Information Review. 2010 Apr 20.. Available from: https://doi.org/10.1108/14684521011036990 .

FSTA - Food Science & Technology Abstracts
IFIS Collections
Resources Hub
Diversity Statement
Sustainability Commitment
Company news
Frequently Asked Questions
Privacy Policy
Terms of Use for IFIS Collections

Ground Floor, 115 Wharfedale Road, Winnersh Triangle, Wokingham, Berkshire RG41 5RB

Get in touch with IFIS

Systematic review
Open access
Published: 11 September 2024

Evaluation of research co-design in health: a systematic overview of reviews and development of a framework

Sanne Peters ORCID: orcid.org/0000-0001-6235-1752 1 ,
Lisa Guccione 2 , 3 ,
Jill Francis 1 , 2 , 3 , 4 ,
Stephanie Best 1 , 2 , 3 , 5 ,
Emma Tavender 6 , 7 ,
Janet Curran 8 , 9 ,
Katie Davies 10 ,
Stephanie Rowe 1 , 8 ,
Victoria J. Palmer 11 &
Marlena Klaic 1

Implementation Science volume 19 , Article number: 63 ( 2024 ) Cite this article

13 Altmetric

Metrics details

Co-design with consumers and healthcare professionals is widely used in applied health research. While this approach appears to be ethically the right thing to do, a rigorous evaluation of its process and impact is frequently missing. Evaluation of research co-design is important to identify areas of improvement in the methods and processes, as well as to determine whether research co-design leads to better outcomes. We aimed to build on current literature to develop a framework to assist researchers with the evaluation of co-design processes and impacts.

A multifaceted, iterative approach, including three steps, was undertaken to develop a Co-design Evaluation Framework: 1) A systematic overview of reviews; 2) Stakeholder panel meetings to discuss and debate findings from the overview of reviews and 3) Consensus meeting with stakeholder panel. The systematic overview of reviews included relevant papers published between 2000 and 2022. OVID (Medline, Embase, PsycINFO), EBSCOhost (Cinahl) and the Cochrane Database of Systematic reviews were searched for papers that reported co-design evaluation or outcomes in health research. Extracted data was inductively analysed and evaluation themes were identified. Review findings were presented to a stakeholder panel, including consumers, healthcare professionals and researchers, to interpret and critique. A consensus meeting, including a nominal group technique, was applied to agree upon the Co-design Evaluation Framework.

A total of 51 reviews were included in the systematic overview of reviews. Fifteen evaluation themes were identified and grouped into the following seven clusters: People (within co-design group), group processes, research processes, co-design context, people (outside co-design group), system and sustainment. If evaluation methods were mentioned, they mainly included qualitative data, informal consumer feedback and researchers’ reflections. The Co-Design Evaluation Framework used a tree metaphor to represent the processes and people in the co-design group (below-ground), underpinning system- and people-level outcomes beyond the co-design group (above-ground). To evaluate research co-design, researchers may wish to consider any or all components in the tree.

Conclusions

The Co-Design Evaluation Framework has been collaboratively developed with various stakeholders to be used prospectively (planning for evaluation), concurrently (making adjustments during the co-design process) and retrospectively (reviewing past co-design efforts to inform future activities).

Peer Review reports

Contributions to the literature

While stakeholder engagement in research seems ethically the right thing to do, a rigorous evaluation of its process and outcomes is frequently missing.

Fifteen evaluation themes were identified in the literature, of which research process , cognitive and emotional factors were the most frequently reported.

The Co-design Evaluation Framework can assist researchers with research co-design evaluation and provide guidance regarding what and when to evaluate.

The framework can be used prospectively, concurrently, and retrospectively to make improvements to existing and future research co-design projects.

Introduction

Lots of money is wasted in health research that does not lead to meaningful benefits for end-users, such as healthcare professionals and consumers [ 1 , 2 , 3 ]. One contributor to this waste is that research often focusses on questions and outcomes that are of limited importance to end-users [ 4 , 5 ]. Engaging relevant people in research co-design has increased in order to respond to this issue. There is a lack of consensus in the literature on the definition and processes involved in undertaking a co-design approach. For the purposes of this review, we define research co-design as meaningful end-user engagement that occurs across any stage of the research process , from the research planning phase to dissemination of research findings [ 6 ]. Meaningful end-user engagement refers to an explicit and measurable responsibility, such as contributing to writing a study proposal [ 6 ]. The variety of research co-design methods can be seen as a continuum ranging from limited involvement, such as consulting with end-users, to the much higher effort research approaches in which end-users and researchers aim for equal decision-making power and responsibility across the entire research process [ 6 ]. Irrespective of the intensity of involvement, it is generally recommended that a co-design approach should be based on several important principles such as equity, inclusion and shared ownership [ 7 ].

Over time, increasing attention has been given to research co-design [ 6 , 8 ]. Funding bodies encourage its use and it is recommended in the updated UK MRC framework on developing and evaluating complex interventions [ 9 ]. End-user engagement has an Equator reporting checklist [ 10 ] and related work has been reported by key organisations, such as the James Lind Alliance in the UK ( www.jla.nihr.ac.uk ), Patient Centered Outcomes Research Institute in the US ( www.pcori.org ) and Canadian Institutes of Health Research ( https://cihrirsc.gc.ca/e/41592.html ). In addition, peer reviewed publications involving co-design have risen from 173 per year in 2000 to 2617 in 2022 (PubMed), suggesting a growing importance in research activities.

Engaging end-users in the health research process is arguably the right thing to do, but the processes and outcomes of co-design have rarely been evaluated in a rigorous way [ 6 ]. Existing anecdotal evidence suggests that research co-design can benefit researchers, end-users and lead to more robust research processes [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ]. Both researchers and end-users have reported positive experiences of engaging in the co-design process. Potential benefits include a better understanding of community needs, more applicable research questions, designs and materials and improved trust between the researchers and end-users. Several reviews on conducting research co-design have concluded that co-design can be feasible, though predominantly used in the early phases of research, for example formulating research questions and developing a study protocol [ 6 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ]. However, these reviews highlighted that engagement of end-users in the research process required extra time and funding and had the risk of becoming tokenistic [ 6 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ].

The use of resources in co-design studies might need to be justified to the funder as well as its impacts. A rigorous evaluation of research co-design processes and outcomes is needed to identify areas of potential improvement and to determine the impact of research co-design. Several overviews of reviews on research co-design have been published but with no or limited focus on evaluation [ 20 , 21 , 22 , 23 ]. Moreover, current literature provides little guidance around how and what to evaluate, and which outcomes are key.

This study thus had two aims:

To conduct a systematic overview of reviews to identify evaluation methods and process and outcome variables reported in the published health research co-design literature.

To develop a framework to assist researchers with the evaluation of co-design processes and impacts.

This project used a multifaceted, iterative approach to develop a Co-design Evaluation Framework. It consisted of the following steps: 1) A systematic overview of reviews; 2) Stakeholder panel meetings to discuss and debate findings from the overview of reviews and 3) Consensus meeting with stakeholder panel. The reporting checklist for overviews of reviews was applied in Additional file 1 [ 24 ].

Step 1: A systematic overview of reviews

We conducted a systematic overview of reviews [ 25 ], reviewing literature reviews rather than primary studies, to investigate the following question: What is known in the published literature about the evaluation of research co-design in health research? The protocol of our systematic overview of reviews was published in the PROSPERO database (CRD42022355338).

Sub questions:

What has been co-designed and what were the objectives of the co-design process?

Who was involved and what was the level of involvement?

What methods were used to evaluate the co-design processes and outcomes?

What was evaluated (outcome and process measures) and at what timepoint (for example concurrently, or after, the co-design process)?

Was a co-design evaluation framework used to guide evaluation?

Search strategy

We searched OVID (Medline, Embase, PsycINFO), EBSCOhost (Cinahl) and the Cochrane Database of Systematic reviews on the 11th of October 2022 for literature reviews that reported co-design evaluation or outcomes in health research. The search strategy was based on previous reviews on co-design [ 6 , 14 , 26 ] and refined with the assistance of a research librarian and the research team (search terms in Additional file 2). Papers published from January 2000 to September 2022 were identified and retrieved by one author (SP).

Study selection

Database records were imported into EndNote X9 (The EndNote Team, Philadelphia, 2013) and duplicates removed. We managed the study selection process in the software program Covidence (Veritas Health Innovation, Melbourne, Australia). Two independent reviewers (SP, MK or LG) screened the titles and abstracts of all studies against the eligibility criteria (Table 1 ). Discrepancies were resolved through discussion or with a third reviewer (either SP, MK or LG, depending on which 2 reviewers disagreed). If there was insufficient information in the abstract to decide about eligibility, the paper was retained to the full-text screening phase. Full-text versions of studies not excluded at the title and abstract screening phase were retrieved and independently screened by two reviewers (SP, MK or LG) against eligibility criteria. Disagreements were resolved through discussion, or with a third reviewer, and recorded in Covidence.

Data extraction of included papers was conducted by one of three reviewers (SP, MK or LG). A second reviewer checked a random sample of 20% of all extracted data (LG or SP). Disagreements were resolved through regular discussion. Data were extracted using an excel spreadsheet developed by the research team and included review characteristics (such as references, type of review, number of included studies, review aim), details about the co-design process (such as who was involved in the co-design, which topics the co-design focused on, what research phase(s) the co-design covered, in which research phase the co-design took place and what the end-users’ level of involvement was) and details about the co-design evaluation (what outcomes were reported, methods of data collection, who the participants of the evaluation were, the timepoint of evaluation, whether an evaluation framework was used or developed and conclusions about co-design evaluation).

Types of end-users’ involvement were categorised into four groups based on the categories proposed by Hughes et al. (2018): 1. Targeted consultation; 2. Embedded consultation; 3. Collaboration and co-production and 4. User-led research, see Table 2 .

Data extraction and analysis took place in three iterative phases (Fig. 1 ), with each phase containing one third of the included studies. Each phase of data extraction and analysis was followed by stakeholder panel meetings (see step 2 below). This stepwise approach enabled a form of triangulation wherein themes that emerged through each phase were discussed with the stakeholder panel and incorporated both retrospectively (re-coding data in the prior phase) and prospectively (coding new data in the next phase).

Iterative phases in the process of the Co-design evaluation framework development

All reported outcomes of research co-design in the first phase (one third of all data) were inductively coded into themes, according to the principles of thematic analysis [ 28 ]. Two researchers (SP and MK) double coded 10% of all data and reached consensus through discussion. Given that consensus was high, one researcher (SP) continued the coding while having frequent discussions and reviews within the research team. In phase 2 (also one third of all data), deductive coding was based on the themes identified in the first round. Data of the first phase were re-coded, if new codes emerged during the stakeholder panel meeting. The same process took place for the third phase.

Step 2: Stakeholder panel meetings to discuss and debate findings from the overview of reviews

Results from step 1 were presented to the stakeholder panel to interpret and critique the review findings. The panel consisted of ten people, including a mix of consumers, healthcare professionals and researchers. Stakeholders were selected for their experience or expertise in research co-design. The number of meetings was not pre-determined, rather, it was informed by the outcomes from step 1. The number of stakeholders in each meeting ranged from six to ten.

A core group from the broader stakeholder panel (SP, MK, LG, JF) with a breadth of research experience and methodological expertise discussed the themes arising from both steps 1 and 2 and considered various ways of presenting them. Multiple design options were considered and preliminary frameworks were developed. Following discussion with the stakeholder panel, it was agreed that the evaluation themes could be grouped into several clusters to make the framework more comprehensible. The grouping of evaluation themes into clusters was informed by reported proposed associations between evaluation themes in the literature as well as the stakeholder panel’s co-design experience and expertise. Evaluation themes as well as clusters were agreed upon during the stakeholder panel meetings.

Step 3: Consensus meeting with stakeholder panel

The consensus meeting included the same stakeholder panel as in step 2. The meeting was informed by a modified Nominal Group Technique (NGT). The NGT is a structured process for obtaining information and reaching consensus with a target group who have some association or experience with the topic [ 29 ]. Various adaptations of the NGT have been used and additional pre-meeting information has been suggested to enable more time for participants to consider their contribution to the topic [ 30 ]. The modified NGT utilised in this study contained the following: (i) identification of group members to include experts with depth and diverse experiences. They were purposively identified at the start of this study for their expertise or experience in research co-design and included: a patient consumer, a clinician, three clinician researchers and six researchers with backgrounds in behavioural sciences, psychology, education, applied ethics and participatory design. All authors on this paper were invited by e-mail to attend an online meeting; (ii) provision of information prior to the group meeting included findings of the overview of reviews, a draft framework and objectives of the meeting. Five authors with extensive research co-design experience were asked to prepare a case example of one of their co-design projects for sharing at the group meeting. The intention of this exercise was to discuss the fit between a real-world example and the proposed framework; (iii) hybrid meeting facilitated by two researchers (SP & JF) who have experience in facilitating consensus meetings. Following presentation of the meeting materials, including the preliminary framework, group members were invited to silently consider the preliminary framework and generate ideas and critiques; iv) participants sharing their ideas and critiques; v) clarification process where group members shared their co-design example project and discussed the fit with components of the initial framework, and vi) silent voting and/or agreement on the framework via a personal email to one of the researchers (SP).

Step 1: Systematic overview of reviews

The database searches identified a total of 8912 papers. After removing 3016 duplicates and screening 5896 titles and abstracts, 148 full texts were sought for retrieval. Sixteen were not retrieved as they were not available in English ( n = 2) or full-text was not available ( n = 14). Of the remaining 132 papers assessed for eligibility, 81 were excluded. The final number of papers included in this overview of reviews was 51 (See Fig. 2 ).

PRISMA flow chart (based on [ 31 ]) of overview of reviews

Characteristics of the included studies

Of the 51 included reviews [ 11 , 12 , 14 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 ], 17 were systematic reviews, 12 were scoping reviews, 14 did not report the type or method of review, three were narrative reviews, two were qualitative evidence synthesis, another two were a structured literature search and one was a realist review. The number of studies included in the reviews ranged from 7 to 260. Nineteen reviews focused on co-design with specific populations, for example older people, people with intellectual disabilities, people living with dementia and 32 reviews included co-design with a range of end-users. The co-design focused in most cases on a mix of topics ( n = 31). Some reviews were specifically about one clinical topic, for example critical care or dementia. In ten cases, the clinical topics were not reported. Co-design took place during multiple research phases. Thirty-six reviews covered co-design in agenda/priority setting, 36 in study design, 30 in data collection, 25 in data analysis and 27 in dissemination. With regards to the research translation continuum, most of the co-design was reported in practice and community-based research ( n = 32), three reviews were conducted in basic research and 11 in human research. The types of end-users’ involvement in co-design ranged from targeted consultation ( n = 14) to embedded consultation ( n = 20), collaboration and co-production ( n = 14) to end-user- led research ( n = 6), including papers covering multiple types of involvement. Seventeen papers did not report the types of involvement. The reported co-design included a variety of time commitments, from a minimum of a one-off 60-min meeting to multiple meetings over multiple years. Twenty-seven reviews did not report details about the end-users’ types of involvement.

Identified evaluation themes

Fifteen evaluation themes were identified and were arranged into two higher level groups: 1. within the co-design team and 2. broader than co-design team (Table 3 ). The themes related to the first group (within the co-design team) included: Structure and composition of the co-design group, contextual enablers/barriers, interrelationships between group members, decision making process, emotional factors, cognitive factors, value proposition, level/ quality of engagement, research process, health outcomes for co-design group and sustainment of the co-design team or activities. The themes within the second group (broader than co-design team) included: Healthcare professional-level outcomes, healthcare system level outcomes, organisational level outcomes and patient and community outcomes.

The research process was the most frequently reported evaluation theme in the reviews ( n = 44, 86% of reviews), followed by cognitive factors ( n = 35, 69%) and emotional factors ( n = 34, 67%) (Table 4 ). Due to variability in reporting practices, it was not possible to specify the number of primary studies that reported specific evaluation themes. Evaluation methods for the themes were not reported in the majority of reviews ( n = 43, 84%). If evaluation methods were mentioned, they were mainly based on qualitative data, including interviews, focus groups, field notes, document reviews and observations (see overview with references in Additional file 3). Survey data was mentioned in three reviews. Many reviews reported informal evaluation based on participant experiences (e.g. informal feedback), reflection meetings, narrative reflections and authors’ hypotheses (Additional file 3). The timing of the evaluation was only mentioned in two papers: 1. Before and after the co-design activities and 2. Post co-design activities. One paper suggested that continuous evaluation might be helpful to improve the co-design process (Additional file 3).

The systematic overview of reviews found that some authors reported proposed positive associations between evaluation themes (Table 5 ). The most frequently reported proposed association was between level/quality of engagement and emotional factors ( n = 5, 10%). However, these proposed associations did not seem to have any empirical evidence and evaluation methods were not reported.

All evaluation themes were grouped into the following clusters (Table 6 ): People (within co-design group), group processes, research processes, co-design context, people (outside co-design group), system and sustainment.

Only one paper reported the evaluation in connection to the research phases (Agenda/priority setting, study design, data collection, data analysis and dissemination). This paper reported the following outcomes for the following research phases [ 58 ]:

Agenda/priority setting: Research process; Level/quality of engagement; Cognitive factors; Attributes of the co-design group; Interrelationships between group members; Sustainment of the co-design team or activities; Patient and community outcomes.

Study design: Attributes of the co-design group; Interrelationships between group members; Level/quality of engagement; Cognitive factors; Emotional factors; Research process.

The various research phases in which consumers could be involved, as well as the clusters of evaluation themes, informed the design of the co-design evaluation framework.

Two main options were voted on and discussed within the stakeholder panel. The two main options can be found in Additional file 4. Draft 2 was the prefered option as it was perceived as more dynamic than draft 1, representing a clearer interplay between the two contexts. The stakeholder panel suggested a few edits to the draft, such as the inclusion of bi-directional arrows in the tree trunk and a vertical arrow from underground to above ground with the label ‘impact’.

The final version of the Co-design Evaluation framework is presented in Fig. 3 .

Research Co-design evaluation framework

Figure 3 presents co-design evaluation as the below-ground and above-ground structures of a tree. The tree metaphor presents the processes and people in the co-design group (below-ground) as the basis for system- and people-level outcomes beyond the co-design group (above-ground). To evaluate research co-design, researchers may wish to consider any or all components in this Figure. These evaluation components relate to the methods, processes, and outcomes of consumer involvement in research.

The context within the co-design group (the roots of the tree) consists of the people, group processes and research processes, with various evaluation themes (dot points) related to them, as well as contextual barriers and enablers that relate to situational aspects that might enable or hinder consumer engagement. The context outside the co-design group, i.e., the wider community (the branches and leaves of the tree), comprises people who were not involved in the research co-design process, the system-level and sustainment-related outcomes. These above ground groups are potential beneficiaries or targets of the co-design activities.

The arrows in the middle of the trunk represent the potential mutual influence of the two contexts, suggesting that an iterative approach to evaluation might be beneficial. For example, when deciding the composition of the co-design group, it may be important to have an appropriate representation of the people most impacted by the problem issue or topic at hand. Or, if a co-designed healthcare intervention does not achieve the desired outcomes in the wider context, the co-design group might consider potential ways to improve the intervention or how it was delivered. Evaluation of a research co-design process might start with the foundations (the roots of the tree) and progress to above ground (the tree grows and might develop fruit). Yet, depending on the aim of the evaluation, a focus on one of the two contexts, either below or above ground, might be appropriate.

Which, and how many, components are appropriate to evaluate depends on the nature of the co-design approach and the key questions of the evaluation. For example, if a co-design approach is used in the very early stages of a research program, perhaps to identify priorities or to articulate a research question, then 'below' the ground components are key. While a randomised study comparing the effects of a co-designed intervention versus a researcher-designed intervention might only consider 'above' the ground components.

The white boxes on the right-hand side of Fig. 3 indicate the research phases, from agenda/priority setting to dissemination, in which consumers can and should be involved. This co-design evaluation framework may be applied at any phase of the research process or applied iteratively with a view to improving future co-design activities.

This systematic overview of reviews aimed to build on current literature and develop a framework to assist researchers with the evaluation of research co-design. Fifty-one included reviews reported on fifteen evaluation themes, which were grouped into the following clusters: People (within co-design group), group processes, research processes, co-design context, people (outside co-design group), system and sustainment. Most reviews did not report measurement methods for the evaluation themes. If methods were mentioned, they mostly included qualitative data, informal consumer feedback and researchers’ reflections. This finding strengthens our argument that a framework may be helpful in supporting methodologically robust studies to assess co-design processes and impacts. The Co-Design Evaluation Framework has adopted a tree metaphor. It presents the processes and people in the co-design group (below-ground) as the underpinning system- and people-level outcomes beyond the co-design group (above-ground). To evaluate stakeholder involvement in research, researchers may wish to consider any or all components in the tree. Which, and how many, components are appropriate to evaluate depends on the nature of the co-design approach and the key questions that stakeholders aim to address. Nonetheless, it will be important that evaluations delineate what parts of the research project have incorporated a co-design approach.

The Equator reporting checklist for Research Co-Design, GRIPP2, provides researchers with a series of concepts that should be considered and reported on when incorporating patient and public involvement in research [ 10 ]. These concepts include, but are not limited to, methods of involving patients and the public in research and intensity of engagement. The Co-Design Evaluation Framework is not intended as a replacement for the GRIPP2, rather, it can be used prospectively to inform development of the co-design project or retropsectively to inform completion of the GRIPP2. Table 7 provides hypothetical examples of research questions that co-design evaluation projects might address. The framework could be used at multiple points within co-design projects, including prospectively (planning for evaluation before the co-design process has started), concurrently ( incorporating improvements during the co-design process) and retrospectively (reviewing past co-design efforts to inform future projects).

Our systematic overview of reviews identified multiple evaluation themes. Some of these overlapped with reported values associated with public involvement in research [ 80 ], community engagement measures [ 15 ] and reported impacts of patient and public involvement in research, as described by others [ 16 , 81 , 82 ]. The added value of our systematic overview of reviews is that we went beyond a list of items and took it one step further by looking at evaluation themes, potential associations between evaluation themes, clusters of evaluation themes and ultimately developed a framework to assist others with research co-design evaluation.

Some reviews in our overview of reviews proposed potential associations between evaluation themes. Yet, these proposed associations were not empirically tested. One of the included studies [ 58 ] proposed conditions and mechanisms involved in co-design processes and outcomes related to diabetes research. Although it is a promising starting point, this should be further explored. A realist evaluation including other research topics and other approaches, such as the use of logic models, which was also recognised in the updated MRC framework [ 9 ], might help to build on explorations of included mechanisms of action [ 83 ] and give insight into how core ingredients contribute to certain co-design processes and outcomes. As recognised by others [ 6 , 84 ], the reporting practice of research co-design in the literature could be improved as details about context, mechanisms and expected outcomes are frequently missing. This will also help us to gain a better understanding of what works for whom, why, how and in which circumstances.

The lack of a consistent definition of co-design makes it challenging to identify and synthesise the literature, as recognised by others [ 6 ]. Given that there are so many different terms used in the literature, there is a risk that we might have missed some relevant papers in our overview of reviews. Nevertheless, we tried to capture as many as possible synonyms of co-design in our search terms. The absence of quality assessment of included studies in our overview of reviews can be seen as a limitation. However, our overview of reviews did not aim to assess existing literature on the co-design process, but rather focused on what to evaluate, how and when. We did note whether the reported evaluation themes were based on empirical evidence or authors’ opinions. Primary studies reported in the included reviews were not individually reviewed as this was outside the scope of this paper. A strength in our methods was the cyclical process undertaken between steps 1 and 2. Analysis of the data extracted from the overview was refined over three phases following rigorous discussions with a diverse and experienced stakeholder panel. It was a strength of our project that a mix of stakeholders were involved, including consumers, healthcare professionals and researchers.

Stakeholders are frequently engaged in research but if research co-design processes and outcomes are not evaluated, there will be limited learning from past experiences. Evaluation is essential to make refinements during existing projects and improve future co-design activities. It is also critical for ensuring commitments to the underpinning values of c-odesign are embedded within activities.

A systematic review of all primary studies within the included reviews of this overview of reviews, would allow greater depth relating to the practicalities of how to evaluate certain themes. It would lead to a better understanding of existing measures and methods and which evaluation areas need further development. Future research should also focus on whether co-design leads to better outcomes than no co-design (only researcher-driven research). To our knowledge, this has not been explored yet. Moreover, future research could gain better insight into the mechanisms of change within co-design and explore potential associations between evaluation themes for example, those proposed in the included reviews between level/quality of engagement and emotional factors.

We followed a systematic, iterative approach to develop a Co-Design Evaluation Framework that can be applied to various phases of the research co-design process. Testing of the utility of the framework is an important next step. We propose that the framework could be used at multiple points within co-design projects, including prospectively (planning for evaluation before the co-design process has started), concurrently (to incorporate improvements during the co-design process) and retrospectively (reviewing past co-design efforts to inform future projects).

Availability of data and materials

All data generated during this study are included either within the text or as a supplementary file.

Abbreviations

Medical Research Council

Guidance for Reporting Involvement of Patients and the Public

Chalmers I, Glasziou P. Systematic reviews and research waste. Lancet. 2016;387(10014):122–3.

Article PubMed Google Scholar

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76.

Ioannidis JP. Why Most Clinical Research Is Not Useful. PLoS Med. 2016;13(6):e1002049.

Article PubMed PubMed Central Google Scholar

Oliver S. Patient involvement in setting research agendas. Eur J Gastroenterol Hepatol. 2006;18(9):935–8.

Slattery P, Saeri AK, Bragge P. Research co-design in health: a rapid overview of reviews. Health Res Policy Syst. 2020;18(1):17.

Ní Shé É, Harrison R. Mitigating unintended consequences of co-design in health care. Health Expect. 2021;24(5):1551–6.

Peters S, Sukumar K, Blanchard S, Ramasamy A, Malinowski J, Ginex P, et al. Trends in guideline implementation: an updated scoping review. Implement Sci. 2022;17:50.

Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061.

Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017;358:j3453.

Article CAS PubMed PubMed Central Google Scholar

Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.

Manafo E, Petermann L, Mason-Lai P, Vandall-Walker V. Patient engagement in Canada: a scoping review of the “how” and “what” of patient engagement in health research. Health Res Policy Syst. 2018;16(1):5.

Fergusson D, Monfaredi Z, Pussegoda K, Garritty C, Lyddiatt A, Shea B, et al. The prevalence of patient engagement in published trials: a systematic review. Res Involv Engagem. 2018;4:17.

Vat LE, Finlay T, Jan Schuitmaker-Warnaar T, Fahy N, Robinson P, Boudes M, et al. Evaluating the “return on patient engagement initiatives” in medicines research and development: A literature review. Health Expect. 2020;23(1):5–18.

Luger TM, Hamilton AB, True G. Measuring Community-Engaged Research Contexts, Processes, and Outcomes: A Mapping Review. Milbank Q. 2020;98(2):493–553.

Modigh A, Sampaio F, Moberg L, Fredriksson M. The impact of patient and public involvement in health research versus healthcare: A scoping review of reviews. Health Policy. 2021;125(9):1208–21.

Clavel N, Paquette J, Dumez V, Del Grande C, Ghadiri DPS, Pomey MP, et al. Patient engagement in care: A scoping review of recently validated tools assessing patients’ and healthcare professionals’ preferences and experience. Health Expect. 2021;24(6):1924–35.

Newman B, Joseph K, Chauhan A, Seale H, Li J, Manias E, et al. Do patient engagement interventions work for all patients? A systematic review and realist synthesis of interventions to enhance patient safety. Health Expect. 2021;24:1905 No Pagination Specified.

Lowe D, Ryan R, Schonfeld L, Merner B, Walsh L, Graham-Wisener L, et al. Effects of consumers and health providers working in partnership on health services planning, delivery and evaluation. Cochrane Database Syst Rev. 2021;9:CD013373.

PubMed Google Scholar

Price A, Albarqouni L, Kirkpatrick J, Clarke M, Liew SM, Roberts N, et al. Patient and public involvement in the design of clinical trials: An overview of systematic reviews. J Eval Clin Pract. 2018;24(1):240–53.

Sarrami-Foroushani P, Travaglia J, Debono D, Braithwaite J. Implementing strategies in consumer and community engagement in health care: results of a large-scale, scoping meta-review. BMC Health Serv Res. 2014;14:402.

Abrams R, Park S, Wong G, Rastogi J, Boylan A-M, Tierney S, et al. Lost in reviews: Looking for the involvement of stakeholders, patients, public and other non-researcher contributors in realist reviews. Research Synthesis Methods. 2021;12(2):239–47.

Zych MM, Berta WB, Gagliardi AR. Conceptualising the initiation of researcher and research user partnerships: a meta-narrative review. Health Res Policy Syst. 2020;18(1):24.

Gates M, Gates A, Pieper D, Fernandes RM, Tricco AC, Moher D, et al. Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. BMJ. 2022;378:e070849.

Pollock A, Campbell P, Brunton G, Hunt H, Estcourt L. Selecting and implementing overview methods: implications from five exemplar overviews. Syst Rev. 2017;6(1):145.

Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: Systematic review and co-design pilot. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy. 2019;22(4):785–801.

Article Google Scholar

Hughes M, Duffy C. Public involvement in health and social sciences research: A concept analysis. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy. 2018;21(6):1183–90.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2008;3(2):77–101.

Waggoner J, Carline JD, Durning SJ. Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research. Acad Med. 2016;91(5):663–8.

Harvey N, Holmes CA. Nominal group technique: an effective method for obtaining group consensus. Int J Nurs Pract. 2012;18(2):188–94.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Baldwin JN, Napier S, Neville S, Clair VAWS. Impacts of older people’s patient and public involvement in health and social care research: a systematic review. Age Ageing. 2018;47(6):801–9.

Bench S, Eassom E, Poursanidou K. The nature and extent of service user involvement in critical care research and quality improvement: A scoping review of the literature. Int J Consum Stud. 2018;42(2):217–31.

Bethell J, Commisso E, Rostad HM, Puts M, Babineau J, Grinbergs-Saull A, et al. Patient engagement in research related to dementia: a scoping review. Dementia. 2018;17(8):944–75.

Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. The Patient: Patient-Centered Outcomes Research. 2014;7(4):387–95.

Di Lorito C, Birt L, Poland F, Csipke E, Gove D, Diaz-Ponce A, et al. A synthesis of the evidence on peer research with potentially vulnerable adults: how this relates to dementia. Int J Geriatr Psychiatry. 2017;32(1):58–67.

Di Lorito C, Bosco A, Birt L, Hassiotis A. Co-research with adults with intellectual disability: A systematic review. J Appl Res Intellect Disabil. 2018;31(5):669–86.

Fox G, Fergusson DA, Daham Z, Youssef M, Foster M, Poole E, et al. Patient engagement in preclinical laboratory research: A scoping review. EBioMedicine. 2021;70:103484.

Frankena TK, Naaldenberg J, Cardol M, Linehan C, van Schrojenstein Lantman-de Valk H. Active involvement of people with intellectual disabilities in health research - a structured literature review. Res Dev Disabil. 2015;45–46:271–83.

Fudge N, Wolfe CD, McKevitt C. Involving older people in health research. Age Ageing. 2007;36(5):492–500.

Article CAS PubMed Google Scholar

George AS, Mehra V, Scott K, Sriram V. Community participation in health systems research: A systematic review assessing the state of research, the nature of interventions involved and the features of engagement with communities. PLoS One. 2015;10(10):ArtID e0141091.

Legare F, Boivin A, van der Weijden T, Pakenham C, Burgers J, Legare J, et al. Patient and public involvement in clinical practice guidelines: A knowledge synthesis of existing programs. Med Decis Making. 2011;31(6):E45–74.

McCarron TL, Clement F, Rasiah J, Moran C, Moffat K, Gonzalez A, et al. Patients as partners in health research: a scoping review. Health Expect. 2021;24:1378 No Pagination Specified.

Miller J, Knott V, Wilson C, Roder D. A review of community engagement in cancer control studies among indigenous people of Australia, New Zealand, Canada and the USA. Eur J Cancer Care. 2012;21(3):283–95.

Article CAS Google Scholar

Shen S, Doyle-Thomas KAR, Beesley L, Karmali A, Williams L, Tanel N, et al. How and why should we engage parents as co-researchers in health research? A scoping review of current practices. Health Expect. 2017;20(4):543–54.

Velvin G, Hartman T, Bathen T. Patient involvement in rare diseases research: a scoping review of the literature and mixed method evaluation of Norwegian researchers’ experiences and perceptions. Orphanet J Rare Dis. 2022;17(1):212.

Wiles LK, Kay D, Luker JA, Worley A, Austin J, Ball A, et al. Consumer engagement in health care policy, research and services: A systematic review and meta-analysis of methods and effects. PLoS One. 2022;17:e0261808.no pagination

Cook N, Siddiqi N, Twiddy M, Kenyon R. Patient and public involvement in health research in low and middle-income countries: a systematic review. BMJ Open. 2019;9(5):e026514.

Chambers E, Gardiner C, Thompson J, Seymour J. Patient and carer involvement in palliative care research: An integrative qualitative evidence synthesis review. Palliat Med. 2019;33(8):969–84.

Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2014;17(5):637–50.

Boaz A, Hanney S, Jones T, Soper B. Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review. BMJ Open. 2015;5(12):e009415.

Boivin A, L’Esperance A, Gauvin FP, Dumez V, Macaulay AC, Lehoux P, et al. Patient and public engagement in research and health system decision making: A systematic review of evaluation tools. Health Expect. 2018;21(6):1075–84.

Anderst A, Conroy K, Fairbrother G, Hallam L, McPhail A, Taylor V. Engaging consumers in health research: a narrative review. Aust Health Rev. 2020;44(5):806–13.

Arnstein L, Wadsworth AC, Yamamoto BA, Stephens R, Sehmi K, Jones R, et al. Patient involvement in preparing health research peer-reviewed publications or results summaries: a systematic review and evidence-based recommendations. Res Involv Engagem. 2020;6:34.

Becerril-Montekio V, Garcia-Bello LA, Torres-Pereda P, Alcalde-Rabanal J, Reveiz L, Langlois EV. Collaboration between health system decision makers and professional researchers to coproduce knowledge, a scoping review. Int J Health Plann Manage. 2022;28:28.

Google Scholar

Bird M, Ouellette C, Whitmore C, Li L, Nair K, McGillion MH, et al. Preparing for patient partnership: A scoping review of patient partner engagement and evaluation in research. Health Expect. 2020;23:523 No Pagination Specified.

Dawson S, Campbell SM, Giles SJ, Morris RL, Cheraghi-Sohi S. Black and minority ethnic group involvement in health and social care research: A systematic review. Health Expect. 2018;21(1):3–22.

Harris J, Haltbakk J, Dunning T, Austrheim G, Kirkevold M, Johnson M, et al. How patient and community involvement in diabetes research influences health outcomes: A realist review. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy. 2019;22(5):907–20.

Hubbard G, Kidd L, Donaghy E. Involving people affected by cancer in research: a review of literature. Eur J Cancer Care. 2008;17(3):233–44.

Hubbard G, Kidd L, Donaghy E, McDonald C, Kearney N. A review of literature about involving people affected by cancer in research, policy and planning and practice. Patient Educ Couns. 2007;65(1):21–33.

Jones EL, Williams-Yesson BA, Hackett RC, Staniszewska SH, Evans D, Francis NK. Quality of reporting on patient and public involvement within surgical research: a systematic review. Ann Surg. 2015;261(2):243–50.

Drahota A, Meza RD, Brikho B, Naaf M, Estabillo JA, Gomez ED, et al. Community-Academic Partnerships: A Systematic Review of the State of the Literature and Recommendations for Future Research. Milbank Q. 2016;94(1):163–214.

Forsythe LP, Szydlowski V, Murad MH, Ip S, Wang Z, Elraiyah TA, et al. A systematic review of approaches for engaging patients for research on rare diseases. J Gen Intern Med. 2014;29(Suppl 3):788–800.

Article PubMed Central Google Scholar

Lander J, Hainz T, Hirschberg I, Strech D. Current practice of public involvement activities in biomedical research and innovation: a systematic qualitative review. PLoS One [Electronic Resource]. 2014;9(12):e113274.

Lee DJ, Avulova S, Conwill R, Barocas DA. Patient engagement in the design and execution of urologic oncology research. Urol Oncol. 2017;35(9):552–8.

Malterud K, Elvbakken KT. Patients participating as co-researchers in health research: A systematic review of outcomes and experiences. Scandinavian Journal of Public Health. 2020;48(6):617–28.

Miah J, Dawes P, Edwards S, Leroi I, Starling B, Parsons S. Patient and public involvement in dementia research in the European Union: a scoping review. BMC geriatr. 2019;19(1):220.

Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database Syst Rev. 2006;2006(3):CD004563.

PubMed PubMed Central Google Scholar

Oliver S, Clarke-Jones L, Rees R, Milne R, Buchanan P, Gabbay J, et al. Involving consumers in research and development agenda setting for the NHS: developing an evidence-based approach. Health Technol Assess. 2004;8(15):1-148 III-IV.

Orlowski SK, Lawn S, Venning A, Winsall M, Jones GM, Wyld K, et al. Participatory Research as One Piece of the Puzzle: A Systematic Review of Consumer Involvement in Design of Technology-Based Youth Mental Health and Well-Being Interventions. JMIR Hum Factors. 2015;2(2):e12.

Pii KH, Schou LH, Piil K, Jarden M. Current trends in patient and public involvement in cancer research: A systematic review. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy. 2019;22(1):3–20.

Sandoval JA, Lucero J, Oetzel J, Avila M, Belone L, Mau M, et al. Process and outcome constructs for evaluating community-based participatory research projects: a matrix of existing measures. Health Educ Res. 2012;27(4):680–90.

Sangill C, Buus N, Hybholt L, Berring LL. Service user’s actual involvement in mental health research practices: A scoping review. Int J Ment Health Nurs. 2019;28(4):798–815.

Schelven F, Boeije H, Marien V, Rademakers J. Patient and public involvement of young people with a chronic condition in projects in health and social care: A scoping review. Health Expect. 2020;23:789 No Pagination Specified.

Schilling I, Gerhardus A. Methods for Involving Older People in Health Research-A Review of the Literature. International Journal of Environmental Research & Public Health [Electronic Resource]. 2017;14(12):29.

Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66.

Vaughn LM, Jacquez F, Lindquist-Grantz R, Parsons A, Melink K. Immigrants as research partners: A review of immigrants in community-based participatory research (CBPR). J Immigr Minor Health. 2017;19(6):1457–68.

Walmsley J, Strnadova I, Johnson K. The added value of inclusive research. J Appl Res Intellect Disabil. 2018;31(5):751–9.

Weschke S, Franzen DL, Sierawska AK, Bonde LS, Strech D. Schorr SG. Reporting of patient involvement: A mixed-methods analysis of current practice in health research publications. medRxiv; 2022. p. 21.

Gradinger F, Britten N, Wyatt K, Froggatt K, Gibson A, Jacoby A, et al. Values associated with public involvement in health and social care research: A narrative review. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy. 2015;18(5):661–75.

Hoekstra F, Mrklas KJ, Khan M, McKay RC, Vis-Dunbar M, Sibley KM, et al. A review of reviews on principles, strategies, outcomes and impacts of research partnerships approaches: A first step in synthesising the research partnership literature. Health Res Pol Syst. 2020;18(1):51 no pagination.

Stallings SC, Boyer AP, Joosten YA, Novak LL, Richmond A, Vaughn YC, et al. A taxonomy of impacts on clinical and translational research from community stakeholder engagement. Health Expect. 2019;22(4):731–42.

Grindell C, Coates E, Croot L, O’Cathain A. The use of co-production, co-design and co-creation to mobilise knowledge in the management of health conditions: a systematic review. BMC Health Serv Res. 2022;22(1):877.

Staley K. “Is it worth doing?” Measuring the impact of patient and public involvement in research. Res Involv Engagem. 2015;1:6.

Download references

Acknowledgements

The authors would like to thank the graphic designers, Jenni Quinn and Kevin Calthorpe, for their work on Fig. 3 .

Not applicable.

Author information

Authors and affiliations.

School of Health Sciences, The University of Melbourne, Melbourne, Australia

Sanne Peters, Jill Francis, Stephanie Best, Stephanie Rowe & Marlena Klaic

Department of Health Services Research, Peter MacCallum Cancer Centre, Melbourne, Australia

Lisa Guccione, Jill Francis & Stephanie Best

Sir Peter MacCallum Department of Oncology, Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Melbourne, Australia

Centre for Implementation Research, Ottawa Hospital Research Institute, Ottawa, Canada

Jill Francis

Victorian Comprehensive Cancer Centre, Melbourne, VIC, Australia

Stephanie Best

Emergency Research, Murdoch Children’s Research Institute, Melbourne, Australia

Emma Tavender

Department of Critical Care, The University of Melbourne , Melbourne, Australia

School of Nursing, Faculty of Health, Ottawa, Canada

Janet Curran & Stephanie Rowe

Emergency Medicine, Faculty of Medicine, Ottawa, Canada

Janet Curran

Neurological Rehabilitation Group Mount Waverley, Mount Waverley, Australia

Katie Davies

The ALIVE National Centre for Mental Health Research Translation, The University of Melbourne, Melbourne, Australia

Victoria J. Palmer

You can also search for this author in PubMed Google Scholar

Contributions

SP coordinated the authorship team, completed the systematic literature searches, synthesis of data, framework design and substantial writing. MK and LG were the second reviewers for the systematic overview of reviews. MK, LG and JF assisted with framework design. SP, LG, JF, SB, ET, JC, KD, SR, VP and MK participated in the stakeholder meetings and the consensus process. All authors commented on drafts and approved the final submitted version of the manuscript.

Corresponding author

Correspondence to Sanne Peters .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., supplementary material 4., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Peters, S., Guccione, L., Francis, J. et al. Evaluation of research co-design in health: a systematic overview of reviews and development of a framework. Implementation Sci 19 , 63 (2024). https://doi.org/10.1186/s13012-024-01394-4

Download citation

Received : 01 April 2024

Accepted : 31 August 2024

Published : 11 September 2024

DOI : https://doi.org/10.1186/s13012-024-01394-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Research co-design
Stakeholder involvement
End-user engagement
Consumer participation
Outcome measures

Implementation Science

ISSN: 1748-5908

Submission enquiries: Access here and click Contact Us
General enquiries: [email protected]

Open access
Published: 11 September 2024

Too stressed to think? A scoping review of the literature for healthcare educators utilising high acuity clinical scenarios

Jason Betson ORCID: orcid.org/0000-0002-9883-8586 1 , 2 ,
Erich C. Fein ORCID: orcid.org/0000-0003-4119-0130 4 ,
David Long ORCID: orcid.org/0000-0002-6278-7377 2 &
Peter Horrocks ORCID: orcid.org/0000-0002-0220-175X 3

BMC Medical Education volume 24 , Article number: 990 ( 2024 ) Cite this article

1 Altmetric

Metrics details

The practise of paramedicine can be highly stressful particularly where urgent lifesaving decisions need to be made. Traditionally, educators have adopted the approach of placing students in simulated stressful situations as a way of learning to cope with these challenges. It is unclear from the literature whether traditional stress inoculation enhances or hinders learning. This scoping review aims to identify and examine both the peer-reviewed and grey literature reporting physiological stress responses to high-acuity scenarios in paramedicine and cognate healthcare disciplines.

Adhering strictly to JBI Evidence Synthesis Manual for conducting a scoping review, medical subject headings and areas, keywords and all other possible index terms were searched across EBSCOhost (Medline, CINAHL and APA PsycInfo), Scopus and, PubMed. English language articles both published (peer-reviewed academic papers, reports and conference proceedings) and unpublished (grey literature, Google Scholar reports) were included, and publications citing retrieved articles were also checked.

Searches performed across five electronic databases identified 52 articles where abstracts indicated potential inclusion. From this, 22 articles which reported physiological or psychophysiological responses to stressful scenario-based education were included.

This review identified that an acceptable level of stress during simulation can be beneficial, however a point can be exceeded where stress becomes a hinderance to learning resulting in underperformance. By identifying strategies to moderate the impact of acute stress, educators of paramedic and other healthcare students can utilise high-acuity clinical scenarios to their andragogical armamentarium which has the potential to improve real-world clinical outcomes.

Peer Review reports

Introduction

In many high-income countries, paramedic education has progressively moved from a post-employment vocational training model of the latter part of the 20th century to now sit firmly within the pre-employment tertiary education sector [ 1 ]. This evolution of education enables paramedics to provide high-level emergency care as new-to-practice clinicians in high pressure, time-critical environments [ 2 ]. To do this, education providers often utilise high-fidelity simulations to apply clinical or other skills in realistic environments. These simulations, are often comprised of high-acuity scenarios which are designed to depict a high severity of illness or injury [ 3 ] requiring rapid medical interventions, which can invoke increased physiological and cognitive stress. It may be the case that if these simulations are too stressful, clinical learnings from them may be lost due to the high stress load the participant is exposed to as shown by Takahashi, et al. [ 4 ] who identified higher cortisol levels post stress exposure in university students, which correlated with an increased level of memory impairment and poorer performance.

Links between physiological stress and knowledge application have also been reported in the paramedicine sector. LeBlanc, et al. [ 5 ] demonstrated that clinicians made more drug calculation errors following exposure to stressful events, whilst senior paramedics exhibited clinical and documentational vulnerabilities during high-acuity scenarios [ 6 ]. In the emerging field of undergraduate paramedicine education research, few studies have explored high-acuity scenario-based education and any associated physiological and cognitive stress. This is in contrast to Harvey, et al. [ 7 ], LeBlanc, et al. [ 6 ] and, more recently Hase, et al. [ 8 ] who have recommended that training in high-acuity areas of medicine should include challenge-promoting interventions specifically relevant to stress mitigation.

In the expanding cohort of university-trained paramedicine students, research on empathy [ 9 ], prevention of mental health and psychological disorders [ 10 ], workplace violence [ 11 ], physical characteristics [ 12 ] and pre-employment fitness testing [ 13 ] have been published. However, linkages between time critical high-acuity scenario-based education with resultant physiological stress and its potential impacts on cognitive decision-making has not been studied. A recent systematic review explored the physiological responses to acute stress in workers of several occupations, mostly within the human service industry [ 14 ]. Whilst this paper draws appropriate conclusions about acute physiological changes leading to performance decrement, possible implications for frontline healthcare workers were limited by a small number of healthcare-based studies included within the review. A gap also exists between self-awareness of one’s own physiological stress and how this may impact clinical judgement. Therefore, the purpose of this review was to better understand the physiological and cognitive stress responses observed in the participants undertaking high-acuity clinical scenarios. By appreciating the existence of contributory factors and how they influence stress, educators of paramedics and other healthcare workers can determine which elements of physiologically and mentally stressful scenario-based education can be considered in the design of their own programs.

Study design

Full systematic literature reviews (SLRs) are generally considered to be the foundation for evidence-based practice, particularly in healthcare [ 15 ]. This form of evidence synthesis relies on an extensive base of published literature and is frequently used to validate or refute current practice [ 16 ]. However, within the scope of the present study, little extant research reports on physiological changes triggered by high stress learning situations or the consequences this effect has on clinical performance. Given the inter-relationship between acute stress and the degradation of cognitive decision-making ability [ 7 , 17 ], further research is warranted to characterise this physiological response in undergraduate paramedicine students. In this paper, we employed a scoping review method to explore the extent of published and unpublished literature from cognate heath disciplines to identify key characteristics or factors related to our topic of interest.

Our final protocol was registered on the 21st March 2023, and is publicly available on the Open Science Framework platform ( https://osf.io/dxchy/ ).

Identifying the research question

This scoping review aims to identify and map the scope of current published literature related to physiological stress responses to high-acuity scenarios and, importantly, identify and analyse the knowledge gaps [ 18 ]. To achieve the aim, the following search strategy was employed:

Participants: higher education students or students in non-university training programs studying towards a recognised healthcare qualification.

Concept: any study that incorporates clinical scenarios / simulations where physiological (cardiovascular or endocrine) /or psychophysiological data is recorded.

Context: any undergraduate or postgraduate higher education setting or equivalent non-university training facility for the participants mentioned above.

Search strategy and eligibility criteria

The latest version [ 19 ] of Joanna Briggs Institute (JBI) comprehensive guide for authors conducting a scoping reviews [ 20 ] has been followed step-by-step within this review. A search period restriction from 2000 onwards was applied due to the rapid expansion of wearable technology including augmented and virtual reality. To ensure the review examined the acute physiological stress response, it was necessary to focus on articles that assessed markers of stress in real-time as participants were exposed to a stress-inducing task. An initial search was conducted across three prominent databases (Medline, PubMed and Scopus) to determine key terms as a guide to developing a thorough search strategy. From this and with the assistance of a senior research librarian, the secondary search expanded all identified keywords and incorporated medical subject headings (MeSH), major subject areas, and all other possible index terms as noted in the Appendix 1 . The protocol incorporated both published (peer-reviewed academic papers, reports and conference proceedings) and unpublished (incorporating theses and dissertations, research and technical reports) evidence but did exclude non-English language articles. Sources were gathered using EBSCOhost (including Medline, CINAHL and APA PsycInfo) Scopus, and PubMed. Google Scholar was also searched as there is a small body of evidence that suggests this search engine produces highly comprehensive results [ 21 , 22 ] whilst also searching ‘grey literature’ (published informally, non-commercially or remains unpublished), a format neglected by other databases. Selection of papers for inclusion in the study were then undertaken independently by two members of the research team (DL and EF). Finally, any other articles that cited the retrieved articles were also checked using citation alert with the ISI Web of Knowledge (Appendix 1 ).

Extracting and charting the data

Data were extracted from the included studies by two reviewers (JB and PH) utilising the JBI template of evidence details, characteristics and results extraction instrument [ 19 ]. Initial piloting of the data extraction resulted in some additional data being sought from each publication to allow quality appraisal to occur. This refined data extraction gathered details about study year, study country, study aim, study setting, study design, interventions, and comparators. Additionally, the data included sample size, methods, results, and author recommendations. A third member of the review team (DL) performed an accuracy check.

Quality assessment

Methodological validity and risk-of-bias appraisal, undertaken concurrently with data charting, was performed via the Mixed Methods Appraisal Tool (MMAT) version 2018 critical appraisal instrument designed by Hong, et al. [ 23 ]. For the purpose of this scoping review, an overall score was calculated from mean values of each section to determine methodological quality of each reviewed study (Appendix 4 ). The authors agreed that no cut-off scores would be applicable as the use of the MMAT was not for inclusion or exclusion purpose, but rather to describe the quality of the of publications reported in this review.

Synthesis of results

The first author performed narrative synthesis of identified themes and discussed these with the review team for validation. Descriptive results are subsequently reported which align with the intended scope and objectives of this review.

Ethics statement

Ethical approval was not required for this scoping review.

The search strategy yielded 1427 results, of which 52 remained after title and abstract proofing and duplicate removal (Level 1). Consensus was not reached on seven papers with resolution sought from a third member of the review team (PH) (Appendix 2 ). Of the 52 studies, 30 were excluded for reasons outlined in Appendix 3 . In addition, reference lists of three excluded review articles were checked, although nil additional suitable articles were identified. Unpublished (grey) literature was also assessed with no additional studies deemed suitable for inclusion. Figure 1 illustrates a Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flow diagram [ 24 ] of the process and Table 1 lists the 22 studies deemed eligible for inclusion.

PRISMA flow diagram findings

From the twenty-two studies identified that met the inclusion criteria, the majority originated from Europe and North America. Only one study [ 25 ] involved paramedics or paramedicine students. Twelve studies involved medical or surgical trainees [ 7 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 ], five studied nursing or nurse anaesthetist students [ 37 , 38 , 39 , 40 , 41 ], three studies involved physiotherapy students [ 42 , 43 , 44 ], and one study examined psychology students [ 45 ].

No studies were identified that warranted exclusion based on major methodological flaws on any significant risk of bias. However, study design flaws and lower levels of evidence were common. Of most concern were poorly described methodologies and under-powered sample sizes incapable of producing statistically significant results (see Appendix 4 for tabulated quality assessment results). Examining the methodology used, sixteen studies adopted a mixed methods approach and six utilised a quantitative method. Randomised controlled trials were reported in seven papers and a battery of different metrics were recorded across the studies. Heart rate variability and salivary cortisol levels were the most frequently reported objective data, whilst the state trait anxiety inventory was the most common subjective measure (see Table 2 ).

In general terms, the twenty-two included studies had similar aims centring around determining how successful high stress simulation could be at replicating clinical experience. Most studies involved both male and female participants with samples sizes ranging from n = 8 to n = 166, with a mean of n = 53 and a median of n = 33. Multiple studies assessed and compared stress levels of participants in different situations, and then used these results to determine if stress had affected clinical performance. Other studies used similar data to improve education or training with the aim of ultimately increasing student confidence and performance. The key outcomes from the included studies are summarised in Table 3 .

This review identified studies exploring physiological responses of participants undertaking high stress scenario-based education or training. While acknowledging much of the evidence was of low methodological quality [ 46 ] and therefore limits generalisability, the results still provide some useful insights that may be used to inform educators of future paramedics and other healthcare workers.

One of the key findings from this review was the identification of pre-performance or anticipatory anxiety exhibited across multiple studies [ 37 , 38 , 45 ]. This is an area where simulation may not replicate clinical work. Students aware of an upcoming scenario well in advance have ample time to prepare and mount a physiological stress response. This could be controlled if students were given little notice, however this was not commonly reported in the studies. Healthcare educators utilising scenario-based education may choose to restrict prior notification as a means of assessing any changes in the stress response amongst their students. In high-acuity clinical work, paramedics usually have little time to prepare, which may reduce the anticipatory stress response. Potentially this may be seen as positive, as stress has been demonstrated to lead to poorer performance is some paramedic research [ 5 , 6 ]. However, the evident stress of attending high-acuity cases must also be considered and its impact on performance. In the context of anticipatory anxiety predicting future performance, little research has examined its immediate effect on motor task performance.

The reviewed publications also provide contradictory support for simulation as a tool to replicate the psychophysiological stress of high-acuity clinical work. Baker, et al. [ 37 ], in a study with trainees in the highly specialised field of anaesthetics, found simulation was able to replicate the physical and procedural forms of clinical work, however it was unable to replicate the intrinsic level of stress the trainees exhibited when working with a real patient in an operating theatre. These results are potentially influenced by small participant numbers ( n = 8) and may also be applicable to highly specialised and highly technical fields such as anaesthesia. For paramedicine, contemporary literature [ 47 ] highlights simulation allowing for the training of skills that are rarely needed or rarely practiced in the field and supports recommendation made by O’Meara, et al. [ 2 ]. For educators of paramedics and other healthcare workers, simulating high-acuity situations is a crucial way to expose students to potential clinical scenarios they may face early in their career. In designing programs of study, careful use of stress-inducing high-acuity simulation can be a beneficial but can also lead to continued underperformance if the stress is chronically too high.

Barbadoro, et al. [ 26 ] and Judd, et al. [ 43 ] found simulation provided a higher level of stress in their participants when compared to equitable clinical work, whilst Demaria, et al. [ 28 ] found that high stress situations can be beneficial for learning. This benefit of high stress learning was also supported by the work of Keitel, et al. [ 32 ], who found increased levels of the key stress hormone cortisol correlated with improved memory retention and medical performance amongst medical trainees. An increased stress response was also reported when supervisors or assessors were present within the simulation [ 30 , 31 ] and, unsurprisingly, vital signs as a measure of physiological stress, increased when the simulation itself was exertive [ 25 , 27 , 36 ]. The stress placed on students involved in high-acuity simulation must be further studied to allow educators to determine what level of anxiety may enhance learning without impeding performance.

Performance ability or academic standing was also found to correlate with stress. McKay, et al. [ 38 ] found low performers increased stress and performed poorly, whereas high performers also increased stress but performed superbly in a cohort of student nurses. Paramedicine courses may show similar trends, with students likely to self-assess their academic abilities and stress tolerance. Educators could potentially use real-time learning analytics to offer tailored support and guidance based on live biometric data, proactively aiding students. This would be resource intensive for academics with large student numbers; but in smaller cohorts, the individual feedback around acceptable stress to achieve simulated clinical success may enhance the education program.

Lacking from the literature is a detailed discussion of a variety of variables related to student stress responses from the level of acuity of a scenario. These confounding variables, such as pre-established coping styles and perceived stress intensity within participants, need to be quantified to accurately gauge the success of any interventions aimed at alleviating the stress response, and in examining what levels of anxiety may enhance learning without impeding performance.

Limitations

Whilst the systematic approach to this scoping review explored multiple electronic bibliographic repositories, there is potential some contemporary conference proceedings, dissertations and theses, along with grey literature not readily available in electronic databases or Google Scholar, may have been missed. Non-English literature may have added value to this review and we attempted to seek translated papers where possible, but we accept that some results may have been missed through this process.

Directions and recommendations for future research

This scoping review identified inconsistencies and varying methodologies for the assessment of participant stress response in scenario-based education. Recommendations should be developed to identify gold standard quantification of psychophysiological stress responses during high stress scenarios. This would then allow meta-analysis or other systematic synthesis of data to be undertaken to accurately determine any inter-relationship between acute stress and the degradation of cognitive decision-making for healthcare education programs. In addition, variables related to student stress responses from the level of acuity of a scenario should be investigated. For example, individual differences in participants such as pre-established coping styles and strategies, perceived stress intensity, perceived control of stress or coping skill, as well as context specific stressors such as the outcomes associated with scenario performance (e.g., high stakes versus low stakes outcomes) may all be important variables for future research.

The studies identified in this scoping review have shown high-acuity simulation can induce stress comparable with paramedicine clinical practice. For educators, understanding the factors or elements which contribute to an acceptable level of stress can allow participants the opportunity to fail and learn from their errors during simulation. This further provides opportunities to improve student outcomes in paramedicine and other healthcare education by facilitating high-acuity clinical scenarios that challenge students without inducing stress levels that hinder performance. As educational and wearable technology further evolves, utilisation of real-time biofeedback through passive measurement devices also hold promise as an intervention to reduce the negative effects of acute physiological stress during training scenarios.

Data availability

Data supporting Fig. 1 ; Tables 1 , 2 and 3 and available within the Supplementary Information (Appendices).

Hou XY, Rego J, Service M. Review article: paramedic education opportunities and challenges in Australia. Emerg Med Australasia. 2013;25(2):114–9.

Article Google Scholar

O’Meara P, Furness S, Gleeson R. Educating paramedics for the future: a holistic approach. J Health Human Serv Adm. 2017;40(2):219–53.

Google Scholar

Chrimes N. The Vortex: a universal ‘high-acuity implementation tool’ for emergency airway management. Br J Anaesth. 2016;117(Suppl 1):i20–7.

Takahashi T, Ikeda K, Ishikawa M, Tsukasaki T, Nakama D, Tanida S, et al. Social stress-induced cortisol elevation acutely impairs social memory in humans. Neurosci Lett. 2004;363(2):125–30.

LeBlanc VR, MacDonald RD, McArthur B, King K, Lepine T. Paramedic performance in calculating drug dosages following stressful scenarios in a human patient simulator. Prehospital Emerg Care. 2005;9(4):439–44.

LeBlanc VR, Regehr C, Tavares W, Scott AK, Macdonald R, King K. The impact of stress on paramedic performance during simulated critical events. Prehosp Disaster Med. 2012;27(4):369–74.

Harvey A, Nathens AB, Bandiera G, Leblanc VR. Threat and challenge: cognitive appraisal and stress responses in simulated trauma resuscitations. Med Educ. 2010;44(6):587–94.

Hase A, O’Brien J, Moore LJ, Freeman P. The relationship between challenge and threat states and performance: a systematic review. Sport Exerc Perform Psychol. 2019;8(2):123–44.

Williams B, Boyle M, Earl T. Measurement of empathy levels in undergraduate paramedic students. Prehosp Disaster Med. 2013;28(2):145–9.

Wild J, El-Salahi S, Tyson G, Lorenz H, Pariante CM, Danese A, et al. Preventing PTSD, depression and associated health problems in student paramedics: protocol for PREVENT-PTSD, a randomised controlled trial of supported online cognitive training for resilience versus alternative online training and standard practice. BMJ Open. 2018;8(12):e022292.

Boyle M, McKenna L. Paramedic student exposure to workplace violence during clinical placements - a cross-sectional study. Nurse Educ Pract. 2017;22:93–7.

Davies S, Naidoo N, Parr B. Physical performance characteristics of South African male and female Emergency Care students (ECS). Ergon SA: J Ergon Soc South Afr. 2008;20(2):3–14.

Thornton KE, Sayers MG. Unfit for duty? Evaluation of 4 years of paramedic preemployment fitness screening test results. Prehospital Emerg Care. 2014;18(2):201–6.

Frazier SE, Parker SH. Measurement of physiological responses to acute stress in multiple occupations: a systematic review and implications for front line healthcare providers. Transl Behav Med. 2019;9(1):158–66.

Munn Z, Porritt K, Lockwood C, Aromataris E, Pearson A. Establishing confidence in the output of qualitative research synthesis: the ConQual approach. BMC Med Res Methodol. 2014;14(1):108.

Pearson A, Jordan Z, Munn Z. Translational science and evidence-based healthcare: a clarification and reconceptualisation of how knowledge is generated and used in healthcare. Nurs Res Pract. 2012;2012:792519.

Domes G, Heinrichs M, Rimmele U, Reichwald U, Hautzinger M. Acute stress impairs recognition for positive words–association with stress-induced cortisol secretion. Stress. 2004;7(3):173–81.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synthesis. 2020;18(10):2119–26.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141–6.

Bajpai A, Davuluri S, Haridas H, Kasliwal G, Ks HD. S, In search of the right literature search engine(s). Nat Precedings. 2011.

Gehanno JF, Rollin L, Darmoni S. Is the coverage of Google Scholar enough to be used alone for systematic reviews. BMC Med Inf Decis Mak. 2013;13(1):7.

Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The mixed methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inform. 2018;34(4):285–91.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. 2021;18(3):e1003583.

MacQuarrie AS, Hunter JR, Sheridan S, Hlushak A, Sutton C, Wickham J. Paramedic Student Clinical Performance during High-Fidelity Simulation after a physically demanding Occupational Task: a pilot randomized crossover trial. Simul Healthcare: J Soc Simul Healthc. 2022;17(4):234–41.

Barbadoro P, Brunzini A, Dolcini J, Formenti L, Luciani A, Messi D, et al. Stress responses in high-fidelity simulation and standard simulation training among medical students. BMC Med Educ. 2023;23(1):116.

Bialka S, Copik M, Ubych A, Marciniak R, Smereka J, Szarpak L, et al. Effect of high-fidelity simulation on alpha-amylase activity and concentrations of secretory immunoglobulin class A, cortisol, and testosterone among medical students. Endocrine. 2021;73(2):431–8.

Demaria S Jr., Bryson EO, Mooney TJ, Silverstein JH, Reich DL, Bodian C, et al. Adding emotional stressors to training in simulated cardiopulmonary arrest enhances participant performance. Med Educ. 2010;44(10):1006–15.

DeMaria S, Silverman ER, Lapidus KA, Williams CH, Spivack J, Levine A, et al. The impact of simulated patient death on medical students’ stress response and learning of ACLS. Med Teach. 2016;38(7):730–7.

Feeley AA, Feeley IH, McManus R, Lunn JV, Sheehan E, Merghani K. Evaluating the impact of Supervision on Surgical trainees stress response during simulated Surgical procedures; a crossover randomized Trial. J Surg Educ. 2022;79(6):1379–86.

Flinn JT, Miller A, Pyatka N, Brewer J, Schneider T, Cao CG. The effect of stress on learning in surgical skill acquisition. Med Teach. 2016;38(9):897–903.

Keitel A, Ringleb M, Schwartges I, Weik U, Picker O, Stockhorst U, et al. Endocrine and psychological stress responses in a simulated emergency situation. Psychoneuroendocrinology. 2011;36(1):98–108.

Los K, Chmielewski J, Cebula G, Bielecki T, Torres K, Luczynski W. Relationship between mindfulness, stress, and performance in medical students in pediatric emergency simulations. GMS J Med Educ. 2021;38(4):Doc78.

Martin-Rodriguez F, Castro Villamor MA, Lopez-Izquierdo R, Portillo Rubiales RM, Ortega GJ, Sanz-Garcia A. Can anxiety in undergraduate students in a high-fidelity clinical simulation be predicted? A randomized, sham-controlled, blinded trial. Nurse Educ Today. 2021;98:104774.

Rieber N, Betz L, Enck P, Muth E, Nikendei C, Schrauth M, et al. Effects of medical training scenarios on heart rate variability and motivation in students and simulated patients. Med Educ. 2009;43(6):553–6.

Tramèr L, Becker C, Hochstrasser S, Marsch S, Hunziker S. Association of electrocardiogram alterations of rescuers and performance during a simulated cardiac arrest: a prospective simulation study. PLoS ONE. 2018;13(6):e0198661.

Baker BG, Bhalla A, Doleman B, Yarnold E, Simons S, Lund JN, et al. Simulation fails to replicate stress in trainees performing a technical procedure in the clinical environment. Med Teach. 2017;39(1):53–7.

McKay KA, Buen JE, Bohan KJ, Maye JP. Determining the relationship of acute stress, anxiety, and salivary alpha-amylase level with performance of student nurse anesthetists during human-based anesthesia simulator training. AANA J. 2010;78(4):301–9.

Nakayama N, Arakawa N, Ejiri H, Matsuda R, Makino T. Heart rate variability can clarify students’ level of stress during nursing simulation. PLoS ONE. 2018;13(4).

Park HJ, Choi D, Park HA, Lee CA. Nurse evaluation of stress levels during CPR training with heart rate variability using smartwatches according to their personality: a prospective, observational study. PLoS ONE. 2022;17(6).

Stecz P, Makara-Studzińska M, Białka S, Misiołek H. Stress responses in high-fidelity simulation among anesthesiology students. Sci Rep. 2021;11(1):17073.

Beltrán-Velasco AI, Ruisoto-Palomera P, Bellido-Esteban A, García-Mateos M, Clemente-Suárez VJ. Analysis of Psychophysiological Stress Response in higher education students undergoing clinical practice evaluation. J Med Syst. 2019;43(3):68.

Judd BK, Alison JA, Waters D, Gordon CJ. Comparison of psychophysiological stress in Physiotherapy Students Undertaking Simulation and Hospital-based Clinical Education. Simul Healthcare: J Soc Simul Healthc. 2016;11(4):271–7.

Palekar TJ, Mokashi MG, Anwer S, Kakrani AL, Khandare SD, Alghadir AH. Effect of galvanic skin resistance-aided Biofeedback Training in reducing the pulse rate, respiratory rate, and blood pressure due to perceived stress in Physiotherapy students. / Fizyoterapi Öğrencilerinde Algılanan Strese Bağlı Nabız Hızı, Solunum Hızı ve Kan Basıncının Azaltılmasında Galvanik Deri Direnci Destekli Bio-geri Bildirim Eğitiminin Etkisi. Turkish J Phys Med Rehabilitation / Turkiye Fiziksel Tip ve Rehabilitasyon Dergisi. 2015;61(2):116–9.

Beltrán-Velasco AI, Bellido-Esteban A, Ruisoto-Palomera P, Clemente-Suárez VJ. Use of Portable Digital devices to analyze autonomic stress response in psychology objective structured clinical examination. J Med Syst. 2018;42(2):35.

Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based Medicine. Plast Reconstr Surg. 2011;128(1):305–10.

Diamond A, Bilton N. The current state on the Use of Simulation in Paramedic Education. Australasian J Paramedicine. 2021;18:1–5.

Download references

Acknowledgements

The authors wish to acknowledge and thank senior research librarian Meena Gupta for assistance with developing and refining key search terms within the research protocol.

No funding sources or any specific materials need to be disclosed.

Author information

Authors and affiliations.

Faculty of Health, Australian Catholic University, Building 403, Daniel Mannix Building, 8 – 14 Brunswick St, Fitzroy, VIC, 3065, Australia

Jason Betson

School of Health and Medical Sciences, Centre for Health Research, University of Southern Queensland, Ipswich, Australia

Jason Betson & David Long

School of Clinical Sciences, Queensland University of Technology, Brisbane, Australia

Peter Horrocks

School of Psychology and Wellbeing, Centre for Health Research, University of Southern Queensland, Toowoomba, Australia

Erich C. Fein

You can also search for this author in PubMed Google Scholar

Contributions

JB, DL, EF and PH conceived the study. JB and DL equally designed the study approach. DL, EF and PH undertook the review. JB and PH undertook data extraction and quality control. JB interpreted the data. JB, DL and EF drafted the manuscript and circulated to authors for contribution. All authors edited drafts and approved the current manuscript for publication. JB as the corresponding author is responsible for the overall content.

Corresponding author

Correspondence to Jason Betson .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Betson, J., Fein, E.C., Long, D. et al. Too stressed to think? A scoping review of the literature for healthcare educators utilising high acuity clinical scenarios. BMC Med Educ 24 , 990 (2024). https://doi.org/10.1186/s12909-024-05949-3

Download citation

Received : 05 February 2024

Accepted : 23 August 2024

Published : 11 September 2024

DOI : https://doi.org/10.1186/s12909-024-05949-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Physiological stress
Decision-making
Paramedicine

BMC Medical Education

ISSN: 1472-6920

General enquiries: [email protected]

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Study Protocol

Health professionals’ attitudes towards traditional healing for mental illness: A systematic review protocol

Roles Conceptualization, Methodology, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Faculty of Medicine and Health, School of Health, University of New England, Armidale, NSW, Australia, Department of Psychiatry, College of Medicine and Health Sciences, Injibara University, Injibara, Ethiopia

Roles Supervision, Validation, Visualization, Writing – review & editing

Affiliation Faculty of Medicine and Health, School of Health, University of New England, Armidale, NSW, Australia

Alemayehu Molla Wollie,
Kim Usher,
Reshin Maharaj,
Md Shahidul Islam

Published: September 9, 2024
https://doi.org/10.1371/journal.pone.0310255
Peer Review
Reader Comments

Mental illness is a global problem that receives less attention, particularly in developing countries. Integrating modern treatment with traditional healing approaches has been proposed as one way to address mental health problems, especially in developing countries. Despite health professionals’ participation in traditional healing being crucial to integrative approaches, their participation is limited to date. This review protocol is designed to explore the attitudes of health professionals towards traditional healing practices in mental health services.

The review will follow the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines. Searching databases, including PubMed/Medline, PsychINFO, EMBASE, Scopus, and the Web of sciences will be conducted. Additionally, Google and Google Scholar will be searched for other information, including grey literature. Moreover, a manual search of identified articles’ reference lists will also be conducted to help ensure all potential papers are included in the review. Qualitative, quantitative, and mixed study methods published in English between January 2014 and April 2024 will be included. The qualities of the included studies will be assessed using the Mixed Methods Appraisal Tool (MMAT) Version 2018. A mixed-method synthesis will be used to synthesis the results.

It is crucial for healthcare professionals to provide culturally sensitive care to empower people to manage their health. This systematic review will summarize the attitudes of health professionals towards the adoption and delivery of traditional healing approaches to people experiencing mental illness. Therefore, the findings of this review will support integration between traditional healers and modern mental health practitioners for the treatment of mental illness.

Trial registration

Protocol registration number: CRD42024535136 .

Citation: Wollie AM, Usher K, Maharaj R, Islam MS (2024) Health professionals’ attitudes towards traditional healing for mental illness: A systematic review protocol. PLoS ONE 19(9): e0310255. https://doi.org/10.1371/journal.pone.0310255

Editor: Madhulika Sahoo, Kalahandi University, INDIA

Received: May 8, 2024; Accepted: August 27, 2024; Published: September 9, 2024

Copyright: © 2024 Wollie et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data are in the manuscript.

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that co competing interest exists.

Mental health is the overall wellbeing and functioning of an individual, family, and community [ 1 , 2 ]. It is “a state of wellbeing in which the individual realizes his or her own abilities, is able to contribute to his or her environment, copes with the normal stress of life, works fruitfully, and is able to cooperate with others [ 3 ].” In contrast, mental illness has a substantial impact on a person’s feelings, thoughts, behaviour, and social interactions [ 4 , 5 ]. It is influenced by numerous factors, including psychological, physical, social, cultural, and spiritual [ 6 , 7 ].

Currently, mental illness is a major public health burden and accounts for 32.4% of years lived with disability worldwide [ 8 ]. Even though mental health issues are a global health concern, they are highly prevalent in low-income countries [ 9 , 10 ]. Developing nations have fewer health professionals [ 11 ], resulting in a treatment gap for people with mental illnesses in these countries [ 12 ]. Furthermore, negative attitudes, stigma, limited resources, and low priority are reasons for the high magnitude of mental illness, in addition to a shortage of health professionals [ 13 – 16 ].

People with mental health issues often seek care from indigenous or traditional healers [ 17 – 19 ]. Traditional healing practices include but are not limited to a set of beliefs that use culturally accepted spiritual treatments and plant products [ 20 ]. Traditional healers form a major part of the mental health workforce in developing countries [ 21 ]. Nearly 80% of people from Africa seek medical attention from traditional healers [ 21 ], largely because traditional healers provide culturally and socially accepted care for individuals and communities. This is the preferred treatment approach for most people due to its perceived effectiveness, affordability, and accessibility within communities [ 22 – 24 ]. In order to close the significant treatment gap in low-and middle-income countries, “the World Health Organization’s (WHO) 2003–2020 Mental Health Action Plan recommended that government health programs incorporate traditional healers as treatment resources [ 25 ].”

Studies indicate that traditional healers can offer efficacious therapies that may be beneficial for prevalent mental illnesses such as depression and anxiety [ 26 , 27 ]. Healers usually come from extended family branches with experience providing care like counselling, bible interpretation, and using prayer aids such as holy water and oil for treatment [ 27 , 28 ]. They either obtain experience through on-the-job training or apply their practices through ancestors who serve as mediators by providing access to spiritual guidance and power [ 15 , 29 , 30 ].

Despite the fact that traditional healing approaches are practiced by large groups of people, particularly in low-income countries, modern health professionals tend to differ in their opinions about this approach [ 31 , 32 ]. Studies indicate that some health professionals hold a positive view of traditional healing modalities [ 33 ], while many others express doubt about the effectiveness of traditional treatments. Furthermore, there is a negative attitude among health professionals regarding the integration of traditional healing with biomedical therapies [ 34 , 35 ]. Traditional healers are known to consult with modern health professionals when faced with problems beyond their ability to treat, but it is not common for biomedical professionals to refer back to healers [ 27 ]. This is often due to their negative attitudes toward traditional healing practices, including the belief that traditional healers do not incorporate a human right approach and/or compassionate care when dealing with people with mental health problems [ 35 , 36 ]. Their negative attitudes have a detrimental effect on the WHO’s recommended collaboration of biomedical treatment and traditional healing to close the treatment gap in low and middle-income countries because cooperation between the two needs mutual respect and recognition. In addition to providing biological therapies, it is good to emphasize spiritual and cultural approaches for mental illness since mental illness is due to multidimensional factors like spiritual, social, psychological, and physical. Moreover, it is crucial for healthcare providers to understand their patients’ cultural beliefs and practices in order to manage and counsel them appropriately [ 34 ] and empower them towards healthful healing. There is no summarized research regarding the attitudes of health professionals toward traditional healing approaches for mental illness, despite the existence of numerous singe studies. Therefor, this review protocol is aimed to explore the summarized evidences on health care providers’ attitudes toward traditional healing for mental illness, and this will be crucial in developing appropriate guidelines for holistic treatment.

Research question

What are the health professionals’ attitudes towards traditional healing for mental illness?

Methods and materials

The Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) 2020 guidelines will be used to conduct this review [ 37 ]. This protocol is also prepared in accordance with the PRISAM-Protocol guidelines (Supporting Information). Primary studies conducted using quantitative, quantitative, and mixed-methods will be included. The review protocol is registered with the International Prospective Register of Systematic Reviews (PROSPERO) with a registration number of CRD42014535136.

Eligibility criteria

The eligibility assessment format will be used to select articles to be included in the systematic review. Globally published original articles focusing on health professionals’ attitude and/or perception towards traditional healing of mental illness will be considered. Qualitative, quantitative, and mixed-method studies will be considered without restrictions in their study design. Studies published in English between January 2014 and April 2024 will be considered to summarize contemporary evidence on the topic. Conference summaries, reviews, dissertations, case studies, governmental and non-government reports, and letters will be excluded from the study.

Searching strategy

A literature search of databases such as PubMed/Medline, PsychINFO, EMBASE, Scopus, and the Web of Sciences will be conducted. The key terms will be searched by connecting Boolean operators OR/AND to specifically address published studies. Searching words will be (attitude) OR (perception) OR (belief) OR (opinion) OR (view) AND (health professionals) OR (health practitioners) OR (health personnel) OR (nurses) OR (medical doctors) OR (psychiatrist) OR (psychologist) AND (traditional healing) OR (traditional medicine) OR (herbal medicine) OR (indigenous treatment) OR (spiritual therapy) OR (religious healing) AND (mental illness) OR (mental disorder) OR (mental health) OR (psychological distress) OR (psychiatric disorder). In addition to database and manual searches, we will use references to articles obtained from the database to filter the remaining studies. Database searching and the initial screening process will be finalized in consultation with a senior health librarian from the University of New England (UNE) Dixon Library and other authors.

An example of a search strategy for Scopus ( (ALL ("health professional") OR TITLE-ABS-KEY ("health personnel") OR TITLE-ABS-KEY ("medical doctor") OR TITLE-ABS-KEY ("medical practitioner") OR TITLE-ABS-KEY (nurse) OR TITLE-ABS-KEY (psychiatrist) OR TITLE-ABS-KEY (psychologist) ) ) AND ( (ALL (perception) OR TITLE-ABS-KEY (attitude) OR TITLE-ABS-KEY (belief) OR TITLE-ABS-KEY (view) OR TITLE-ABS-KEY (opinion) ) ) AND ( (ALL ("traditional healing") OR TITLE-ABS-KEY ("traditional medicine") OR TITLE-ABS-KEY ("indigenous treatment") OR TITLE-ABS-KEY ("spiritual therapy") OR TITLE-ABS-KEY ("religious healing") OR TITLE-ABS-KEY ("herbal medicine") ) ) AND ( (ALL ("mental illness") OR TITLE-ABS-KEY ("mental disorder") OR TITLE-ABS-KEY ("mental health") OR TITLE-ABS-KEY ("psychiatric disorder") OR TITLE-ABS-KEY ("psychological distress") ) ) AND PUBYEAR > 2013 AND PUBYEAR < 2025 AND (LIMIT-TO (DOCTYPE, "ar") ) AND (LIMIT-TO (LANGUAGE, "English") )

Study selection process

PRISMA (preferred reporting items for the systematic review and meta-analysis) guidelines [ 37 ] will be followed to ensure the quality and transparency of the review process. After critically searching articles from databases, we will start the selection process by importing all records to an Endnote library. Duplicates will be detected and removed automatically. At the beginning, the selection process will be carried out by the first author (AMW), and it will be checked by one of the co-authors (KU, RM, and SI). The reviewer will evaluate article titles and abstracts for relevance, keeping those that are relevant to the outcome variable attitudes, and/or perceptions of health professionals towards traditional healing. The full texts of possible relevant papers will then be screened for eligibility. Any disagreements will be handled through a discussion to finalize the selection of the papers.

Data extraction or collection process

With the use of a predetermined, uniform data extraction structure, the first author will extract important data, and then it will be confirmed by another author. Any significant differences will be discussed, and if a consensus cannot be reached, a third author will adjudicate on the decision. The first author’s name, the year of publication, the country where the study is conducted, the methodology used, the health professional type, the data collection and analysis method, the major findings, and the conclusions will be included in the standardized data extraction form.

Quality assessment

Each study’s methodological quality and bias risk will be assessed to determine the validity of the findings. The qualities of the included articles will be assessed using the Mixed Methods Appraisal Tool (MMAT) Version 2018 Checklist [ 38 ]. This tool is designed to assess the methodological qualities of different types of studies, like qualitative, quantitative, and mixed studies. The differences in quality evaluation scores will be resolved by through discussion within a team.

Data synthesis

An integrated mixed-method synthesis technique will be used to summarize all quantitative, qualitative, and mixed-method data in one combined synthesis [ 39 ]. Quantitative data will be described qualitatively to facilitate integration with qualitative data. The overall nature of the articles will be presented in a table. Themes and subthemes will be formulated based on the data generated. All the data will be analysed and reported as a narrative summary. To minimize bias, the research team will thoroughly evaluate and discuss the entire process, and any disagreements will be resolved through discussion.

The review will be conducted to summarize the attitudes of health professionals towards traditional healing approaches for mental illness in a global context. Even though independent studies have been undertaken on health professionals’ attitudes and/or perceptions towards traditional healing, there is no globally summarized general evidence on the topic. Some previous studies present opposing evidence on the attitudes of health professionals towards traditional healing practices. Even if some health professionals encourage traditional methods, there are other studies that show negative attitudes of health professionals towards traditional healing [ 34 , 35 ]. Traditional healing practices are considered appropriate options for mental illness, particularly in low-income countries where there is a scarcity of modern mental health professionals [ 21 ]. Most people with mental health problems seek these treatments because they provide culturally and socially accepted care for individuals and the community [ 22 – 24 ]. But there is limited collaboration between modern treatment and traditional healing. Therefore, this review will summarize all the literature and clearly show the positions of health professionals toward traditional healing approaches. The findings of this review will be important in developing directions for governmental organizations and other concerned bodies to strengthen communication and integration of traditional healing with biomedical treatment options for mental illness.

Expected limitations

This systematic review has the limitation of excluding articles that are not published in English. In addition, the exclusion of conference reports, letter, short communications, and review articles may be a reason to miss important data.

Publication status

This protocol is prepared for the ongoing systematic review of articles entitled “The health professionals’ attitude towards traditional healing approaches to mental illness.” If there is any change during the review, an updated version will be published with a final systematic review.

Supporting information

S1 checklist. prisma-protocol guidelines..

https://doi.org/10.1371/journal.pone.0310255.s001

View Article
Google Scholar
PubMed/NCBI

Browse by Year
Browse by Journal Title
Browse by Author

and Khairatun Hisan Idris Shazali, and Sulistya Rusgianto, (2024) Islamiyyat : Jurnal Antarabangsa Pengajian Islam; International Journal of Islamic Studies, 46 (1). pp. 113-133. ISSN 0216-5636

997kB

Official URL: https://ejournal.ukm.my/islamiyyat/issue/view/1700

The study aims to expand the existing body of knowledge by highlighting and summarising current studies on Islamic inheritance. Thus, this study presented a map of the research interest and determined primary issues to demonstrate current trends, future research directions, and theory development. A systematic literature review (SLR) approach was employed using the four-step reporting standard for the Systematic Evidence Syntheses (ROSES) review: research question formulation, systematic searching, quality assessment, and data extraction. A total of 33 screened articles between 2008 and 2022 were retrieved from two primary databases (Scopus and Emerald and one supporting database, Google Scholar). The factors were categorised into three primary themes: general, motives and preference studies, and determinant analysis. The themes were subsequently divided into seven sub-themes. Notably, most studies examined unclaimed asset issues based on the determinant analysis. This study offered useful insights into the current research interest regarding the subject. The issues and domains in the current study contribute new knowledge and theory development for future directions in the field. This study addressed empirical gaps and presents an excellent reference relevant for regulators, policymakers, and researchers in comprehending the issues and existing solutions in Islamic inheritance.

Item Type:	Article
Keywords:	Islamic inheritance; SLR; Islamic wealth management; Islamic finance; Islamic Estate Planning
Journal:
ID Code:	24155
Deposited By:	Noor Marina Yusof
Deposited On:	10 Sep 2024 07:40
Last Modified:	10 Sep 2024 07:40

Repository Staff Only: item control page

archive-creating software, which generates eprints archives that are compliant with the Open Archives Protocol for Metadata Harvesting.Installed and configured by Division of Information System and Technology,

IMAGES

How to use Google Scholar for Effective Literature Review
How to do a literature review using Google Scholar
How To Conduct A Literature Review Using Google Scholar Step By Step
Literature reviews and Google Scholar
Literature Reviews: Google Scholar
Use Google Scholar

VIDEO

Google Scholar Trick for Literature Review🔥 #chatgpt #literature #researchpaper #viralshorts
Top 10 Resources & Tools For Research
Ultimate Guide: 8 Databases Every Researcher Must Know
How To Find And Download Research Paper, Academic Paper, Journal & Publications Free
How to find Research Papers Sinhala
Google Scholar

COMMENTS

Google Scholar
Google Scholar provides a simple way to broadly search for scholarly literature. Search across a wide variety of disciplines and sources: articles, theses, books, abstracts and court opinions.
Literature Review: Google Scholar
To see links to BenU Library subscription content in your Google Scholar search results: Go to Google Scholar > Settings > Library Links. Search " Benedictine ". Check the boxes. Click Save and you're done! Google Scholar Library Links Tutorial. This tutorial will guide you step-by-step through the quick setup process.
How to do a literature review using Google Scholar
This video shows a few tips and tricks for how to find academic literature using Google Scholar.
how to do a literature review using google scholar
how to do a literature review using google scholarA literature review is an overview of the previously published works on a specific topic. The term can refe...
Approaching literature review for academic purposes: The Literature
LITERATURE REVIEW CHECKLIST. In contrast to review papers, the LR of a dissertation/thesis should not be a standalone piece or work. Instead, it should present the student as a scholar and should maintain the interest of the audience in how that dissertation/thesis will provide solutions for the current gaps in a particular field.
Writing a literature review
A formal literature review is an evidence-based, in-depth analysis of a subject. There are many reasons for writing one and these will influence the length and style of your review, but in essence a literature review is a critical appraisal of the current collective knowledge on a subject. Rather than just being an exhaustive list of all that ...
Literature Review & Research Skills Guide: Use Google Scholar
Google Scholar allows you to locate resources such as articles, theses and books. Unlike Primo Search, which is set to search the Library's holdings only, Google Scholar searches beyond Charles Sturt University Library and will include resources that are not available to you.
How To Conduct A Literature Review Using Google Scholar Step By Step
Step By Step Guide of using Google Scholar for Literature search. Step 1: Type " Google Schola r" into the search bar and click and find the google scholar dashboard. Note: The first thing to keep in mind is that you may type "google scholar" in the search field and use the same Google tactics as you would when conducting a regular ...
How to write a superb literature review
Attribute. Manubot. Overleaf. Google Docs. Cost. Free, open source. $15-30 per month, comes with academic discounts. Free, comes with a Google account. Writing language
The Use of Google Scholar for Research and Research Dissemination
In this way, Google Scholar offers a collaborative literature search effort that extends beyond the initial results listing and potentially expanding the breadth of literature that can inform a literature review. In Google Scholar, when a research topic is searched, a list of publications is created.
Ten Simple Rules for Writing a Literature Review
use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and ... A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue . Rule 9: Include ...
How-to conduct a systematic literature review: A quick guide for
Method details Overview. A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure [12].An SLR updates the reader with current literature about a subject [6].The goal is to review critical points of current knowledge on a ...
How to Write a Literature Review
Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.
What is a Literature Review?
Use Google Scholar to find articles from a widevariety of academic publishers, professional societies, preprint repositories and universities, as well as scholarly articles available across the web Humanities and Social Science Retrospective Bibliographic database that provides citations to articles in a wide range of English language journals ...
The Role of Google Scholar in Evidence Reviews and Its Applicability to
Only one review had an included article list that was fully covered by Google Scholar, the review with the smallest evidence base of only 37 studies . ... The grey literature returned by Google Scholar may be seen by some as disadvantageous given its perceived lack of verification (through formal academic peer-review), particularly where ...
Chapter 9 Methods for Literature Reviews
The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ... To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar. The search was performed using several terms and free text words ...
Literature review as a research methodology: An overview and guidelines
As mentioned previously, there are a number of existing guidelines for literature reviews. Depending on the methodology needed to achieve the purpose of the review, all types can be helpful and appropriate to reach a specific goal (for examples, please see Table 1).These approaches can be qualitative, quantitative, or have a mixed design depending on the phase of the review.
The Literature Review: A Foundation for High-Quality Medical Education
The literature review helps any researcher "join the conversation" by providing context, informing methodology, identifying innovation, minimizing duplicative research, and ensuring that professional standards are met. ... Google Scholar. Once relevant articles are located, it is useful to mine those articles for additional citations. One ...
Reviewing the Literature
Use Google Scholar or Scopus to identify more recent articles that have cited that one. Structure of a Literature Review. When you're writing a literature review, you won't just summarize all of the articles that you've found on your topic. Instead, you'll synthesize the evidence. This means that you'll look at the state of the evidence as a ...
Guidance on Conducting a Systematic Literature Review
Literature reviews establish the foundation of academic inquires. However, in the planning field, we lack rigorous systematic reviews. In this article, through a systematic search on the methodology of literature review, we categorize a typology of literature reviews, discuss steps in conducting a systematic literature review, and provide suggestions on how to enhance rigor in literature ...
A practical guide to data analysis in general literature reviews
The general literature review is a synthesis and analysis of published research on a relevant clinical issue, and is a common format for academic theses at the bachelor's and master's levels in nursing, physiotherapy, occupational therapy, public health and other related fields. ... Google Scholar. 3. Hart C. Doing a literature review ...
The Role of Google Scholar in Evidence Reviews and Its ...
Google Scholar (GS), a commonly used web-based academic search engine, catalogues between 2 and 100 million records of both academic and grey literature (articles not formally published by commercial academic publishers). Google Scholar collates results from across the internet and is free to use. As a result it has received considerable ...
Google Scholar: a review of literature examining its ...
By Helena Korjonen, PhD Postdoctoral Researcher, Sustainable Food Practices, University of Luxembourg. Google Scholar (GS) is the top search engine used by those who are looking for scholarly content 1.There are many reasons for this, not least that it feels familiar 2, 3.However, a review of the literature reveals that there are significant limitations to its effectiveness.
The Role of Google Scholar in Evidence Reviews and Its ...
Google Scholar (GS), a commonly used web-based academic search engine, catalogues between 2 and 100 million records of both academic and grey literature (articles not formally published by commercial academic publishers). Google Scholar collates results from across the internet and is free to use. As a result it has received considerable attention as a method for searching for literature ...
Evaluation of research co-design in health: a systematic overview of
Article CAS Google Scholar Hubbard G, Kidd L, Donaghy E, McDonald C, Kearney N. A review of literature about involving people affected by cancer in research, policy and planning and practice. Patient Educ Couns. 2007;65(1):21-33. Article PubMed Google Scholar
Too stressed to think? A scoping review of the literature for
Google Scholar was also searched as there is a small body of evidence that suggests this search engine produces highly comprehensive results [21, 22] whilst also searching 'grey literature' (published informally, non-commercially or remains unpublished), a format neglected by other databases. Selection of papers for inclusion in the study ...
Health professionals' attitudes towards traditional healing for mental
Additionally, Google and Google Scholar will be searched for other information, including grey literature. Moreover, a manual search of identified articles' reference lists will also be conducted to help ensure all potential papers are included in the review.
A systematic literature review of Islamic inheritance in Malaysia
The study aims to expand the existing body of knowledge by highlighting and summarising current studies on Islamic inheritance. Thus, this study presented a map of the research interest and determined primary issues to demonstrate current trends, future research directions, and theory development. A systematic literature review (SLR) approach was employed using the four-step reporting standard ...

Literature Review: Google Scholar

Ask a Librarian

Google Scholar Library Links

Finding Academic Literature

Literature Review & Research Skills Guide: Use Google Scholar

Using Google Scholar

Google Scholar

Citation Searching

Using Google effectively

Citation searching

How To Conduct A Literature Review Using Google Scholar Step By Step Guide

Step By Step Guide of using Google Scholar for Literature search

One thought on “How To Conduct A Literature Review Using Google Scholar Step By Step Guide”

Leave a Reply Cancel reply

How to write a superb literature review

Access options

Updates & Corrections

Related Articles

Faculty Position

Postdoctoral Associate

Associate or Senior Editor, Nature Energy

Open Rank Tenure-track/Tenured Faculty Positions in the Department of Genetic and Genomic Sciences

Assistant Professor (Tenure Track) of Robotics

Quick links

Have a language expert improve your writing

How to Write a Literature Review | Guide, Examples, & Templates

Instantly correct all language mistakes in your text

Table of contents

Don't submit your assignments before you do this

Make a list of keywords

Search for relevant sources

Take notes and cite your sources

Chronological

Methodological

Theoretical

Cite this Scribbr article

Is this article helpful?

Shona McCombes

Literature Reviews and Annotated Bibliographies

The Literature Review: A Foundation for High-Quality Medical Education Research

The Literature Review Defined

Purpose and Importance of the Literature Review

Approaching the Literature Review

Locating and Organizing the Literature

Human Resources

Search Tools and Related Literature

Box Information Resources

Getting Organized

Knowing When to Say When

Putting It All Together

Health Professions Education for Doctoral Students: Reviewing the Literature

What is a Literature Review?

Search Strategies

Structure of a Literature Review

The Matrix Method

Save citation to file

Add to My Bibliography

The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching

Conflict of interest statement

Similar articles

Publication types

Related information

Other Literature Sources

Research Materials

Start your free trial

Google Scholar: a review of literature examining its effectiveness as a search tool

Lack of curation

Search results rankings

Replicating your search

Using your results

How to search more effectively

Get in touch with IFIS

Evaluation of research co-design in health: a systematic overview of reviews and development of a framework

Conclusions

Contributions to the literature

Introduction

Step 1: A systematic overview of reviews

Search strategy

Study selection

Step 2: Stakeholder panel meetings to discuss and debate findings from the overview of reviews