peer reviewed research article

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Peer Review? | Types & Examples

What Is Peer Review? | Types & Examples

Published on December 17, 2021 by Tegan George . Revised on June 22, 2023.

Peer review, sometimes referred to as refereeing , is the process of evaluating submissions to an academic journal. Using strict criteria, a panel of reviewers in the same subject area decides whether to accept each submission for publication.

Peer-reviewed articles are considered a highly credible source due to the stringent process they go through before publication.

There are various types of peer review. The main difference between them is to what extent the authors, reviewers, and editors know each other’s identities. The most common types are:

• Single-blind review
• Double-blind review
• Triple-blind review

Collaborative review

Open review.

Relatedly, peer assessment is a process where your peers provide you with feedback on something you’ve written, based on a set of criteria or benchmarks from an instructor. They then give constructive feedback, compliments, or guidance to help you improve your draft.

Table of contents

What is the purpose of peer review, types of peer review, the peer review process, providing feedback to your peers, peer review example, advantages of peer review, criticisms of peer review, other interesting articles, frequently asked questions about peer reviews.

Many academic fields use peer review, largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the manuscript. For this reason, academic journals are among the most credible sources you can refer to.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure.

Peer assessment is often used in the classroom as a pedagogical tool. Both receiving feedback and providing it are thought to enhance the learning process, helping students think critically and collaboratively.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Depending on the journal, there are several types of peer review.

Single-blind peer review

The most common type of peer review is single-blind (or single anonymized) review . Here, the names of the reviewers are not known by the author.

While this gives the reviewers the ability to give feedback without the possibility of interference from the author, there has been substantial criticism of this method in the last few years. Many argue that single-blind reviewing can lead to poaching or intellectual theft or that anonymized comments cause reviewers to be too harsh.

Double-blind peer review

In double-blind (or double anonymized) review , both the author and the reviewers are anonymous.

Arguments for double-blind review highlight that this mitigates any risk of prejudice on the side of the reviewer, while protecting the nature of the process. In theory, it also leads to manuscripts being published on merit rather than on the reputation of the author.

Triple-blind peer review

While triple-blind (or triple anonymized) review —where the identities of the author, reviewers, and editors are all anonymized—does exist, it is difficult to carry out in practice.

Proponents of adopting triple-blind review for journal submissions argue that it minimizes potential conflicts of interest and biases. However, ensuring anonymity is logistically challenging, and current editing software is not always able to fully anonymize everyone involved in the process.

In collaborative review , authors and reviewers interact with each other directly throughout the process. However, the identity of the reviewer is not known to the author. This gives all parties the opportunity to resolve any inconsistencies or contradictions in real time, and provides them a rich forum for discussion. It can mitigate the need for multiple rounds of editing and minimize back-and-forth.

Collaborative review can be time- and resource-intensive for the journal, however. For these collaborations to occur, there has to be a set system in place, often a technological platform, with staff monitoring and fixing any bugs or glitches.

Lastly, in open review , all parties know each other’s identities throughout the process. Often, open review can also include feedback from a larger audience, such as an online forum, or reviewer feedback included as part of the final published product.

While many argue that greater transparency prevents plagiarism or unnecessary harshness, there is also concern about the quality of future scholarship if reviewers feel they have to censor their comments.

In general, the peer review process includes the following steps:

• First, the author submits the manuscript to the editor.
• Reject the manuscript and send it back to the author, or
• Send it onward to the selected peer reviewer(s)
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made.
• Lastly, the edited manuscript is sent back to the author. They input the edits and resubmit it to the editor for publication.

In an effort to be transparent, many journals are now disclosing who reviewed each article in the published product. There are also increasing opportunities for collaboration and feedback, with some journals allowing open communication between reviewers and authors.

It can seem daunting at first to conduct a peer review or peer assessment. If you’re not sure where to start, there are several best practices you can use.

Summarize the argument in your own words

Summarizing the main argument helps the author see how their argument is interpreted by readers, and gives you a jumping-off point for providing feedback. If you’re having trouble doing this, it’s a sign that the argument needs to be clearer, more concise, or worded differently.

If the author sees that you’ve interpreted their argument differently than they intended, they have an opportunity to address any misunderstandings when they get the manuscript back.

Separate your feedback into major and minor issues

It can be challenging to keep feedback organized. One strategy is to start out with any major issues and then flow into the more minor points. It’s often helpful to keep your feedback in a numbered list, so the author has concrete points to refer back to.

Major issues typically consist of any problems with the style, flow, or key points of the manuscript. Minor issues include spelling errors, citation errors, or other smaller, easy-to-apply feedback.

Tip: Try not to focus too much on the minor issues. If the manuscript has a lot of typos, consider making a note that the author should address spelling and grammar issues, rather than going through and fixing each one.

The best feedback you can provide is anything that helps them strengthen their argument or resolve major stylistic issues.

Give the type of feedback that you would like to receive

No one likes being criticized, and it can be difficult to give honest feedback without sounding overly harsh or critical. One strategy you can use here is the “compliment sandwich,” where you “sandwich” your constructive criticism between two compliments.

Be sure you are giving concrete, actionable feedback that will help the author submit a successful final draft. While you shouldn’t tell them exactly what they should do, your feedback should help them resolve any issues they may have overlooked.

As a rule of thumb, your feedback should be:

• Easy to understand
• Constructive

Below is a brief annotated research example. You can view examples of peer feedback by hovering over the highlighted sections.

Influence of phone use on sleep

Studies show that teens from the US are getting less sleep than they were a decade ago (Johnson, 2019) . On average, teens only slept for 6 hours a night in 2021, compared to 8 hours a night in 2011. Johnson mentions several potential causes, such as increased anxiety, changed diets, and increased phone use.

The current study focuses on the effect phone use before bedtime has on the number of hours of sleep teens are getting.

For this study, a sample of 300 teens was recruited using social media, such as Facebook, Instagram, and Snapchat. The first week, all teens were allowed to use their phone the way they normally would, in order to obtain a baseline.

The sample was then divided into 3 groups:

• Group 1 was not allowed to use their phone before bedtime.
• Group 2 used their phone for 1 hour before bedtime.
• Group 3 used their phone for 3 hours before bedtime.

All participants were asked to go to sleep around 10 p.m. to control for variation in bedtime . In the morning, their Fitbit showed the number of hours they’d slept. They kept track of these numbers themselves for 1 week.

Two independent t tests were used in order to compare Group 1 and Group 2, and Group 1 and Group 3. The first t test showed no significant difference ( p > .05) between the number of hours for Group 1 ( M = 7.8, SD = 0.6) and Group 2 ( M = 7.0, SD = 0.8). The second t test showed a significant difference ( p < .01) between the average difference for Group 1 ( M = 7.8, SD = 0.6) and Group 3 ( M = 6.1, SD = 1.5).

This shows that teens sleep fewer hours a night if they use their phone for over an hour before bedtime, compared to teens who use their phone for 0 to 1 hours.

Peer review is an established and hallowed process in academia, dating back hundreds of years. It provides various fields of study with metrics, expectations, and guidance to ensure published work is consistent with predetermined standards.

• Protects the quality of published research

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. Any content that raises red flags for reviewers can be closely examined in the review stage, preventing plagiarized or duplicated research from being published.

• Gives you access to feedback from experts in your field

Peer review represents an excellent opportunity to get feedback from renowned experts in your field and to improve your writing through their feedback and guidance. Experts with knowledge about your subject matter can give you feedback on both style and content, and they may also suggest avenues for further research that you hadn’t yet considered.

• Helps you identify any weaknesses in your argument

Peer review acts as a first defense, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process. This way, you’ll end up with a more robust, more cohesive article.

While peer review is a widely accepted metric for credibility, it’s not without its drawbacks.

• Reviewer bias

The more transparent double-blind system is not yet very common, which can lead to bias in reviewing. A common criticism is that an excellent paper by a new researcher may be declined, while an objectively lower-quality submission by an established researcher would be accepted.

• Delays in publication

The thoroughness of the peer review process can lead to significant delays in publishing time. Research that was current at the time of submission may not be as current by the time it’s published. There is also high risk of publication bias , where journals are more likely to publish studies with positive findings than studies with negative findings.

• Risk of human error

By its very nature, peer review carries a risk of human error. In particular, falsification often cannot be detected, given that reviewers would have to replicate entire experiments to ensure the validity of results.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Measures of central tendency
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Thematic analysis
• Discourse analysis
• Cohort study
• Ethnography

Research bias

• Implicit bias
• Cognitive bias
• Conformity bias
• Hawthorne effect
• Availability heuristic
• Attrition bias
• Social desirability bias

Peer review is a process of evaluating submissions to an academic journal. Utilizing rigorous criteria, a panel of reviewers in the same subject area decide whether to accept each submission for publication. For this reason, academic journals are often considered among the most credible sources you can use in a research project– provided that the journal itself is trustworthy and well-regarded.

In general, the peer review process follows the following steps: 

• Reject the manuscript and send it back to author, or 
• Send it onward to the selected peer reviewer(s) 
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made. 
• Lastly, the edited manuscript is sent back to the author. They input the edits, and resubmit it to the editor for publication.

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. It also represents an excellent opportunity to get feedback from renowned experts in your field. It acts as a first defense, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process.

Peer-reviewed articles are considered a highly credible source due to this stringent process they go through before publication.

Many academic fields use peer review , largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the published manuscript.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure. 

A credible source should pass the CRAAP test  and follow these guidelines:

• The information should be up to date and current.
• The author and publication should be a trusted authority on the subject you are researching.
• The sources the author cited should be easy to find, clear, and unbiased.
• For a web source, the URL and layout should signify that it is trustworthy.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). What Is Peer Review? | Types & Examples. Scribbr. Retrieved August 12, 2024, from https://www.scribbr.com/methodology/peer-review/

Is this article helpful?

Tegan George

Other students also liked, what are credible sources & how to spot them | examples, ethical considerations in research | types & examples, applying the craap test & evaluating sources, what is your plagiarism score.

Find Resources for Your Midterms or Finals: Scholarly (Peer-reviewed) Journal Articles

• Scholarly (Peer-reviewed) Journal Articles
• Popular Sources
• Getting Research Help

What is Peer Review?

Peer review is the formal process scholarly journals employ to ensure that a manuscript's writing, methodology, arguments, and conclusions are sound. Peer review has long been a marker of quality that sets scholarly articles apart from popular articles (like those you would find in a magazine or newspaper).

Check out the video below for more information on peer review!

Tutorial: Peer Review

If your browser does not display frames, please use the direct link to the video provided on this page.

• Peer Review

Library Databases

You'll want to use the Pfau Library's databases to access peer-reviewed scholarly journal articles. The library subscribes to these databases, which give you (as a student) FREE access. If you don't use a library database and try to locate articles through a Google Search or by going directly to a journal's website, for example, you'll often hit a paywall and be asked to pay.

• Starter Databases for Finding Articles Try these recommended databases for locating scholarly articles.
• Explore our Databases by Subject Find databases for particular subjects, from anthropology and geography to nursing and world languages.

Getting a Copy of the Actual Article

Library databases often include complete copies of the articles themselves, or full text .  On your results list, look for a link or an icon indicating that full text is available.

If the article is available in any of Pfau Library's databases, or is free on the Web, you'll be given a link to get it.

If the article might be in the library's hard-copy journals, this will be indicated.

If the article isn't available, you'll get a chance to request a copy through Interlibrary Loan.

Database Search Tips

Think of keywords, or important words describing each aspect of your topic, such as:

food insecurity college students

If you are not getting the results you want, think of synonyms or related terms that might get at your topic. For example:

hunger university students

You can search related terms at the same time. To do so, put OR between the related terms, then bracket them off with parentheses like this:

(hunger OR food insecurity)(university students OR college students)

Keep track of the keywords you use! You will want to try the same searches in different databases.

* Be sure to limit your results to peer-reviewed articles .  To do so, select the Scholarly (Peer Reviewed) Journals  or Peer Reviewed  box – most databases have this option. If you're not sure whether what you're seeing is peer-reviewed or not, contact a librarian or your professor. 

Citation Chasing

When you find an article that's on point, check out its citations/references/works cited list. This will likely lead you to other relevant articles. If you have the name of an article you want, the easiest way to get it is to enter the full title in OneSearch.

• OneSearch Try a search in our online catalog!
• << Previous: Home
• Next: Popular Sources >>
• Last Updated: Oct 14, 2022 2:10 PM
• URL: https://libguides.csusb.edu/finals

Navigation group

Home banner.

Where scientists empower society

Creating solutions for healthy lives on a healthy planet.

9.4 million

2.8 billion

article views and downloads

Main Content

• Editors and reviewers
• Collaborators

Find a journal

We have a home for your research. Our community led journals cover more than 1,500 academic disciplines and are some of the largest and most cited in their fields.

Submit your research

Start your submission and get more impact for your research by publishing with us.

Author guidelines

Ready to publish? Check our author guidelines for everything you need to know about submitting, from choosing a journal and section to preparing your manuscript.

Peer review

Our efficient collaborative peer review means you’ll get a decision on your manuscript in an average of 61 days.

Article publishing charges (APCs) apply to articles that are accepted for publication by our external and independent editorial boards

Press office

Visit our press office for key media contact information, as well as Frontiers’ media kit, including our embargo policy, logos, key facts, leadership bios, and imagery.

Institutional partnerships

Join more than 555 institutions around the world already benefiting from an institutional membership with Frontiers, including CERN, Max Planck Society, and the University of Oxford.

Publishing partnerships

Partner with Frontiers and make your society’s transition to open access a reality with our custom-built platform and publishing expertise.

Policy Labs

Connecting experts from business, science, and policy to strengthen the dialogue between scientific research and informed policymaking.

How we publish

All Frontiers journals are community-run and fully open access, so every research article we publish is immediately and permanently free to read.

Editor guidelines

Reviewing a manuscript? See our guidelines for everything you need to know about our peer review process.

Become an editor

Apply to join an editorial board and collaborate with an international team of carefully selected independent researchers.

My assignments

It’s easy to find and track your editorial assignments with our platform, 'My Frontiers' – saving you time to spend on your own research.

Three-step plan to cut overlooked methane emissions could help us stop global warming faster

Methane, the second most important greenhouse gas, has been neglected—but now scientists lay out a new plan and a new tool to cut emissions down to size

Safeguarding peer review to ensure quality at scale

Making scientific research open has never been more important. But for research to be trusted, it must be of the highest quality. Facing an industry-wide rise in fraudulent science, Frontiers has increased its focus on safeguarding quality.

Ecopreneur Prof Thomas Crowther to showcase the power of nature-based solutions at Frontiers Forum virtual event

Visionary ecologist and pioneer in reforestation and ecosystem restoration Prof Thomas Crowther will explore the vital role that nature plays in our fight against climate change, at a unique Frontiers Forum virtual event on 11 September.

Pompeii skeleton discovery shows another natural disaster may have made Vesuvius eruption even more deadly

The death of inhabitants of Pompeii during the eruption of Vesuvius in 79CE is mostly attributed to volcanic causes, such as falling ashes and hot gas. This, however, may not be the full picture.

Dune-inspired upgrade for spacesuits allow astronauts to recycle urine into water

Inspired by the sci-fi series, Dune, Cornell researchers created a prototype novel urine collection and filtration system for spacesuits.

What happens when the ocean’s biggest predators start hunting each other? Here are five Frontiers articles you won’t want to miss

At Frontiers, we bring some of the world’s best research to a global audience. But with tens of thousands of articles published each year, it’s impossible to cover all of them. Here are just five amazing papers you may have missed.

Five Research Topics on the human impact of population growth

We highlight five Research Topics tackling some of the most urgent issues emerging from our planet's growing population.

Get the latest research updates, subscribe to our newsletter

Peer review

A key convention in the publication of research is the peer review process, in which the quality and potential contribution of each manuscript is evaluated by one's peers in the scientific community.

Like other scientific journals, APA journals utilize a peer review process to guide manuscript selection and publication decisions.

Toward the goal of impartiality, the majority of APA journals follow a masked review policy, in which authors' and reviewers' identities are concealed from each other. Reviewer identities are never shared unless the reviewer requests to sign their review.

APA journal reviewers are qualified individuals selected by the action editor (typically, the journal editor or associate editor) to review a manuscript on the basis of their expertise in particular content areas of their field.

The role of a peer reviewer is to highlight unique, original manuscripts that fit within the scope of the journal.

To aid the editor's objectivity, two to three peer reviewers are selected to evaluate a manuscript.

These reviewers should be able to provide fair reviews, free from conflicts of interest, as well as submit the reviews on time.

In addition to technical expertise, criteria for selection of reviewers may include familiarity with a particular controversy or attention to a balance of perspectives (APA, 2010, p. 226).

Whereas the journal editor holds final responsibility for a manuscript, the action editor usually weighs reviewers' inputs heavily.

Authors can expect their manuscripts to be reviewed fairly, in a skilled, conscientious manner. The comments received should be constructive, respectful and specific.

Reviewers must present a clear decision recommendation regarding publication, considering the quality of the manuscript, its scientific contribution, and its appropriateness for the particular journal; support the recommendation with a detailed, comprehensive analysis of the quality and coherence of the study's conceptual basis, methods, results, and interpretations; and offer specific, constructive suggestions to authors.

Journal editors may request that reviewers evaluate manuscripts based on specific criteria, which may vary across journals or for non-empirical article types, such as commentaries or reviews.

The action editor scans the paper to gain an independent view of the work. This "quick read" provides a foundation for the more thorough reading that follows — it by no means determines the final decision, but does parallel how authors can expect many reviewers (and readers) to approach their papers.

First, the editor scans the paper from beginning to end for obvious flaws in the research substance and writing style. If problems show on the surface, a deeper reading is likely to uncover other matters needing attention.

After this initial examination of your manuscript, the action editors, as well as any peer reviewers, will follow these general guidelines:

Read the abstract

Major problems in the abstract often reflect internal flaws.

The major goal in reading the abstract is to understand the research question:

• Is it clearly defined, relevant, and supported by the methodology?
• What is the sense of the research question, methodology, findings, and interpretations?

APA publication policy emphasizes conclusion-oriented abstracts: What did the research find, and what do the findings mean?

Examine the full manuscript

If it is more than 35 typed, double-spaced pages (including references, tables, and figures), this could pose a problem for some journals.

• How long are the Introduction and the Discussion sections relative to other sections of the paper?
• Does the paper adhere to journal-specific guidelines?

These guidelines can be found on the Manuscript Submission tab of each journal's webpage.

Scan the paper's headings

• Are they well organized?
• Does a clear structure emerge?

If not, the author has not achieved coherence.

Scan the references

• Are they in APA Style?

If not, the author is not using APA publication format.

Scan the tables and figures

• Do they portray the information clearly?
• Can they stand alone without captions?
• Are they well constructed and in APA Style?

A "no" to any of these questions suggests problems in the author's presentation of findings.

• If the text contains a large number of statistics, could they be more appropriately put into tables or figures?

The editor drafting the decision letter should be synthesizing the input from multiple reviewers into a cohesive list of improvements that should be made to the manuscript. Any comments from the reviewers will be appended to the official decision letter.

These categories constitute the editorial actions that may be taken on a manuscript.

The flaws that lead to this decision generally center on substantive or methodological issues. A manuscript is usually rejected because it is outside the area of coverage of the journal; it contains serious flaws of design, methodology, analysis, or interpretation; or it is judged to make only a limited novel contribution to the field.

Revise and resubmit

In most cases, manuscripts may have publication potential but are not yet ready for final publication. The study as presented may not merit acceptance as is but may warrant consideration after substantive revision (e.g., reorganizing the conceptual structure, conducting additional experiments, or modifying analyses).

The action editor will give the author an invitation to revise and resubmit for another round of reviews (usually with the same reviewers). An action editor cannot guarantee acceptance of a revised manuscript, but authors who respond flexibly and attend closely to suggested revisions enhance their chances for an acceptance.

Authors must include a detailed cover letter outlining their responses to the revisions. Authors may receive this decision multiple times prior to acceptance.

In very few cases, a manuscript may be accepted for publication on first reading, with only minor revisions required. More typically, acceptances follow the successful revision of a manuscript previously rejected with invitation to revise and resubmit.

Once a manuscript is accepted and appropriate paperwork has been obtained, it enters the production phase of publication. At this point, no further changes can be made by the author other than those suggested by the copyeditor.

• Guidelines for Effective Manuscript Evaluation (from Psychotherapy )
• Peer review ethics: Six things every author should know (from Division Dialogue , March 2018)
• Current Peer Review Trends and Standards

If your manuscript is rejected, and if you believe a pertinent point was overlooked or misunderstood by the reviewers, you may appeal the editorial decision by contacting the editor responsible for the journal.

The editor might then decide to send the appeal to the (associate) editor who handled the initial submission.

If you appeal to the editor and are not satisfied with the editor's response, the next step in the APA editorial appeal procedure is to contact the APA chief editorial advisor .

If a satisfactory resolution is still not achieved, and you still believe that the process was unfair, you may appeal to the Publications and Communications (P&C) Board.

An initial review by the journals publisher and P&C Board chair and chair-elect will determine if the appeal will go before the full board for final decision.

Cases in which an appeal might not go before the full board are those in which an author submitted a manuscript against a journal’s policy (e.g., if a rejected submission was revised and submitted as a new submission without invitation to do so).

• APA publishing resources
• Reviewer Resource Center
• Editor resource center
• Why publish with APA Journals™
• Equity, diversity, and inclusion in APA Journals™

APA Publishing Insider

APA Publishing Insider is a free monthly newsletter with tips on APA Style, open science initiatives, active calls for papers, research summaries, and more.

Visit the APA Style website for style and grammar guidelines, free instructional aids, reference examples, the APA Style blog, APA Style products, and more.

Contact Journals

• USU Library

Articles: Finding (and Identifying) Peer-Reviewed Articles: What is Peer Review?

• What is Peer Review?
• Finding Peer Reviewed Articles
• Databases That Can Determine Peer Review

Peer Review in 3 Minutes

What is "Peer-Review"?

What are they.

Scholarly articles are papers that describe a research study. 

Why are scholarly articles useful?

They report original research projects that have been reviewed by other experts before they are accepted for publication, so you can reasonably be assured that they contain valid information. 

How do you identify scholarly or peer-reviewed articles?

• They are usually fairly lengthy - most likely at least 7-10 pages
• The authors and their credentials should be identified, at least the company or university where the author is employed
• There is usually a list of References or Works Cited at the end of the paper, listing the sources that the authors used in their research

How do you find them? 

Some of the library's databases contain scholarly articles, either exclusively or in combination with other types of articles. 

Google Scholar is another option for searching for scholarly articles. 

Know the Difference Between Scholarly and Popular Journals/Magazines

Peer reviewed articles are found in scholarly journals.  The checklist below can help you determine if what you are looking at is peer reviewed or scholarly.

• Both kinds of journals and magazines can be useful sources of information.
• Popular magazines and newspapers are good for overviews, recent news, first-person accounts, and opinions about a topic.
• Scholarly journals, often called scientific or peer-reviewed journals, are good sources of actual studies or research conducted about a particular topic. They go through a process of review by experts, so the information is usually highly reliable.

• Next: Finding Peer Reviewed Articles >>
• Last Updated: May 21, 2024 8:45 AM
• URL: https://libguides.usu.edu/peer-review

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

• Library databases
• Library website

Evaluating Resources: Peer Review

What is peer review.

The term peer review can be confusing, since in some of your courses you may be asked to review the work of your peers. When we talk about peer-reviewed journal articles, this has nothing to do with your peers!

Peer-reviewed journals, also called refereed journals, are journals that use a specific scholarly review process to try to ensure the accuracy and reliability of published articles. When an article is submitted to a peer-reviewed journal for publication, the journal sends the article to other scholars/experts in that field and has them review the article for accuracy and reliability.

Find out more about peer review with our Peer Review Guide:

• Peer Review Guide

Types of peer review

Single blind.

In this process, the names of the reviewers are not known to the author(s). The reviewers do know the name of the author(s).

Double blind

Here, neither reviewers or authors know each other's names.

In the open review process, both reviewers and authors know each other's names.

What about editorial review?

Journals also use an editorial review process. This is not the same as peer review. In an editorial review process an article is evaluated for style guidelines and for clarity. Reviewers here do not look at technical accuracy or errors in data or methodology, but instead look at grammar, style, and whether an article is well written.

What is the difference between scholarly and peer review?

Not all scholarly journals are peer reviewed, but all peer-reviewed journals are scholarly.

• Things that are written for a scholarly or academic audience are considered scholarly writing.
• Peer-reviewed journals are a part of the larger category of scholarly writing.
• Scholarly writing includes many resources that are not peer reviewed, such as books, textbooks, and dissertations.

Scholarly writing does not come with a label that says scholarly . You will need to evaluate the resource to see if it is

• aimed at a scholarly audience
• reporting research, theories or other types of information important to scholars
• documenting and citing sources used to help authenticate the research done

The standard peer review process only applies to journals. While scholarly writing has certainly been edited and reviewed, peer review is a specific process only used by peer-reviewed journals. Books and dissertations may be scholarly, but are not considered peer reviewed.

Check out Select the Right Source for help with what kinds of resources are appropriate for discussion posts, assignments, projects, and more:

• Select the Right Source

How do I locate or verify peer-reviewed articles?

The peer review process is initiated by the journal publisher before an article is even published. Nowhere in the article will it tell you whether or not the article has gone through a peer review process.

You can locate peer-reviewed articles in the Library databases, typically by checking a limiter box.

• Quick Answer: How do I find scholarly, peer reviewed journal articles?

You can verify whether a journal uses a peer review process by using Ulrich's Periodicals Directory.

• Quick Answer: How do I verify that my article is peer reviewed?

What about resources that are not peer-reviewed?

Limiting your search to peer review is a way that you can ensure that you're looking at scholarly journal articles, and not popular or trade publications. Because peer-reviewed articles have been vetted by experts in the field, they are viewed as being held to a higher standard, and therefore are considered to be a high quality source. Professors often prefer peer-reviewed articles because they are considered to be of higher quality.

There are times, though, when the information you need may not be available in a peer-reviewed article.

• You may need to find original work on a theory that was first published in a book.
• You may need to find very current statistical data that comes from a government website.
• You may need background information that comes from a scholarly encyclopedia.

You will want to evaluate these resources to make sure that they are the best source for the information you need.

Note: If you are required for an assignment to find information from a peer-reviewed journal, then you will not be able to use non-peer-reviewed sources such as books, dissertations, or government websites. It's always best to clarify any questions over assignments with your professor.

• Previous Page: Evaluation Methods
• Next Page: Primary & Secondary Sources
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

Preserving the Quality of Scientific Research: Peer Review of Research Articles

• First Online: 20 January 2017

Cite this chapter

• Pali U. K. De Silva 3 &
• Candace K. Vance 3  

Part of the book series: Fascinating Life Sciences ((FLS))

1414 Accesses

10 Citations

Peer review of scholarly articles is a mechanism used to assess and preserve the trustworthiness of reporting of scientific findings. Since peer reviewing is a qualitative evaluation system that involves the judgment of experts in a field about the quality of research performed by their colleagues (and competitors), it inherently encompasses a strongly subjective element. Although this time-tested system, which has been evolving since the mid-eighteenth century, is being questioned and criticized for its deficiencies, it is still considered an integral part of the scholarly communication system, as no other procedure has been proposed to replace it. Therefore, to improve and strengthen the existing peer review process, it is important to understand its shortcomings and to continue the constructive deliberations of all participants within the scientific scholarly communication system . This chapter discusses the strengths, issues, and deficiencies of the peer review system, conventional closed models (single-blind and double-blind), and the new open peer review model and its variations that are being experimented with by some journals.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Ensuring the Quality, Fairness, and Integrity of Journal Peer Review: A Possible Role of Editors

The limitations to our understanding of peer review, reviewing peer review: a flawed system: with immense potential.

Evaluative criteria may also vary depending on the scope of the specific journal.

Krebs and Johnson ( 1937 ).

McClintock ( 1950 ).

Bombardier et al. ( 2000 ).

“Nature journals offer double-blind review” Nature announcement— http://www.nature.com/news/nature-journals-offer-double-blind-review-1.16931 .

Contains all versions of the manuscript, named reviewer reports, author responses, and (where relevant) editors’ comments (Moylan et al. 2014 ).

https://www.elsevier.com/about/press-releases/research-and-journals/peer-review-survey-2009-preliminary-findings .

Review guidelines, Frontiers in Neuroscience http://journal.frontiersin.org/journal/synaptic-neuroscience#review-guidelines .

Editorial policies - BioMed Central  http://www.biomedcentral.com/getpublished/editorial-policies#peer+review .

Hydrology and Earth System Sciences Interactive Public Peer Review  http://www.hydrology-and-earth-system-sciences.net/peer_review/interactive_review_process.html .

Copernicus Publications  http://publications.copernicus.org/services/public_peer_review.html .

Copernicus Publications - Interactive Public Peer Review  http://home.frontiersin.org/about/impact-and-tiering .

Biology Direct http://www.biologydirect.com/ .

F1000 Research  http://f1000research.com .

GigaScience  http://www.gigasciencejournal.com

Journal of Negative Results in Biomedicine  http://www.jnrbm.com/ .

BMJOpen  http://bmjopen.bmj.com/ .

PeerJ  http://peerj.com/ .

ScienceOpen  https://www.scienceopen.com .

ArXiv  http://arxiv.org .

Retraction of articles from Springer journals. London: Springer, August 18, 2015 ( http://www.springer.com/gp/about-springer/media/statements/retraction-of-articles-from-springer-journals/735218 ).

COPE statement on inappropriate manipulation of peer review processes ( http://publicationethics.org/news/cope-statement-inappropriate-manipulation-peer-review-processes ).

Hindawi concludes an in-depth investigation into peer review fraud, July 2015 ( http://www.hindawi.com/statement/ ).

.Wakefield, A. J., Murch, S. H., Anthony, A., Linnell, J., Casson, D. M., Malik, M., ... & Valentine, A. (1998). Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet , 351 (9103), 637–641. (RETRACTED:See The Lancet 375 (9713) p.445)

A practice used by researchers to increase the number of articles publishing multiple papers using very similar pieces of a single dataset. The drug industry also uses this tactic to increase publications with positive findings on their products.

Neuroscience Peer Reviewer Consortium  http://nprc.incf.org/ .

“About 80% of submitted manuscripts are rejected during this initial screening stage, usually within one week to 10 days.” http://www.sciencemag.org/site/feature/contribinfo/faq/ (accessed on October 18, 2016); “Nature has space to publish only 8% or so of the 200 papers submitted each week” http://www.nature.com/nature/authors/get_published/ (accessed on October 18, 2016).

Code of Conduct and Best Practice Guidelines for Journal Editors  http://publicationethics.org/files/Code%20of%20Conduct_2.pdf .

Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals  http://www.icmje.org/icmje-recommendations.pdf .

Alberts, B., Hanson, B., & Kelner, K. L. (2008). Reviewing peer review. Science, 321 (5885), 15.

Article   PubMed   Google Scholar  

Ali, P. A., & Watson, R. (2016). Peer review and the publication process. Nursing Open . doi: 10.1002/nop2.51 .

Baggs, J. G., Broome, M. E., Dougherty, M. C., Freda, M. C., & Kearney, M. H. (2008). Blinding in peer review: the preferences of reviewers for nursing journals. Journal of Advanced Nursing, 64 (2), 131–138.

Bjork, B.-C., Roos, A., & Lauri, M. (2009). Scientific journal publishing: yearly volume and open access availability. Information Research: An International Electronic Journal, 14 (1).

Google Scholar  

Bohannon, J. (2013). Who’s afraid of peer review. Science, 342 (6154).

Boldt, A. (2011). Extending ArXiv. org to achieve open peer review and publishing. Journal of Scholarly Publishing, 42 (2), 238–242.

Article   Google Scholar  

Bombardier, C., Laine, L., Reicin, A., Shapiro, D., Burgos-Vargas, R., Davis, B., … & Kvien, T. K. (2000). Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis. New England Journal of Medicine, 343 (21), 1520–1528

Bornmann, L. (2011). Scientific peer review. Annual Review of Information Science and Technology, 45 (1), 197–245.

Bornmann, L., & Daniel, H.-D. (2009). Reviewer and editor biases in journal peer review: An investigation of manuscript refereeing at Angewandte Chemie International Edition. Research Evaluation, 18 (4), 262–272.

Bornmann, L., & Daniel, H. D. (2010). Reliability of reviewers’ ratings when using public peer review: A case study. Learned Publishing, 23 (2), 124–131.

Bornmann, L., Mutz, R., & Daniel, H.-D. (2007). Gender differences in grant peer review: A meta-analysis. Journal of Informetrics, 1 (3), 226–238.

Borsuk, R. M., Aarssen, L. W., Budden, A. E., Koricheva, J., Leimu, R., Tregenza, T., et al. (2009). To name or not to name: The effect of changing author gender on peer review. BioScience, 59 (11), 985–989.

Bosch, X., Pericas, J. M., Hernández, C., & Doti, P. (2013). Financial, nonfinancial and editors’ conflicts of interest in high-impact biomedical journals. European Journal of Clinical Investigation, 43 (7), 660–667.

Brown, R. J. C. (2007). Double anonymity in peer review within the chemistry periodicals community. Learned Publishing, 20 (2), 131–137.

Budden, A. E., Tregenza, T., Aarssen, L. W., Koricheva, J., Leimu, R., & Lortie, C. J. (2008). Double-blind review favours increased representation of female authors. Trends in Ecology & Evolution, 23 (1), 4–6.

Burnham, J. C. (1990). The evolution of editorial peer review. JAMA, 263 (10), 1323–1329.

Article   CAS   PubMed   Google Scholar  

Callaham, M. L., & Tercier, J. (2007). The relationship of previous training and experience of journal peer reviewers to subsequent review quality. PLoS Med, 4 (1), e40.

Article   PubMed   PubMed Central   Google Scholar  

Campbell, P. (2006). Peer Review Trial and Debate. Nature .  http://www.nature.com/nature/peerreview/debate/

Campbell, P. (2008). Nature peer review trial and debate. Nature: International Weekly Journal of Science, 11

Campos-Arceiz, A., Primack, R. B., & Koh, L. P. (2015). Reviewer recommendations and editors’ decisions for a conservation journal: Is it just a crapshoot? And do Chinese authors get a fair shot? Biological Conservation, 186, 22–27.

Cantor, M., & Gero, S. (2015). The missing metric: Quantifying contributions of reviewers. Royal Society open science, 2 (2), 140540.

CDC. (2016). Measles: Cases and Outbreaks. Retrieved from http://www.cdc.gov/measles/cases-outbreaks.html

Ceci, S. J., & Williams, W. M. (2011). Understanding current causes of women’s underrepresentation in science. Proceedings of the National Academy of Sciences, 108 (8), 3157–3162.

Article   CAS   Google Scholar  

Chan, A. W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA, 291 (20), 2457–2465.

Charlton, B. G. (2004). Conflicts of interest in medical science: peer usage, peer review andCoI consultancy’. Medical Hypotheses, 63 (2), 181–186.

Cressey, D. (2014). Journals weigh up double-blind peer review. Nature news .

Dalton, R. (2001). Peers under pressure. Nature, 413 (6852), 102–104.

DeVries, D. R., Marschall, E. A., & Stein, R. A. (2009). Exploring the peer review process: What is it, does it work, and can it be improved? Fisheries, 34 (6), 270–279. doi: 10.1577/1548-8446-34.6.270

Emerson, G. B., Warme, W. J., Wolf, F. M., Heckman, J. D., Brand, R. A., & Leopold, S. S. (2010). Testing for the presence of positive-outcome bias in peer review: A randomized controlled trial. Archives of Internal Medicine, 170 (21), 1934–1939.

Fanelli, D. (2010). Do pressures to publish increase scientists’ bias? An empirical support from US States Data. PLoS ONE, 5 (4), e10271.

Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, 109 (42), 17028–17033. doi: 10.1073/pnas.1212247109

Ferguson, C., Marcus, A., & Oransky, I. (2014). Publishing: The peer-review scam. Nature, 515 (7528), 480.

Ford, E. (2015). Open peer review at four STEM journals: An observational overview. F1000Research, 4 .

Fountain, H. (2014). Science journal pulls 60 papers in peer-review fraud. Science, 3, 06.

Freda, M. C., Kearney, M. H., Baggs, J. G., Broome, M. E., & Dougherty, M. (2009). Peer reviewer training and editor support: Results from an international survey of nursing peer reviewers. Journal of Professional Nursing, 25 (2), 101–108.

Gillespie, G. W., Chubin, D. E., & Kurzon, G. M. (1985). Experience with NIH peer review: Researchers’ cynicism and desire for change. Science, Technology and Human Values, 10 (3), 44–54.

Greaves, S., Scott, J., Clarke, M., Miller, L., Hannay, T., Thomas, A., et al. (2006). Overview: Nature’s peer review trial. Nature , 10.

Grieneisen, M. L., & Zhang, M. (2012). A comprehensive survey of retracted articles from the scholarly literature. PLoS ONE, 7 (10), e44118.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Grivell, L. (2006). Through a glass darkly. EMBO Reports, 7 (6), 567–570.

Harrison, C. (2004). Peer review, politics and pluralism. Environmental Science & Policy, 7 (5), 357–368.

Hartog, C. S., Kohl, M., & Reinhart, K. (2011). A systematic review of third-generation hydroxyethyl starch (HES 130/0.4) in resuscitation: Safety not adequately addressed. Anesthesia and Analgesia, 112 (3), 635–645.

Hojat, M., Gonnella, J. S., & Caelleigh, A. S. (2003). Impartial judgment by the “gatekeepers” of science: Fallibility and accountability in the peer review process. Advances in Health Sciences Education, 8 (1), 75–96.

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Med, 2 (8), e124.

James, M. J., Cook-Johnson, R. J., & Cleland, L. G. (2007). Selective COX-2 inhibitors, eicosanoid synthesis and clinical outcomes: A case study of system failure. Lipids, 42 (9), 779–785.

Janssen, S. J., Bredenoord, A. L., Dhert, W., de Kleuver, M., Oner, F. C., & Verlaan, J.-J. (2015). Potential conflicts of interest of editorial board members from five leading spine journals. PLoS ONE, 10 (6), e0127362.

Jefferson, T., Alderson, P., Wager, E., & Davidoff, F. (2002). Effects of editorial peer review: A systematic review. JAMA, 287 (21), 2784–2786.

Jelicic, M., & Merckelbach, H. (2002). Peer-review: Let’s imitate the lawyers! Cortex, 38 (3), 406–407.

Jinha, A. E. (2010). Article 50 million: An estimate of the number of scholarly articles in existence. Learned Publishing, 23 (3), 258–263.

Khan, K. (2010). Is open peer review the fairest system? No. Bmj, 341, c6425.

Kilwein, J. H. (1999). Biases in medical literature. Journal of Clinical Pharmacy and Therapeutics, 24 (6), 393–396.

Koonin, E. V., Landweber, L. F., & Lipman, D. J. (2013). Biology direct: Celebrating 7 years of open, published peer review. Biology direct, 8 (1), 1.

Kozlowski, L. T. (2016). Coping with the conflict-of-interest pandemic by listening to and doubting everyone, including yourself. Science and Engineering Ethics, 22 (2), 591–596.

Krebs, H. A., & Johnson, W. A. (1937). The role of citric acid in intermediate metabolism in animal tissues. Enzymologia, 4, 148–156.

CAS   Google Scholar  

Kriegeskorte, N., Walther, A., & Deca, D. (2012). An emerging consensus for open evaluation: 18 visions for the future of scientific publishing. Beyond open access: Visions for open evaluation of scientific papers by post-publication peer review , 5.

Langfeldt, L. (2006). The policy challenges of peer review: Managing bias, conflict of interests and interdisciplinary assessments. Research Evaluation, 15 (1), 31–41.

Lawrence, P. A. (2003). The politics of publication. Nature, 422 (6929), 259–261.

Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64 (1), 2–17.

Lippert, S., Callaham, M. L., & Lo, B. (2011). Perceptions of conflict of interest disclosures among peer reviewers. PLoS ONE, 6 (11), e26900.

Link, A. M. (1998). US and non-US submissions: an analysis of reviewer bias. Jama , 280 (3), 246–247.

Lo, B., & Field, M. J. (Eds.). (2009). Conflict of interest in medical research, education, and practice . Washington, D.C.: National Academies Press.

Loonen, M. P. J., Hage, J. J., & Kon, M. (2005). Who benefits from peer review? An analysis of the outcome of 100 requests for review by Plastic and Reconstructive Surgery. Plastic and Reconstructive Surgery, 116 (5), 1461–1472.

Luukkonen, T. (2012). Conservatism and risk-taking in peer review: Emerging ERC practices. Research Evaluation , rvs001.

McClintock, B. (1950). The origin and behavior of mutable loci in maize. Proceedings of the National Academy of Sciences, 36 (6), 344–355.

McCullough, J. (1989). First comprehensive survey of NSF applicants focuses on their concerns about proposal review. Science, Technology and Human Values, 14 (1), 78–88.

McIntyre, W. F., & Evans, G. (2014). The Vioxx ® legacy: Enduring lessons from the not so distant past. Cardiology Journal, 21 (2), 203–205.

Moylan, E. C., Harold, S., O’Neill, C., & Kowalczuk, M. K. (2014). Open, single-blind, double-blind: Which peer review process do you prefer? BMC Pharmacology and Toxicology, 15 (1), 1.

Mulligan, A., Hall, L., & Raphael, E. (2013). Peer review in a changing world: An international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology, 64 (1), 132–161.

Nath, S. B., Marcus, S. C., & Druss, B. G. (2006). Retractions in the research literature: misconduct or mistakes? Medical Journal of Australia, 185 (3), 152.

PubMed   Google Scholar  

Nature Editorial (2008). Working double-blind. Nature, 451, 605–606.

Nature Neuroscience Editorial. (2006). Women in neuroscience: A numbers game. Nature Neuroscience, 9, 853.

Okike, K., Hug, K. T., Kocher, M. S., & Leopold, S. S. (2016). Single-blind vs double-blind peer review in the setting of author prestige. JAMA, 316 (12), 1315–1316.

Olson, C. M., Rennie, D., Cook, D., Dickersin, K., Flanagin, A., Hogan, J. W., … & Pace, B. (2002). Publication bias in editorial decision making. JAMA, 287 (21), 2825–2828.

Palmer, A. R. (2000). Quasireplication and the contract of error: lessons from sex ratios, heritabilities and fluctuating asymmetry. Annual Review of Ecology and Systematics , 441–480.

Peters, D. P., & Ceci, S. J. (1982). Peer-review practices of psychological journals: The fate of published articles, submitted again. Behavioral and Brain Sciences, 5 (02), 187–195.

PLOS MED Editors. (2008). Making sense of non-financial competing interests. PLOS Med, 5 (9), e199.

Pulverer, B. (2010). Transparency showcases strength of peer review. Nature, 468 (7320), 29–31.

Pöschl, U., & Koop, T. (2008). Interactive open access publishing and collaborative peer review for improved scientific communication and quality assurance. Information Services & Use, 28 (2), 105–107.

Relman, A. S. (1985). Dealing with conflicts of interest. New England Journal of Medicine, 313 (12), 749–751.

Rennie, J., & Chief, I. N. (2002). Misleading math about the Earth. Scientific American, 286 (1), 61.

Resch, K. I., Ernst, E., & Garrow, J. (2000). A randomized controlled study of reviewer bias against an unconventional therapy. Journal of the Royal Society of Medicine, 93 (4), 164–167.

CAS   PubMed   PubMed Central   Google Scholar  

Resnik, D. B., & Elmore, S. A. (2016). Ensuring the quality, fairness, and integrity of journal peer review: A possible role of editors. Science and Engineering Ethics, 22 (1), 169–188.

Ross, J. S., Gross, C. P., Desai, M. M., Hong, Y., Grant, A. O., Daniels, S. R., et al. (2006). Effect of blinded peer review on abstract acceptance. JAMA, 295 (14), 1675–1680.

Sandström, U. (2009, BRAZIL. JUL 14-17, 2009). Cognitive bias in peer review: A new approach. Paper presented at the 12th International Conference of the International-Society-for-Scientometrics-and-Informetrics.

Shatz, D. (2004). Peer review: A critical inquiry . Lanham, MD: Rowman & Littlefield.

Schneider, L. (2016, September 4). Beall-listed Frontiers empire strikes back. Retrieved from https://forbetterscience.wordpress.com/2016/09/14/beall-listed-frontiers-empire-strikes-back/

Schroter, S., Black, N., Evans, S., Carpenter, J., Godlee, F., & Smith, R. (2004). Effects of training on quality of peer review: Randomised controlled trial. BMJ, 328 (7441), 673.

Service, R. F. (2002). Scientific misconduct. Bell Labs fires star physicist found guilty of forging data. Science (New York, NY), 298 (5591), 30.

Shimp, C. P. (2004). Scientific peer review: A case study from local and global analyses. Journal of the Experimental Analysis of Behavior, 82 (1), 103–116.

Article   PubMed Central   Google Scholar  

Smith, R. (1999). Opening up BMJ peer review: A beginning that should lead to complete transparency. BMJ, 318, 4–5.

Smith, R. (2006). Peer review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99 (4), 178–182. doi: 10.1258/jrsm.99.4.178

Souder, L. (2011). The ethics of scholarly peer review: A review of the literature. Learned Publishing, 24 (1), 55–72.

Spielmans, G. I., Biehn, T. L., & Sawrey, D. L. (2009). A case study of salami slicing: pooled analyses of duloxetine for depression. Psychotherapy and Psychosomatics, 79 (2), 97–106.

Spier, R. (2002). The history of the peer-review process. Trends in Biotechnology, 20 (8), 357–358.

Squazzoni, F. (2010). Peering into peer review. Sociologica, 4 (3).

Squazzoni, F., & Gandelli, C. (2012). Saint Matthew strikes again: An agent-based model of peer review and the scientific community structure. Journal of Informetrics, 6 (2), 265–275.

Steen, R. G. (2010). Retractions in the scientific literature: is the incidence of research fraud increasing? Journal of Medical Ethics , jme-2010.

Stroebe, W., Postmes, T., & Spears, R. (2012). Scientific misconduct and the myth of self-correction in science. Perspectives on Psychological Science, 7 (6), 670–688.

Tite, L., & Schroter, S. (2007). Why do peer reviewers decline to review? A survey. Journal of Epidemiology and Community Health, 61 (1), 9–12.

Travis, G. D. L., & Collins, H. M. (1991). New light on old boys: cognitive and institutional particularism in the peer review system. Science, Technology and Human Values, 16 (3), 322–341.

Tregenza, T. (2002). Gender bias in the refereeing process? Trends in Ecology & Evolution, 17 (8), 349–350.

Valkonen, L., & Brooks, J. (2011). Gender balance in Cortex acceptance rates. Cortex, 47 (7), 763–770.

van Rooyen, S., Delamothe, T., & Evans, S. J. W. (2010). Effect on peer review of telling reviewers that their signed reviews might be posted on the web: Randomised controlled trial. BMJ, 341, c5729.

van Rooyen, S., Godlee, F., Evans, S., Black, N., & Smith, R. (1999). Effect of open peer review on quality of reviews and on reviewers’ recommendations: A randomised trial. British Medical Journal, 318 (7175), 23–27.

Walker, R., & Rocha da Silva, P. (2014). Emerging trends in peer review—A survey. Frontiers in neuroscience, 9, 169.

Walsh, E., Rooney, M., Appleby, L., & Wilkinson, G. (2000). Open peer review: A randomised controlled trial. The British Journal of Psychiatry, 176 (1), 47–51.

Walters, W. P., & Bajorath, J. (2015). On the evolving open peer review culture for chemical information science. F1000Research, 4 .

Ware, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community-Results from an international study. Information Services and Use, 28 (2), 109–112.

Ware, M. (2011). Peer review: Recent experience and future directions. New Review of Information Networking , 16 (1), 23–53.

Webb, T. J., O’Hara, B., & Freckleton, R. P. (2008). Does double-blind review benefit female authors? Heredity, 77, 282–291.

Wellington, J., & Nixon, J. (2005). Shaping the field: The role of academic journal editors in the construction of education as a field of study. British Journal of Sociology of Education, 26 (5), 643–655.

Whittaker, R. J. (2008). Journal review and gender equality: A critical comment on Budden et al. Trends in Ecology & Evolution, 23 (9), 478–479.

Wiedermann, C. J. (2016). Ethical publishing in intensive care medicine: A narrative review. World Journal of Critical Care Medicine, 5 (3), 171.

Download references

Author information

Authors and affiliations.

Murray State University, Murray, Kentucky, USA

Pali U. K. De Silva & Candace K. Vance

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Pali U. K. De Silva .

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

De Silva, P.U.K., K. Vance, C. (2017). Preserving the Quality of Scientific Research: Peer Review of Research Articles. In: Scientific Scholarly Communication. Fascinating Life Sciences. Springer, Cham. https://doi.org/10.1007/978-3-319-50627-2_6

Download citation

DOI : https://doi.org/10.1007/978-3-319-50627-2_6

Published : 20 January 2017

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-50626-5

Online ISBN : 978-3-319-50627-2

eBook Packages : Biomedical and Life Sciences Biomedical and Life Sciences (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Evidence-Based Health Care

• Learning About EBP
• The 6S Pyramid
• Forming Questions
• Identifying Peer-Reviewed Resources
• Critical Appraisal

What is Peer Review?

If an article is peer reviewed , it was reviewed by scholars who are experts in related academic or professional fields before it was published. Those scholars assessed the quality of the article's research, as well as its overall contribution to the literature in their field. 

When we talk about peer-reviewed journals , we're referring to journals that use a peer-review process.

Related terms you might hear include: 

• Academic: Intended for academic use, or an academic audience. 
• Scholarly:  Intended for scholarly use, or a scholarly audience. 
• Refereed: Refers to a specific kind of peer-review process. 

National University Library System. (2018). "Find Articles: How to Find Scholarly/Peer-Reviewed Articles". Retrieved from: http://nu.libguides.com/articles/PR.

How Peer Review Works

 Here's how it typically works:

• Submission : An author submits their research paper or article to a scholarly journal for publication consideration.
• Editorial Assessment : The journal's editor(s) review the submission to determine if it meets the journal's scope, standards, and criteria for publication. They may reject it outright if it doesn't meet these criteria.
• Peer Review : If the submission passes the initial editorial assessment, it is sent out to experts in the field, known as "peers" or "referees," for thorough evaluation. These experts are typically researchers or scholars who have expertise in the subject matter of the submitted work but are not directly affiliated with the author.
• Peer Feedback : The peer reviewers carefully examine the submission for its originality, significance, methodology, accuracy, and overall quality. They provide detailed feedback, critiques, and suggestions for improvement to the journal's editor(s).
• Editorial Decision : Based on the feedback from the peer reviewers, the editor(s) make a decision on whether to accept the submission for publication, request revisions from the author(s) to address specific concerns, or reject it if it does not meet the journal's standards.
• Revision and Resubmission (if applicable): If revisions are requested, the author(s) revise their work in response to the reviewers' feedback and resubmit it to the journal. The revised version may undergo further rounds of peer review until it meets the journal's requirements.
• Publication : Once the submission has successfully passed peer review and any necessary revisions, it is accepted for publication and included in the journal's forthcoming issue.

Peer review serves as a critical checkpoint in the academic publishing process, helping to ensure that only high-quality, rigorously researched, and credible scholarly work is disseminated to the academic community and the public. It helps to uphold standards of academic integrity, accuracy, and reliability.

How Do I Know If a Journal is Peer-Reviewed?

The easiest way to find out if a journal is peer-reviewed is to search for the title in a serials directory like UlrichsWeb:

• UlrichsWeb Global Serials Directory Includes in each record: ISBN, title, publisher, country of publication, status (Active, ceased, etc.), start year, frequency, refereed (Yes/No), media, language, price, subject, Dewey #, circulation, editor(s), email, URL, brief description Also known as: Ulrichs

How to Use Ulrichs

1. Type the name of the journal in the search bar and click the search button. NOTE : you need to use the full name of the journal, not an abbreviation.

2. Locate the journal in the results list. You may see multiple entries for one journal because Ulrichs lists print, electronic, and international version separately.​​​​​​​

Other Techniques for Determining Peer Review Status

Determining whether an article has been peer-reviewed without a service like Ulrichs typically involves a few steps:

• Journal Reputation: Look at the journal where the article is published. Reputable academic journals usually have a peer-review process in place. Check the journal's website or databases like PubMed, Scopus, or Web of Science to see if it's peer-reviewed.
• Article Information: Sometimes, journals explicitly state whether articles undergo peer review. This information can usually be found on the journal's website, alongside other details about submission and publication processes.
• Author Guidelines: Journals often provide authors with guidelines that include information about the peer-review process. Authors are usually instructed to submit their work for peer review as part of the publication process.
• Editorial Policies: Review the journal's editorial policies. Peer-reviewed journals typically have detailed descriptions of their review processes, including how they select reviewers, criteria for acceptance, and timelines for review.
• Check the Article Itself: While this is not always conclusive, some peer-reviewed articles will include a statement indicating that the article has undergone peer review. Look for phrases like "peer-reviewed" and "refereed."
• Indexing Databases: Many indexing databases only include peer-reviewed journals in their listings. If you find the article indexed in databases like PubMed, you can generally assume it has been peer-reviewed.

Remember that while these methods can help you determine whether an article has undergone peer review, it's always good practice to critically evaluate the content of the article regardless of its peer-review status.

How Do I Know If an Article is Peer-Reviewed?

Even if an article was published in a peer-reviewed journal, it may not necessarily be peer-reviewed itself; for example, a commentary article may undergo editorial review instead, meaning it was only reviewed by the journal editor.

There are some clues you can look for to help you identify if an article is peer-reviewed:

• Does the abstract discuss the author's/authors' research process?
• Does the abstract include a variation of the phrase "This study..."?
• Is there a Methodology or Data header in the text of the article?
• Does the paper discuss related research in a literature review?
• Is there an analysis of a need for further research, or gaps in the literature?
• Are the references for scholarly articles and books?

If an article published in a verified peer-reviewed journal includes these elements, it is most likely a peer-reviewed article.

• National University Library: Scholarly Checklist Use this printable checklist to help you identify scholarly, research-based articles

Identifying Peer Reviewed Materials in Scholarly Databases

Peer reviewed material in pubmed and medline.

Good news! Most of the journals in Medline and PubMed are peer reviewed.  Generally speaking, if you find a journal citation in Medline and PubMed you should be just fine. However, there is no way to limit your results within PubMed or the Medline EBSCO interface to knock out the few publications that are not considered refereed titles.

However, EBSCO (a third-party vendor) does provide a list of all titles within Medline and lets you see which titles are considered peer reviewed. You can check if your journal is OK - see the "Peer Review" tab in the report below to see the very small list of titles that don't make the cut.

• Medline: List of Full-Text Journals These journals cover a wide range of subjects within the biomedical and health fields containing information needed by doctors, nurses, health professionals, and researchers engaged in clinical care, public health, and health policy development. Information on peer-reviewed status available within table of titles.

Peer Reviewed Material in CINAHL & PsycINFO

In CINAHL and PsycINFO, there is a "Peer Reviewed" box in the advanced search, which allows you to limit your search results to those that have been identified as peer reviewed.

• View the Title List for CINAHL Complete This page links to the full title list for CINAHL Complete in both Excel and HTML formats.
• << Previous: Forming Questions
• Next: Critical Appraisal >>
• Last Updated: Aug 6, 2024 9:06 AM
• URL: https://libguides.libraries.wsu.edu/EBHC

• University of Texas Libraries
• UT Libraries

Finding Journal Articles 101

Peer-reviewed or refereed.

• Research Article
• Review Article
• By Journal Title

What Does "Peer-reviewed" or "Refereed" Mean?

Peer review is a process that journals use to ensure the articles they publish represent the best scholarship currently available. When an article is submitted to a peer reviewed journal, the editors send it out to other scholars in the same field (the author's peers) to get their opinion on the quality of the scholarship, its relevance to the field, its appropriateness for the journal, etc.

Publications that don't use peer review (Time, Cosmo, Salon) just rely on the judgment of the editors whether an article is up to snuff or not. That's why you can't count on them for solid, scientific scholarship.

Note:This is an entirely different concept from " Review Articles ."

How do I know if a journal publishes peer-reviewed articles?

Usually, you can tell just by looking. A scholarly journal is visibly different from other magazines, but occasionally it can be hard to tell, or you just want to be extra-certain. In that case, you turn to Ulrich's Periodical Directory Online . Just type the journal's title into the text box, hit "submit," and you'll get back a report that will tell you (among other things) whether the journal contains articles that are peer reviewed, or, as Ulrich's calls it, Refereed.

Remember, even journals that use peer review may have some content that does not undergo peer review. The ultimate determination must be made on an article-by-article basis.

For example, the journal  Science  publishes  a mix  of peer-reviewed and non-peer-reviewed content. Here are two articles from the same issue of  Science . 

This one is not peer-reviewed:  https://science-sciencemag-org.ezproxy.lib.utexas.edu/content/303/5655/154.1  This one is a peer-reviewed research article:  https://science-sciencemag-org.ezproxy.lib.utexas.edu/content/303/5655/226

That is consistent with the Ulrichsweb  description of  Science , which states, "Provides news of recent international developments and research in all fields of science. Publishes original research results, reviews and short features."

Test these periodicals in Ulrichs :

• Advances in Dental Research
• Clinical Anatomy
• Molecular Cancer Research
• Journal of Clinical Electrophysiology
• Last Updated: Aug 28, 2023 9:25 AM
• URL: https://guides.lib.utexas.edu/journalarticles101

• Collections
• Services & Support

Which Source Should I Use?

• The Right Source For Your Need-Authority
• Finding Subject Specific Sources: Research Guides
• Understanding Peer Reviewed Articles
• Understanding Peer Reviewed Articles- Arts & Humanities
• How to Read a Journal Article
• Locating Journals
• How to Find Streaming Media

The Peer Review Process

So you need to use scholarly, peer-reviewed articles for an assignment...what does that mean? 

Peer review  is a process for evaluating research studies before they are published by an academic journal. These studies typically communicate  original research  or analysis for other researchers. 

The Peer Review Process at a Glance:

Looking for peer-reviewed articles?  Try searching in OneSearch or a library database  and look for options to limit your results to scholarly/peer-reviewed or academic journals. Check out this brief tutorial to show you how:   How to Locate a Scholarly (Peer Reviewed) Article

Part 1: Watch the Video

Part 1: watch the video all about peer review (3 min.) and reflect on discussion questions..

Discussion Questions

After watching the video, reflect on the following questions:

• According to the video, what are some of the pros and cons of the peer review process?
• Why is the peer review process important to scholarship?
• Do you think peer reviewers should be paid for their work? Why or why not?

Part 2: Practice

Part 2: take an interactive tutorial on reading a research article for your major..

Includes a certification of completion to download and upload to Canvas.

Social Sciences

(e.g. Psychology, Sociology)

(e.g. Health Science, Biology)

Arts & Humanities

(e.g. Visual & Media Arts, Cultural Studies, Literature, History)

Click on the handout to view in a new tab, download, or print.

For Instructors

• Teaching Peer Review for Instructors

In class or for homework, watch the video “All About Peer Review” (3 min.) .

Video discussion questions:

• According to the video, what are some of the pros and cons of the peer review process

Assignment Ideas

• Ask students to conduct their own peer review of an important journal article in your field. Ask them to reflect on the process. What was hard to critique?
• Have students examine a journals’ web page with information for authors. What information is given to the author about the peer review process for this journal?
• Assign this reading by CSUDH faculty member Terry McGlynn, "Should journals pay for manuscript reviews?" What is the author's argument? Who profits the most from published research? You could also hold a debate with one side for paying reviewers and the other side against.
• Search a database like Cabell’s for information on the journal submission process for a particular title or subject. How long does peer review take for a particular title? Is it is a blind review? How many reviewers are solicited? What is their acceptance rate?
• Assign short readings that address peer review models. We recommend this issue of Nature on peer review debate and open review and this Chronicle of Higher Education article on open review in Shakespeare Quarterly .

Proof of Completion

Mix and match this suite of instructional materials for your course needs!

Questions about integrating a graded online component into your class, contact the Online Learning Librarian, Rebecca Nowicki ( [email protected] ).

Example of a certificate of completion:

• << Previous: Finding Subject Specific Sources: Research Guides
• Next: Understanding Peer Reviewed Articles- Arts & Humanities >>
• Last Updated: Jun 12, 2024 2:51 PM
• URL: https://libguides.sdsu.edu/WhichSource

Unfortunately we don't fully support your browser. If you have the option to, please upgrade to a newer version or use Mozilla Firefox , Microsoft Edge , Google Chrome , or Safari 14 or newer. If you are unable to, and need support, please send us your feedback .

We'd appreciate your feedback. Tell us what you think! opens in new tab/window

What is peer review?

Reviewers play a pivotal role in scholarly publishing. The peer review system exists to validate academic work, helps to improve the quality of published research, and increases networking possibilities within research communities. Despite criticisms, peer review is still the only widely accepted method for research validation and has continued successfully with relatively minor changes for some 350 years.

Elsevier relies on the peer review process to uphold the quality and validity of individual articles and the journals that publish them.

Peer review has been a formal part of scientific communication since the first scientific journals appeared more than 300 years ago. The Philosophical Transactions opens in new tab/window of the Royal Society is thought to be the first journal to formalize the peer review process opens in new tab/window under the editorship of Henry Oldenburg (1618- 1677).

Despite many criticisms about the integrity of peer review, the majority of the research community still believes peer review is the best form of scientific evaluation. This opinion was endorsed by the outcome of a survey Elsevier and Sense About Science conducted in 2009 opens in new tab/window and has since been further confirmed by other publisher and scholarly organization surveys. Furthermore, a  2015 survey by the Publishing Research Consortium opens in new tab/window , saw 82% of researchers agreeing that “without peer review there is no control in scientific communication.”

To learn more about peer review, visit Elsevier’s free e-learning platform  Researcher Academy opens in new tab/window and see our resources below.

The peer review process

Types of peer review.

Peer review comes in different flavours. Each model has its own advantages and disadvantages, and often one type of review will be preferred by a subject community. Before submitting or reviewing a paper, you must therefore check which type is employed by the journal so you are aware of the respective rules. In case of questions regarding the peer review model employed by the journal for which you have been invited to review, consult the journal’s homepage or contact the editorial office directly.  

Single anonymized review

In this type of review, the names of the reviewers are hidden from the author. This is the traditional method of reviewing and is the most common type by far. Points to consider regarding single anonymized review include:

Reviewer anonymity allows for impartial decisions , as the reviewers will not be influenced by potential criticism from the authors.

Authors may be concerned that reviewers in their field could delay publication, giving the reviewers a chance to publish first.

Reviewers may use their anonymity as justification for being unnecessarily critical or harsh when commenting on the authors’ work.

Double anonymized review

Both the reviewer and the author are anonymous in this model. Some advantages of this model are listed below.

Author anonymity limits reviewer bias, such as on author's gender, country of origin, academic status, or previous publication history.

Articles written by prestigious or renowned authors are considered based on the content of their papers, rather than their reputation.

But bear in mind that despite the above, reviewers can often identify the author through their writing style, subject matter, or self-citation – it is exceedingly difficult to guarantee total author anonymity. More information for authors can be found in our  double-anonymized peer review guidelines .

Triple anonymized review

With triple anonymized review, reviewers are anonymous to the author, and the author's identity is unknown to both the reviewers and the editor. Articles are anonymized at the submission stage and are handled in a way to minimize any potential bias towards the authors. However, it should be noted that: 

The complexities involved with anonymizing articles/authors to this level are considerable.

As with double anonymized review, there is still a possibility for the editor and/or reviewers to correctly identify the author(s) from their writing style, subject matter, citation patterns, or other methodologies.

Open review

Open peer review is an umbrella term for many different models aiming at greater transparency during and after the peer review process. The most common definition of open review is when both the reviewer and author are known to each other during the peer review process. Other types of open peer review consist of:

Publication of reviewers’ names on the article page 

Publication of peer review reports alongside the article, either signed or anonymous 

Publication of peer review reports (signed or anonymous) with authors’ and editors’ responses alongside the article 

Publication of the paper after pre-checks and opening a discussion forum to the community who can then comment (named or anonymous) on the article 

Many believe this is the best way to prevent malicious comments, stop plagiarism, prevent reviewers from following their own agenda, and encourage open, honest reviewing. Others see open review as a less honest process, in which politeness or fear of retribution may cause a reviewer to withhold or tone down criticism. For three years, five Elsevier journals experimented with publication of peer review reports (signed or anonymous) as articles alongside the accepted paper on ScienceDirect ( example opens in new tab/window ).

Read more about the experiment

More transparent peer review

Transparency is the key to trust in peer review and as such there is an increasing call towards more  transparency around the peer review process . In an effort to promote transparency in the peer review process, many Elsevier journals therefore publish the name of the handling editor of the published paper on ScienceDirect. Some journals also provide details about the number of reviewers who reviewed the article before acceptance. Furthermore, in order to provide updates and feedback to reviewers, most Elsevier journals inform reviewers about the editor’s decision and their peers’ recommendations. 

Article transfer service: sharing reviewer comments

Elsevier authors may be invited to  transfer  their article submission from one journal to another for free if their initial submission was not successful. 

As a referee, your review report (including all comments to the author and editor) will be transferred to the destination journal, along with the manuscript. The main benefit is that reviewers are not asked to review the same manuscript several times for different journals. 

Tools and resources

Interesting reads.

Chapter 2 of Academic and Professional Publishing, 2012, by Irene Hames in 2012 opens in new tab/window

"Is Peer Review in Crisis?" Perspectives in Publishing No 2, August 2004, by Adrian Mulligan opens in new tab/window

“The history of the peer-review process” Trends in Biotechnology, 2002, by Ray Spier opens in new tab/window

Reviewers’ Update articles

Peer review using today’s technology

Lifting the lid on publishing peer review reports: an interview with Bahar Mehmani and Flaminio Squazzoni

How face-to-face peer review can benefit authors and journals alike

Innovation in peer review: introducing “volunpeers”

Results masked review: peer review without publication bias

Elsevier Researcher Academy modules

The certified peer reviewer course opens in new tab/window

Transparency in peer review opens in new tab/window

Peer-reviewed journal articles

• Overview of peer review
• Scholarly and academic - good enough?
• Find peer-reviewed articles

Use Library Search

Is a journal peer reviewed, check the journal.

Resources listed in  Library Search that are peer reviewed will include the Peer Reviewed icon.

For example:

If you have not used Library Search to find the article, which may indicate if it's peer reviewed, you can use Ulrichsweb to check.

• Go to Ulrichsweb

Screenshot of search box in UlrichsWeb  © Proquest

•  Enter the journal title in the search box.

Screenshot of results list in UlrichsWeb  © Proquest

•  If there are no results, do a search in Ulrichsweb to find journals in your field that are peer reviewed.

Be aware that not all articles in peer reviewed journals are refereed or peer reviewed, for example, editorials and book reviews.

If the journal is not listed in Ulrichsweb :

• Go to the journal's website
• Check for information on a peer review process for the journal. Try the Author guidelines , Instructions for authors  or About this journal sections.

If you can find no evidence that a journal is peer reviewed, but you are required to have a refereed article, you may need to choose a different article.

• << Previous: Find peer-reviewed articles
• Last Updated: Aug 9, 2024 8:24 AM
• URL: https://guides.library.uq.edu.au/how-to-find/peer-reviewed-articles

• Introduction
• Conclusions
• Article Information

Outcomes are estimated from bivariate and multivariable generalized estimating equation models. aOR, indicates adjusted odds ratio; GAD-7, Generalized Anxiety Disorder 7-item scale; PHQ-9, Patient Health Questionnaire 9-item scale; whiskers, 95% CIs.

eTable 1. Survey Instruments

eTable 2. Prevalence of Exposure Over Time

eTable 3. Prevalence of Outcomes Over Time by Exposure Group

eTable 4. E-Value Calculation for Association Between Puberty Blockers or Gender-Affirming Hormones and Mental Health Outcomes

eTable 5. Examining Association Between Puberty Blockers or Gender-Affirming Hormones and Mental Health Outcomes Separately

eTable 6. Bivariate Model Restricted to Youths Ages 13 to 17 Years

eTable 7. Multivariable Model Restricted to 90 Youths Ages 13 to 17 Years

eTable 8. Sensitivity Analyses using Patient Health Questionnaire 8-item Scale Score of 10 or Greater for Moderate to Severe Depression

eFigure 1. Schematic of Generalized Estimating Equation Model

eFigure 2. Association Between Receipt of Gender-Affirming Hormones or Puberty Blockers and Mental Health Outcomes

eReferences

• Medical Groups Defend Patient-Physician Relationship and Access to Adolescent Gender-Affirming Care JAMA Medical News & Perspectives April 19, 2022 This Medical News article discusses physicians’ advocacy to protect patients and the patient-physician relationship amid efforts by politicians to limit access or criminalize gender-affirming care. Bridget M. Kuehn, MSJ
• As Laws Restricting Health Care Surge, Some US Physicians Choose Between Fight or Flight JAMA Medical News & Perspectives June 13, 2023 In this Medical News article, 13 physicians and health care experts spoke with JAMA about the increasing efforts to criminalize evidence-based medical care in the US. Melissa Suran, PhD, MSJ
• Data Errors in eTables 2 and 3 JAMA Network Open Correction July 26, 2022
• Improving Mental Health Among Transgender and Gender-Diverse Youth JAMA Network Open Invited Commentary February 25, 2022 Brett Dolotina, BS; Jack L. Turban, MD, MHS

See More About

Sign up for emails based on your interests, select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Get the latest research based on your areas of interest.

Others also liked.

• Download PDF
• X Facebook More LinkedIn

Tordoff DM , Wanta JW , Collin A , Stepney C , Inwards-Breland DJ , Ahrens K. Mental Health Outcomes in Transgender and Nonbinary Youths Receiving Gender-Affirming Care. JAMA Netw Open. 2022;5(2):e220978. doi:10.1001/jamanetworkopen.2022.0978

Manage citations:

© 2024

• Permissions

Mental Health Outcomes in Transgender and Nonbinary Youths Receiving Gender-Affirming Care

• 1 Department of Epidemiology, University of Washington, Seattle
• 2 Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle
• 3 School of Medicine, University of Washington, Seattle
• 4 Department of Psychiatry and Behavioral Medicine, Department of Adolescent and Young Adult Medicine, Seattle Children’s Hospital, Seattle, Washington
• 5 University of California, San Diego School of Medicine, Rady Children's Hospital
• 6 Division of Adolescent Medicine, Department of Pediatrics, Seattle Children’s Hospital, Seattle, Washington
• Invited Commentary Improving Mental Health Among Transgender and Gender-Diverse Youth Brett Dolotina, BS; Jack L. Turban, MD, MHS JAMA Network Open
• Medical News & Perspectives Medical Groups Defend Patient-Physician Relationship and Access to Adolescent Gender-Affirming Care Bridget M. Kuehn, MSJ JAMA
• Medical News & Perspectives As Laws Restricting Health Care Surge, Some US Physicians Choose Between Fight or Flight Melissa Suran, PhD, MSJ JAMA
• Correction Data Errors in eTables 2 and 3 JAMA Network Open

Question   Is gender-affirming care for transgender and nonbinary (TNB) youths associated with changes in depression, anxiety, and suicidality?

Findings   In this prospective cohort of 104 TNB youths aged 13 to 20 years, receipt of gender-affirming care, including puberty blockers and gender-affirming hormones, was associated with 60% lower odds of moderate or severe depression and 73% lower odds of suicidality over a 12-month follow-up.

Meaning   This study found that access to gender-affirming care was associated with mitigation of mental health disparities among TNB youths over 1 year; given this population's high rates of adverse mental health outcomes, these data suggest that access to pharmacological interventions may be associated with improved mental health among TNB youths over a short period.

Importance   Transgender and nonbinary (TNB) youths are disproportionately burdened by poor mental health outcomes owing to decreased social support and increased stigma and discrimination. Although gender-affirming care is associated with decreased long-term adverse mental health outcomes among these youths, less is known about its association with mental health immediately after initiation of care.

Objective   To investigate changes in mental health over the first year of receiving gender-affirming care and whether initiation of puberty blockers (PBs) and gender-affirming hormones (GAHs) was associated with changes in depression, anxiety, and suicidality.

Design, Setting, and Participants   This prospective observational cohort study was conducted at an urban multidisciplinary gender clinic among TNB adolescents and young adults seeking gender-affirming care from August 2017 to June 2018. Data were analyzed from August 2020 through November 2021.

Exposures   Time since enrollment and receipt of PBs or GAHs.

Main Outcomes and Measures   Mental health outcomes of interest were assessed via the Patient Health Questionnaire 9-item (PHQ-9) and Generalized Anxiety Disorder 7-item (GAD-7) scales, which were dichotomized into measures of moderate or severe depression and anxiety (ie, scores ≥10), respectively. Any self-report of self-harm or suicidal thoughts over the previous 2 weeks was assessed using PHQ-9 question 9. Generalized estimating equations were used to assess change from baseline in each outcome at 3, 6, and 12 months of follow-up. Bivariate and multivariable logistic models were estimated to examine temporal trends and investigate associations between receipt of PBs or GAHs and each outcome.

Results   Among 104 youths aged 13 to 20 years (mean [SD] age, 15.8 [1.6] years) who participated in the study, there were 63 transmasculine individuals (60.6%), 27 transfeminine individuals (26.0%), 10 nonbinary or gender fluid individuals (9.6%), and 4 youths who responded “I don’t know” or did not respond to the gender identity question (3.8%). At baseline, 59 individuals (56.7%) had moderate to severe depression, 52 individuals (50.0%) had moderate to severe anxiety, and 45 individuals (43.3%) reported self-harm or suicidal thoughts. By the end of the study, 69 youths (66.3%) had received PBs, GAHs, or both interventions, while 35 youths had not received either intervention (33.7%). After adjustment for temporal trends and potential confounders, we observed 60% lower odds of depression (adjusted odds ratio [aOR], 0.40; 95% CI, 0.17-0.95) and 73% lower odds of suicidality (aOR, 0.27; 95% CI, 0.11-0.65) among youths who had initiated PBs or GAHs compared with youths who had not. There was no association between PBs or GAHs and anxiety (aOR, 1.01; 95% CI, 0.41, 2.51).

Conclusions and Relevance   This study found that gender-affirming medical interventions were associated with lower odds of depression and suicidality over 12 months. These data add to existing evidence suggesting that gender-affirming care may be associated with improved well-being among TNB youths over a short period, which is important given mental health disparities experienced by this population, particularly the high levels of self-harm and suicide.

Transgender and nonbinary (TNB) youths are disproportionately burdened by poor mental health outcomes, including depression, anxiety, and suicidal ideation and attempts. 1 - 5 These disparities are likely owing to high levels of social rejection, such as a lack of support from parents 6 , 7 and bullying, 6 , 8 , 9 and increased stigma and discrimination experienced by TNB youths. Multidisciplinary care centers have emerged across the country to address the health care needs of TNB youths, which include access to medical gender-affirming interventions, such as puberty blockers (PBs) and gender-affirming hormones (GAHs). 10 These centers coordinate care and help youths and their families address barriers to care, such as lack of insurance coverage 11 and travel times. 12 Gender-affirming care is associated with decreased rates of long-term adverse outcomes among TNB youths. Specifically, PBs, GAHs, and gender-affirming surgeries have all been found to be independently associated with decreased rates of depression, anxiety, and other adverse mental health outcomes. 13 - 16 Access to these interventions is also associated with a decreased lifetime incidence of suicidal ideation among adults who had access to PBs during adolescence. 17 Conversely, TNB youths who present to care later in adolescence or young adulthood experience more adverse mental health outcomes. 18 Despite this robust evidence base, legislation criminalizing and thus limiting access to gender-affirming medical care for minors is increasing. 19 , 20

Less is known about the association of gender-affirming care with mental health outcomes immediately after initiation of care. Several studies published from 2015 to 2020 found that receipt of PBs or GAHs was associated with improved psychological functioning 21 and body satisfaction, 22 as well as decreased depression 23 and suicidality 24 within a 1-year period. Initiation of gender-affirming care may be associated with improved short-term mental health owing to validation of gender identity and clinical staff support. Conversely, prerequisite mental health evaluations, often perceived as pathologizing by TNB youths, and initiation of GAHs may present new stressors that may be associated with exacerbation of mental health symptoms early in care, such as experiences of discrimination associated with more frequent points of engagement in a largely cisnormative health care system (eg, interactions with nonaffirming pharmacists to obtain laboratory tests, syringes, and medications). 25 Given the high risk of suicidality among TNB adolescents, there is a pressing need to better characterize mental health trends for TNB youths early in gender-affirming care. This study aimed to investigate changes in mental health among TNB youths enrolled in an urban multidisciplinary gender clinic over the first 12 months of receiving care. We also sought to investigate whether initiation of PBs or GAHs was associated with depression, anxiety, and suicidality.

This cohort study received approval from the Seattle Children’s Hospital Institutional Review Board. For youths younger than age 18 years, caregiver consent and youth assent was obtained. For youths ages 18 years and older, youth consent alone was obtained. The 12-month assessment was funded via a different mechanism than other survey time points; thus, participants were reconsented for the 12-month survey. The study follows the Strengthening the Reporting of Observational Studies in Epidemiology ( STROBE ) reporting guideline.

We conducted a prospective observational cohort study of TNB youths seeking care at Seattle Children’s Gender Clinic, an urban multidisciplinary gender clinic. After a referral is placed or a patient self-refers, new patients, their caregivers, or patients with their caregivers are scheduled for a 1-hour phone intake with a care navigator who is a licensed clinical social worker. Patients are then scheduled for an appointment at the clinic with a medical provider.

All patients who completed the phone intake and in-person appointment between August 2017 and June 2018 were recruited for this study. Participants completed baseline surveys within 24 hours of their first appointment and were invited to complete follow-up surveys at 3, 6, and 12 months. Youth surveys were used to assess most variables in this study; caregiver surveys were used to assess caregiver income. Participation and completion of study surveys had no bearing on prescribing of PBs or GAHs.

We assessed 3 internalizing mental health outcomes: depression, generalized anxiety, and suicidality. Depression was assessed using the Patient Health Questionnaire 9-item scale (PHQ-9), and anxiety was assessed using the Generalized Anxiety Disorder 7-item scale (GAD-7). We dichotomized PHQ-9 and GAD-7 scores into measures of moderate or severe depression and anxiety (ie, scores ≥10). 26 , 27 Self-harm and suicidal thoughts were assessed using PHQ-9 question 9 (eTable 1 in the Supplement ).

Participants self-reported if they had ever received GAHs, including estrogen or testosterone, or PBs (eg, gonadotropin-releasing hormone analogues) on each survey. We conducted a medical record review to capture prescription of androgen blockers (eg, spironolactone) and medications for menstrual suppression or contraception (ie, medroxyprogesterone acetate or levonorgestrel-releasing intrauterine device) during the study period.

We a priori considered potential confounders hypothesized to be associated with our exposures and outcomes of interest based on theory and prior research. Self-reported gender was ascertained on each survey using a 2-step question that asked participants about their current gender and their sex assigned at birth. If a participant’s self-reported gender changed across surveys, we used the gender reported most frequently by a participant (3 individuals identified as transmasculine at baseline and as nonbinary on all follow-up surveys). We collected data on self-reported race and ethnicity (available response options were Arab or Middle Eastern; Asian; Black or African American; Latinx; Native American, American Indian, or Alaskan Native or Native Hawaiian; Pacific Islander; and White), age, caregiver income, and insurance type. Race and ethnicity were assessed as potential covariates owing to known barriers to accessing gender-affirming care among transgender youth who are members of minority racial and ethnic groups. For descriptive statistics, Asian and Pacific Islander groups were combined owing to small population numbers. We included a baseline variable reflecting receipt of ongoing mental health therapy other than for the purpose of a mental health assessment to receive a gender dysphoria diagnosis. We included a self-report variable reflecting whether youths felt their gender identity or expression was a source of tension with their parents or guardians. Substance use included any alcohol, marijuana, or other drug use in the past year. Resilience was measured by the Connor-Davidson Resilience Scale (CD-RISC) 10-item score developed to measure change in an individual’s state resilience over time. 28 Resilience scores were dichotomized into high (ie, ≥median) and low (ie, <median). Prior studies of young adults in the US reported mean CD-RISC scores ranging from 27.2 to 30.1. 29 , 30

We used generalized estimating equations to assess change in outcomes from baseline at each follow-up point (eFigure 1 in the Supplement ). We used a logit link function to estimate adjusted odds ratio (aOR) for the association between variables and each mental health outcome. We initially estimated bivariate associations between potential confounders and mental health outcomes. Multivariable models included variables that were statistically significant in bivariate models. For all outcomes and models, statistical significance was defined as 95% CIs that did not contain 1.00. Reported P values are based on 2-sided Wald test statistics.

Model 1 examined temporal trends in mental health outcomes, with time (ie, baseline, 3, 6, and 12 months) modeled as a categorical variable. Model 2 estimated the association between receipt of PBs or GAHs and mental health outcomes adjusted for temporal trends and potential confounders. Receipt of PBs or GAHs was modeled as a composite binary time-varying exposure that compared mean outcomes between participants who had initiated PBs or GAHs and those who had not across all time points (eTable 2 in the Supplement ). All models used an independent working correlation structure and robust standard errors to account for the time-varying exposure variable.

We performed several sensitivity analyses. Because our data were from an observational cohort, we first considered the degree to which they were sensitive to unmeasured confounding. To do this, we calculated the E-value for the association between PBs or GAHs and mental health outcomes in model 2. The E-value is defined as the minimum strength of association that a confounder would need to have with both exposure and outcome to completely explain away their association (eTable 4 in the Supplement ). 31 Second, we performed sensitivity analyses on several subsets of youths. We separately examined the association of PBs and GAHs with outcomes of interest, although we a priori did not anticipate being powered to detect statistically significant outcomes owing to our small sample size and the relatively low proportion of youths who accessed PBs. We also conducted sensitivity analyses using the Patient Health Questionnaire 8-item scale (PHQ-8), in which the PHQ-9 question 9 regarding self-harm or suicidal thoughts was removed, given that we analyzed this item as a separate outcome. Lastly, we restricted our analysis to minor youths ages 13 to 17 years because they were subject to different laws and policies related to consent and prerequisite mental health assessments. We used R statistical software version 3.6.2 (R Project for Statistical Computing) to conduct all analyses. Data were analyzed from August 2020 through November 2021.

A total of 169 youths were screened for eligibility during the study period, among whom 161 eligible youths were approached. Nine youths or caregivers declined participation, and 39 youths did not complete consent or assent or did not complete the baseline survey, leaving a sample of 113 youths (70.2% of approached youths). We excluded 9 youths aged younger than 13 years from the analysis because they received different depression and anxiety screeners. Our final sample included 104 youths ages 13 to 20 years (mean [SD] age, 15.8 [1.6] years). Of these individuals, 84 youths (80.8%), 84 youths, and 65 youths (62.5%) completed surveys at 3, 6, and 12 months, respectively.

Our cohort included 63 transmasculine youths (60.6%), 27 transfeminine youths (26.0%), 10 nonbinary or gender fluid youths (9.6%), and 4 youths who responded “I don’t know” or did not respond to the gender identity question on all completed questionnaires (3.8%) ( Table 1 ). There were 4 Asian or Pacific Islander youths (3.8%), 3 Black or African American youths (2.9%); 9 Latinx youths (8.7%); 6 Native American, American Indian, or Alaskan Native or Native Hawaiian youths (5.8%); 67 White youths (64.4%); and 9 youths who reported more than 1 race or ethnicity (8.7%). Race and ethnicity data were missing for 6 youth (5.8%).

At baseline, 7 youths had ever received PBs or GAHs (including 1 youth who received PBs, 4 youths who received GAHs, and 2 youths who received both PBs and GAHs). By the end of the study, 69 youths (66.3%) had received PBs or GAHs (including 50 youths who received GAHs only [48.1%], 5 youths who received PBs only [4.8%], and 14 youths who received PBs and GAHs [13.5%]), while 35 youths had not received either PBs or GAHs (33.7%) (eTable 3 in the Supplement ). Among 33 participants assigned male sex at birth, 17 individuals (51.5%) had received androgen blockers, and among 71 participants assigned female sex at birth, 25 individuals (35.2%) had received menstrual suppression or contraceptives by the end of the study.

A large proportion of youths reported depressive and anxious symptoms at baseline. Specifically, 59 individuals (56.7%) had baseline PHQ-9 scores of 10 or more, suggesting moderate to severe depression; there were 22 participants (21.2%) scoring in the moderate range, 11 participants (10.6%) in the moderately severe range, and 26 participants (25.0%) in the severe range. Similarly, half of participants had a GAD-7 score suggestive of moderate to severe anxiety at baseline (52 individuals [50.0%]), including 20 participants (19.2%) scored in the moderate range, and 32 participants (30.8%) scored in the severe range. There were 45 youths (43.3%) who reported self-harm or suicidal thoughts in the prior 2 weeks. At baseline, 65 youths (62.5%) were receiving ongoing mental health therapy, 36 youths (34.6%) reported tension with their caregivers about their gender identity or expression, and 34 youths (32.7%) reported any substance use in the prior year. Lastly, we observed a wide range of resilience scores (median [range], 22.5 [1-38], with higher scores equaling more resiliency). There were no statistically significant differences in baseline characteristics by gender.

In bivariate models, substance use was associated with all mental health outcomes ( Table 2 ). Youths who reported any substance use were 4-fold as likely to have PHQ-9 scores of moderate to severe depression (aOR, 4.38; 95% CI, 2.10-9.16) and 2-fold as likely to have GAD-7 scores of moderate to severe anxiety (aOR, 2.07; 95% CI, 1.04-4.11) or report thoughts of self-harm or suicide in the prior 2 weeks (aOR, 2.06; 95% CI, 1.08-3.93). High resilience scores (ie, ≥median), compared with low resilience scores (ie, <median), were associated with lower odds of moderate or severe anxiety (aOR, 0.51; 95% CI, 0.26-0.999).

There were no statistically significant temporal trends in the bivariate model or model 1 ( Table 2 and Table 3 ). However, among all participants, odds of moderate to severe depression increased at 3 months of follow-up relative to baseline (aOR, 2.12; 95% CI, 0.98-4.60), which was not a significant increase, and returned to baseline levels at months 6 and 12 ( Figure ) prior to adjusting for receipt of PBs or GAHs.

We also examined the association between receipt of PBs or GAHs and mental health outcomes in bivariate and multivariable models (eFigure 2 in the Supplement ). After adjusting for temporal trends and potential confounders ( Table 4 ), we observed that youths who had initiated PBs or GAHs had 60% lower odds of moderate to severe depression (aOR, 0.40; 95% CI, 0.17-0.95) and 73% lower odds of self-harm or suicidal thoughts (aOR, 0.27; 95% CI, 0.11-0.65) compared with youths who had not yet initiated PBs or GAHs. There was no association between receipt of PBs or GAHs and moderate to severe anxiety (aOR, 1.01; 95% CI, 0.41-2.51). After adjusting for time-varying exposure of PBs or GAHs in model 2 ( Table 4 ), we observed statistically significant increases in moderate to severe depression among youths who had not received PBs or GAHs by 3 months of follow-up (aOR, 3.22; 95% CI, 1.37-7.56). A similar trend was observed for self-harm or suicidal thoughts among youths who had not received PBs or GAHs by 6 months of follow-up (aOR, 2.76; 95% CI, 1.22-6.26). Lastly, we estimated E-values of 2.56 and 3.25 for the association between receiving PGs or GAHs and moderate to severe depression and suicidality, respectively (eTable 4 in the Supplement ). Sensitivity analyses obtained comparable results and are presented in eTables 5 through 8 in the Supplement .

In this prospective clinical cohort study of TNB youths, we observed high rates of moderate to severe depression and anxiety, as well as suicidal thoughts. Receipt of gender-affirming interventions, specifically PBs or GAHs, was associated with 60% lower odds of moderate to severe depressive symptoms and 73% lower odds of self-harm or suicidal thoughts during the first year of multidisciplinary gender care. Among youths who did not initiate PBs or GAHs, we observed that depressive symptoms and suicidality were 2-fold to 3-fold higher than baseline levels at 3 and 6 months of follow-up, respectively. Our study results suggest that risks of depression and suicidality may be mitigated with receipt of gender-affirming medications in the context of a multidisciplinary care clinic over the relatively short time frame of 1 year.

Our findings are consistent with those of prior studies finding that TNB adolescents are at increased risk of depression, anxiety, and suicidality 1 , 11 , 32 and studies finding long-term and short-term improvements in mental health outcomes among TNB individuals who receive gender-affirming medical interventions. 14 , 21 - 24 , 33 , 34 Surprisingly, we observed no association with anxiety scores. A recent cohort study of TNB youths in Dallas, Texas, found that total anxiety symptoms improved over a longer follow-up of 11 to 18 months; however, similar to our study, the authors did not observe statistically significant improvements in generalized anxiety. 22 This suggests that anxiety symptoms may take longer to improve after the initiation of gender-affirming care. In addition, Olson et al 35 found that prepubertal TNB children who socially transitioned did not have increased rates of depression symptoms but did have increased rates of anxiety symptoms compared with children who were cisgender. Although social transition and access to gender-affirming medical care do not always go hand in hand, it is noteworthy that access to gender-affirming medical care and supported social transition appear to be associated with decreased depression and suicidality more than anxiety symptoms.

Time trends were not significant in our study; however, it is important to note that we observed a transient and nonsignificant worsening in mental health outcomes in the first several months of care among all participants and that these outcomes subsequently returned to baseline by 12 months. This is consistent with findings from a 2020 study 36 in an academic medical center in the northwestern US that observed no change in TNB adolescents’ GAD-7 or PHQ-9 scores from intake to first follow-up appointment, which occurred a mean of 4.7 months apart. Given that receipt of PBs or GAHs was associated with protection against depression and suicidality in our study, it could be that delays in receipt of medications is associated with initially exacerbated mental health symptoms that subsequently improve. It is also possible that mental health improvements associated with receiving these interventions may have a delayed onset, given the delay in physical changes after starting GAHs.

Few of our hypothesized confounders were associated with mental health outcomes in this sample, most notably receipt of ongoing mental health therapy and caregiver support; however, this is not surprising given that these variables were colinear with baseline mental health, which we adjusted for in all models. Substance use was the only variable associated with all mental health outcomes. In addition, youths with high baseline resilience scores were half as likely to experience moderate to severe anxiety as those with low scores. This finding suggests that substance use and resilience may be additional modifiable factors that could be addressed through multidisciplinary gender-affirming care. We recommend more granular assessment of substance use and resilience to better understand support needs (for substance use) and effective support strategies (for resilience) for TNB youths in future research.

This study has a number of strengths. This is one of the first studies to quantify a short-term transient increase in depressive symptoms experienced by TNB youths after initiating gender-affirming care, a phenomenon observed clinically by some of the authors and described in qualitative research. 37 Although we are unable to make causal statements owing to the observational design of the study, the strength of associations between gender-affirming medications and depression and suicidality, with large aOR values, and sensitivity analyses that suggest that these findings are robust to moderate levels of unmeasured confounding. Specifically, E-values calculated for this study suggest that the observed associations could be explained away only by an unmeasured confounder that was associated with both PBs and GAHs and the outcomes of interest by a risk ratio of 2-fold to 3-fold each, above and beyond the measured confounders, but that weaker confounding could not do so. 31

Our findings should be interpreted in light of the following limitations. This was a clinical sample of TNB youths, and there was likely selection bias toward youths with supportive caregivers who had resources to access a gender-affirming care clinic. Family support and access to care are associated with protection against poor mental health outcomes, and thus actual rates of depression, anxiety, and suicidality in nonclinical samples of TNB youths may differ. Youths who are unable to access gender-affirming care owing to a lack of family support or resources require particular emphasis in future research and advocacy. Our sample also primarily included White and transmasculine youths, limiting the generalizability of our findings. In addition, the need to reapproach participants for consent and assent for the 12-month survey likely contributed to attrition at this time point. There may also be residual confounding because we were unable to include a variable reflecting receipt of psychotropic medications that could be associated with depression, anxiety, and self-harm and suicidal thought outcomes. Additionally, we used symptom-based measures of depression, anxiety, and suicidality; further studies should include diagnostic evaluations by mental health practitioners to track depression, anxiety, gender dysphoria, suicidal ideation, and suicide attempts during gender care. 2

Our study provides quantitative evidence that access to PBs or GAHs in a multidisciplinary gender-affirming setting was associated with mental health improvements among TNB youths over a relatively short time frame of 1 year. The associations with the highest aORs were with decreased suicidality, which is important given the mental health disparities experienced by this population, particularly the high levels of self-harm and suicide. Our findings have important policy implications, suggesting that the recent wave of legislation restricting access to gender-affirming care 19 may have significant negative outcomes in the well-being of TNB youths. 20 Beyond the need to address antitransgender legislation, there is an additional need for medical systems and insurance providers to decrease barriers and expand access to gender-affirming care.

Accepted for Publication: January 10, 2022.

Published: February 25, 2022. doi:10.1001/jamanetworkopen.2022.0978

Correction: This article was corrected on July 26, 2022, to fix minor errors in the numbers of patients in eTables 2 and 3 in the Supplement.

Open Access: This is an open access article distributed under the terms of the CC-BY License . © 2022 Tordoff DM et al. JAMA Network Open .

Corresponding Author: Diana M. Tordoff, MPH, Department of Epidemiology, University of Washington, UW Box 351619, Seattle, WA 98195 ( [email protected] ).

Author Contributions : Diana Tordoff had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Diana Tordoff and Dr Wanta are joint first authors. Drs Inwards-Breland and Ahrens are joint senior authors.

Concept and design: Collin, Stepney, Inwards-Breland, Ahrens.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Tordoff, Wanta, Collin, Stepney, Inwards-Breland.

Critical revision of the manuscript for important intellectual content: Wanta, Collin, Stepney, Inwards-Breland, Ahrens.

Statistical analysis: Tordoff.

Obtained funding: Inwards-Breland, Ahrens.

Administrative, technical, or material support: Ahrens.

Supervision: Wanta, Inwards-Breland, Ahrens.

Conflict of Interest Disclosures: Diana Tordoff reported receiving grants from the National Institutes of Health National Institute of Allergy and Infectious Diseases unrelated to the present work and outside the submitted work. No other disclosures were reported.

Funding/Support: This study was supported Seattle Children’s Center for Diversity and Health Equity and the Pacific Hospital Preservation Development Authority.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

• Open access
• Published: 08 August 2024

Evidence for motivational interviewing in educational settings among medical schools: a scoping review

• Leonard Yik Chuan Lei 1 ,
• Keng Sheng Chew 1 ,
• Chee Shee Chai 1 &
• Yoke Yong Chen 1  

BMC Medical Education volume  24 , Article number:  856 ( 2024 ) Cite this article

176 Accesses

Metrics details

Motivational interviewing (MI) is a person-centred approach focused on empowering and motivating individuals for behavioural change. Medical students can utilize MI in patient education to engage with patients’ chronic health ailments and maladaptive behaviours. A current scoping review was conducted to 1) determine the types of MI (conventional, adapted, brief and group MI) education programs in medical schools, delivery modalities and teaching methods used; 2) classify educational outcomes on the basis of Kirkpatrick’s hierarchy; and 3) determine the key elements of MI education via the FRAMES (feedback, responsibility, advice, menu of options, empathy, self-efficacy) model.

This scoping review was conducted via the framework outlined by Arksey and O’Malley. Two online databases, CINAHL and MEDLINE Complete, were searched to identify MI interventions in medical education. Further articles were selected from bibliography lists and the Google Scholar search engine.

From an initial yield of 2019 articles, 19 articles were included. First, there appears to be a bimodal distribution of most articles published between the two time periods of 2004--2008 and 2019--2023. Second, all the studies included in this review did not use conventional MI but instead utilized a variety of MI adaptation techniques. Third, most studies used face-to-face training in MI, whereas only one study used online delivery. Fourth, most studies have used a variety of interactive experiences to teach MI. Next, all studies reported outcomes at Kirkpatrick’s Level 2, but only 4 studies reported outcomes at Kirkpatrick’s Level 3. According to the FRAMES model, all studies ( n =19; 100%) reported the elements of responsibility and advice. The element that was reported the least was self-efficacy ( n = 12; 63.1%).

Our findings suggest that motivational interviewing can be taught effectively in medical schools via adaptations to MI and a variety of teaching approaches. However, there is a need for further research investigating standardized MI training across medical schools, the adequate dose for training in MI and the implementation of reflective practices. Future studies may benefit from exploring and better understanding the relationship between MI and self-efficacy in their MI interventions.

Peer Review reports

Motivational interviewing (MI) is a person-centred approach that focuses on empowering and motivating individuals for behavioural change [ 1 ]. Undoubtedly, the empathetic approach of MI in clinical settings fosters a supportive environment that encourages discussion of the benefits of enhanced self-care [ 2 ]. In this context, MI practitioners utilize a set of essential skills encapsulated by the acronym “OARS”, which stands for O = open-ended questions, A = affirmations, R = reflections and S = summaries to promote active listening [ 3 ]. MI was developed primarily for the treatment of addiction disorders but has since progressed to include other physical and mental ailments as well [ 4 ]. In a study on MI interventions in alcoholism, Miller & Sanchez [ 68 ] identified six common motivational elements that should be covered, represented by the acronym “FRAMES”, where F = feedback (e.g., personalized feedback on the impacts of alcoholism on the client’s own experiences, as opposed to providing generic information); R = responsibility (e.g., empowering clients to make their own choices and take responsibility for their change process); A = advice (e.g., effectively given in a nondirective and noncoercive manner); M = menu (e.g., offering a variety of choices on transition methods and plans); E = empathy (e.g., rendering empathic, reassuring and reflective listening); and S = self-efficacy (e.g., supporting clients to succeed in a specified goal). This review used the FRAMES model to determine the key elements of MI education. FRAMES was a predecessor to MI and was initially designed to address drinking problems [ 5 ]; however, it is also used in other health issues, such as decreasing stroke risk [ 6 ], substance use screening and brief intervention [ 7 ]. The FRAMES model offers a structure that can be used to improve the delivery of MI by ensuring that key elements of MI are present in educational interventions.

Mechanisms of motivational interviewing

Frey et al. [ 8 ] developed mechanisms of the motivational interviewing (MMI) framework and described the mechanisms of fidelity of practice in MI, including a technical component, a relational component and MI-inconsistent practices [ 8 ]. The technical component consists of the interviewer’s ability to evaluate the participant’s language relating to a specific behaviour change target and then build a conversation that evokes change talk. The relational component includes respect for the participant’s self-determination, appropriate empathy, and equal partnership. Non-MI consistent behaviours include confrontation, offering unsolicited advice, and persuasion. Additionally, it is important to identify and understand the mechanisms of change so that MI users and researchers can focus on these mechanisms during training, which can lead to improved outcomes and fidelity [ 8 ].

Types of motivational interviewing

MI can be categorized into four types: conventional, adaptive, brief, and group. Conventional MI is an evidence-based approach and directive form of interviewing developed by Miller & Rollnick [ 9 ]. Throughout the course of MI, four important tasks occur: engaging (building mutual relationships), focusing (setting goals), evoking (developing clients’ motivations for change) and planning (negotiating change) [ 9 ]. In this review, the term conventional MI is defined as an approach that utilizes MI-consistent tasks and behaviours in multiple sessions that target an identified population of clients.

Adapted MI consists of culturally sensitive MI and digitally supported interventions that can be used as adjunct interventions to the primary behavioural program [ 10 ]. This review defines the term adapted MI to include any adaptations made to adapt MI culturally to the setting or delivered by technology through various types of technologies and content (e.g., computers, smartphones, applications, videos and audio). Additionally, it also includes adaptations made to structured curricula, such as using role plays or real patient interactions to facilitate the learning of MI.

Brief MI is a type of MI with varying lengths, ranging from 5--90 minutes in duration, emphasizing the lack of an accepted definition of brief MI [ 10 ]. This review defines the term brief MI as an MI that provides brief consultations centred on typically fewer sessions (e.g., 1--2 sessions) than conventional MI (e.g., 3--4 sessions or more).

Group MI can be defined as groups of clients that apply the MI spirit, processes and methods to increase motivation for change and promote beneficial collaboration among participants and practitioners in a shared location to encourage change [ 11 ]. This review defines the term group MI as MI that is adapted for group format and is MI consistent (e.g., applying MI principles, spirit and techniques in its delivery).

Additionally, MI can be used in patient education to help patients better handle their chronic health conditions and maladaptive behaviours. Therefore, behavioural change is vital in the recovery course of different mental and physical disorders, as a change to a healthier lifestyle has been shown to result in a significant decrease in chronic disease risk [ 12 ]. More than 120 studies have demonstrated the efficacy of MI in addressing a wide range of problematic behaviours, such as substance abuse and risky behaviour, as well as promoting healthy behaviours [ 13 ]. There is specific evidence regarding the effectiveness of MI across different health behaviours (substance abuse, risky behaviours and promoting health behaviours), according to the types of MI: conventional, adaptive, brief and group. For conventional MI, research has shown effectiveness in treating substance abuse [ 14 ], reducing risky behaviours in human immunodeficiency virus (HIV)-positive men [ 15 ] and promoting physical activity in older adults [ 16 ]. Adaptive MI has demonstrated its effectiveness in reducing alcohol problems in women [ 17 ], reducing risky sexual behaviours and psychological symptoms in HIV-positive older adults [ 18 ] and promoting self-management to reduce BMI and improve lifestyle adherence with a computer assistant [ 19 ]. Brief MI has been effective in reduction in alcohol misuse in college students with attention deficit hyperactivity disorder (ADHD) [ 20 ] and improvement in the engagement of physical activity in patients with low physical activity levels [ 21 ]. Research has revealed that group MI is effective in treating drug use among women [ 22 ], reducing risky sexual behaviour among adolescents [ 23 ] and improving self-efficacy and oral health behaviours among pregnant women [ 24 ].

Unhealthy lifestyle-linked behaviours characterize common preventable risk factors that lead to the majority of noncommunicable diseases and their associated mortality and morbidity [ 25 ]. MI provides an approach for healthcare providers to assist patients in investigating and resolving their ambivalence toward changing unhealthy lifestyle behaviour [ 27 ]. Studies have reported the effectiveness of teaching MI to medical students [ 4 , 26 , 28 , 29 , 30 ]. Therefore, considering the prevalence and widespread application of MI in health care settings, this underscores the importance of MI being taught in the initial stages of medical education.

In a recent systematic review, Kaltman and Tankersley [ 31 ] reviewed MI curricula in undergraduate medical education (UME) and revealed important findings. Their research findings suggest that generally being involved in an MI curriculum can be linked to enhanced MI-related knowledge and skills in the short term. Additionally, they noted that 1) the MI curricula were heterogeneous in nature; 2) the curricula were different in terms of timing, duration and number of sessions; 3) the curricula employed in studies were multiple pedagogies; and 4) the quality of the evaluations and research evidence varied. However, this review by Kaltman and Tankersley [ 31 ] was limited to reporting only on MI-specific outcomes such as knowledge, skills, attitudes towards, and self-efficacy in implementing MI. Kaltman and Tankersley [ 31 ] systematic review did not stratify and explore in detail studies on the types of MI (conventional, adaptive, brief, or group). Furthermore, the systematic review did not investigate the key elements of MI education as described by the FRAMES model. The scoping review aimed to bridge the knowledge gap on types of MI (conventional, adapted, brief, group MI) and key elements of MI education covered via the FRAMES model. Specifically, the objectives of this study were to 1) determine the types of MI education programs in medical schools, the delivery modalities, and the teaching methods used; 2) classify educational outcomes on the basis of Kirkpatrick’s hierarchy [ 32 ]; and 3) determine the key elements of MI education covered via the FRAMES model.

This study adopted the methodological 5-step framework of Arksey and O’Malley for this scoping review. The five steps are as follows: 1) define our research objectives; 2) identify relevant studies; 3) identify studies based on our selection criteria; 4) chart and analyse the data; and 5) collate, summarize, and disseminate the results.

Eligibility criteria

Relevant peer-reviewed articles on MI studies conducted in medical education settings, published in academic journals only, in the English language, with no time limit imposed on the publication period, were identified. Studies involving nonmedical students as well as grey literature, such as conference proceedings, technical reports, videos, and informal communications, were excluded. Studies in languages other than English were also excluded. The search strategy was guided by the methodology of Aromataris and Riitano [ 33 ]. The Boolean operators and keywords used in this search strategy were ("medical education" OR "medical teaching*" OR "medical graduate*" OR "medical postgraduate*” OR “medical student*”) AND ("motivational interview*" OR "motivational enhanc*" OR "motivational chang*" OR "motivational behavior”) AND ("psycholog*" OR "health*"). The search utilized databases from the Medical Literature Analysis and Retrieval System Online (MEDLINE Complete) and Cumulative Index of Nursing and Allied Health Literature (CINAHL Complete) databases via the EBSCOHost database search query, covering all study designs (i.e., quantitative, qualitative, and mixed studies). The protocol was developed a priori before the search process was conducted, including establishing the objectives and eligibility criteria for determining the studies selected. The reference lists of the selected studies were further checked for additional sources, including traditional and systematic reviews. Articles that met the eligibility criteria were selected through a consensus among the authors and were charted according to the Preferred Reporting Items for Systematic reviews and Meta-analysis extension for Scoping Reviews (PRISMA-ScR) guidelines [ 34 ]. The first author conducted the searches and screened the articles using the search strategy and the inclusion and exclusion criteria stated above. This process resulted in the identification of 59 articles. The decision process resulted in 19 studies for inclusion in this review based on the inclusion and exclusion criteria. The data were extracted and charted by the first author. Notably, the following data were extracted: 1) the study characteristics of the identified articles (publication year, country of origin, type of MI, and medical student phase) and 2) a detailed description of the key findings of the articles (i.e., author, year, objectives, participants, delivery, duration, teaching methods, assessments, and educational outcomes based on Kirkpatrick’s hierarchy). Proforma was developed by all the authors and used to extract and chart the data. The study characteristics are then charted in Table 1 , and detailed descriptions of the key findings of the articles are charted in Table 2 . The other authors assisted in identifying specific data elements to be charted onto Tables 1 and 2 . All the authors contributed to analysing the charted data to ensure the consistency and accuracy of the analysis. The outcomes of educational intervention were classified under the four levels of Kirkpatrick’s hierarchy. Studies classified as Level 3 consists of simulations and observations of behaviours in activities (e.g., roleplay, standardized patients, real patients) after a learning activity such as a workshop. Although Level 3 is usually linked to students applying what they have acquired in training to job settings, our classification extends to controlled settings simulating real-life applications. The most recent search of MEDLINE Complete, CINAHL Complete and Google Scholar was carried out in October 2023.

From an initial pool of 2,019 articles, after removing duplicates and screening for relevance, 19 articles were included in this review. The detailed selection process is illustrated in the PRISMA flow diagram in Fig. 1 .

Prism flow diagram

Characteristics of the identified articles

The study characteristics, country of origin, and phase of study are presented in Table 1 . The detailed descriptions of the key findings of these articles (i.e., author, year, objectives, participants, delivery, duration, teaching methods, assessments, and educational outcomes based on Kirkpatrick’s hierarchy) are provided in Table 2 . Most of the studies were published between 2004–2008 and 2019–2023, with each period accounting for 31.5% of the total articles. The majority of MI studies originated from the US (57.8%).

Types and characteristics of MI

With respect to the first research objective, none of the 19 studies in this scoping review conducted conventional MI. Rather, most studies in this scoping review used adapted MI ( n =8; 42.1%) [ 4 , 36 , 38 , 42 , 44 , 46 , 47 , 49 ], followed by group MI ( n =7; 36.8%) [ 26 , 29 , 35 , 40 , 45 , 48 , 39 ] and brief MI ( n =4; 21%) [ 37 , 41 , 43 , 50 ].

Adapted motivational interviewing was utilized in 8 studies. This approach includes any adaptations utilized to adjust MI culturally to the situation or facilitated by technology via different types of content and technologies (e.g., computers, smartphones, applications, videos and audio). Additionally, it also includes adaptations made to structured curricula, such as using role plays via standardized patients or real patient interactions to facilitate the learning of MI. Adapted MI was reported in 8 studies. Specifically, 5 studies [ 36 , 38 , 42 , 44 , 47 ] adapted their curricula to teach MI via role playing standardized patients or real patients. Additionally, 3 studies [ 4 , 46 , 49 ] utilized technological adaptations and blended learning (face-to-face and online) to teach motivational interviewing.

In group MI, this approach consists of MI that is adapted for group format and is MI consistent (e.g., applying MI principles, spirit and techniques in its delivery). Group MI was carried out in 7 studies. Two studies [ 26 , 45 ] used training workshops to teach and practice MI in smaller groups. The remaining 5 studies [ 29 , 35 , 39 , 40 , 48 ] used a small group format to teach MI skills consisting of lectures, roleplay, a case-based curriculum and demonstrations.

Brief MI provides brief consultations centred on typically shorter number sessions (e.g., 1--2 sessions) than conventional MI (e.g., 3--4 sessions or more). A brief MI was conducted in 4 studies. Two studies [ 41 , 51 ] delivered a single session of MI training within two hours. Another study [ 50 ] conducted four (10–15 minute) sessions teaching MI, with a total of less than 1 hour of training. Opheim et al. [ 43 ] conducted a four-hour workshop on MI, which is a relatively brief training intervention.

More than half of the studies focused on clinical medical students ( n =10; 52.6%) [ 4 , 35 , 37 , 38 , 41 , 42 , 43 , 45 , 46 , 49 ], and the least studied was the combination of preclinical and clinical students ( n =2; 10.5%) [ 40 , 47 ]. There was a diverse number of participants, ranging from 17 to 339 students. The median number of participants in these studies was 93. The most common delivery mode identified was face-to-face learning ( n =15; 78.9%) [ 26 , 29 , 35 , 36 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 47 , 48 , 51 ], followed by blended learning ( n =3; 15.7%) [ 4 , 46 , 49 ], and the least common delivery mode was online learning ( n =1, 5.2%) [ 50 ]. The duration of intervention for brief MI ( n =4; 21.0%) [ 37 , 41 , 43 , 50 ] ranged from 10 minutes to 2 hours per session. The duration of adapted MI ( n =8; 42.1%) [ 4 , 36 , 38 , 42 , 44 , 46 , 47 , 49 ] and group MI ( n =7; 36.8%) [ 26 , 29 , 35 , 40 , 39 , 45 , 48 ] ranged from 3 hours to 12 hours. The teaching methods include workshops, lectures, videos, role plays, demonstrations, interviews, interactive exercises, small and large group activities, simulated patients, and online forums.

Classifying educational outcomes based on Kirkpatrick’s hierarchy

With respect to the second research objective (i.e., classifying educational outcomes on the basis of Kirkpatrick’s hierarchy [ 32 ]), all 19 studies [ 4 , 26 , 29 , 35 , 36 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 ] were categorized at Kirkpatrick’s Level 2 (knowledge/skills/attitudes). This is followed by 16 out of 19 studies [ 4 , 26 , 29 , 35 , 36 , 39 , 40 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 ] categorized at Kirkpatrick’s Level 1. Only 4 out of 19 studies [ 35 , 38 , 41 , 47 ] are categorized at Kirkpatrick’s Level 3 (Behaviour). One of the studies [ 38 ] compared the effectiveness of standardized patients versus role plays from colleagues and reported that both were equally effective for teaching basic MI skills among medical students. The students were evaluated in a simulated environment and demonstrated their MI skills in terms of student roleplay or standardized patients. The study reported that standardized patient role play is as effective as student role play in teaching basic MI skills. The sessions focused on demonstrating skills in a simulated setting, suggesting that the student’s behaviour (i.e., adherence to MI skills) was evaluated and improved via the educational intervention. In another study, Bell et al. [ 35 ] investigated the use of a curriculum to teach medical students the principles of MI to increase their knowledge, skills and confidence in counselling patients with the aim of health behaviour change. The research indicated that video-recorded interactions between students and patients enabled students to effectively apply MI skills to real-life patients. None of the studies included reported outcomes at Level 4 (results).

Key elements of the reported FRAMES model and assessment methods used

With respect to the third research objective, all 6 elements in the FRAMES model were covered in 9 out of 19 studies [ 4 , 29 , 35 , 36 , 39 , 40 , 44 , 45 , 51 ], 5 elements were identified in another 4 studies [ 26 , 41 , 48 , 49 ], and 4 elements were identified in 4 studies [ 42 , 43 , 46 , 47 ]. The most reported element in all 19 studies was responsibility and advice ( n =19; 100%), and the least reported element was self-efficacy in only 12 studies ( n =12; 63.1%) [ 4 , 29 , 35 , 36 , 39 , 40 , 41 , 44 , 45 , 46 , 48 , 51 ]. Figure 2 shows additional details on the important elements present in the MI interventions.

Important elements of MI interventions ( n  = 19) identified as “reported” via the FRAMES model

The primary assessment method used across the studies was the use of pre- and posttest surveys, which are used to measure knowledge ( n =10, 52.6%), skills ( n =5, 26.3%) and attitudes ( n =3, 15.8%) pertaining to MI. Moreover, the specific instruments employed for focused assessments were (1) MITI to measure fidelity of MI in 5 out of 19 studies ( n =5, 26.3%), (2) Video Assessment of Simulated Encounters (VASE-R) to measure MI skills in 2 out of 19 ( n =2, 10.5%) (3) Behaviour Change Counselling Index (BECCI) to measure practitioner’s skill and competence in delivering effective MI in 2 studies out of 19 ( n =2, 10.5%), (4) Objective Structured Clinical Examination (OSCE) to measure clinical competence in 2 studies out of 19 ( n =2, 10.5%), (5) Motivational Interviewing Knowledge and Attitudes Test (MIKAT) to measure the practitioner’s knowledge and attitude pertaining to MI in 1 study out of 19 ( n =1, 5.2%), (6) Motivational interviewing skill code (MISC) to measure adherence to MI in 1 study out of 19 ( n =1, 5.2%), (7) the Calgary-Cambridge Observation Guide (C-CG) to measure communication skills between practitioners and patients was used in 1 study out of 19 ( n =1, 5.2%), (8) Motivational interviewing confidence scale (MICS) to measure confidence in health behaviour change dialogues in 1 study out of 19 ( n =1, 5.2%) and (8) the Jefferson Scale of Physician Empathy (JSPE) to measure empathy in patient care among health practitioners in 1 study out of 19 ( n =1, 5.2%).

Our scoping review sheds light on the current trends and key findings to determine the types of MI education programs in medical schools, the delivery modalities and teaching methods used, classify educational outcomes on based on Kirkpatrick’s hierarchy [ 32 ] and determine the key elements of MI education covered via the FRAMES model. First, there appears to be a bimodal distribution of most articles published between the two time periods of 2004--2008 and 2019--2023. Second, all the studies included in this review did not use conventional MI but instead utilized a variety of MI adaptation techniques. Third, most studies used face-to-face training in MI, whereas only one study used online delivery. Fourth, most studies have used a variety of interactive experiences to teach MI. Next, all studies reported outcomes at Kirkpatrick’s Level 2, but only 4 studies reported outcomes at Kirkpatrick’s Level 3. Finally, the most covered elements of MI training in these studies were responsibility and advice ( n = 19; 100%), and the least covered element in MI training was self-efficacy ( n = 12; 63.1%) [ 4 , 29 , 35 , 36 , 39 , 40 , 41 , 44 , 45 , 46 , 48 , 51 ]. This review expands on the evidence of MI interventions among medical schools. The results of our findings generally suggest that MI can be effectively taught in medical schools. Furthermore, we have provided several recommendations for further research to improve the implementation of MI in medical schools.

There appears to be a bimodal distribution of published articles between the two time periods, i.e., between 2004 and 2008 and between 2019 and 2023. A decline in the number of articles published was observed between 2009 and 2019. This decline could be due to the shift in the applications of MI beyond treating addictive behaviours to include a broad range of other behavioural conditions [ 52 ], such as its expanded applications in school education [ 53 , 54 , 55 ], lifestyle coaching [ 56 , 57 , 58 ], probation and parole [ 59 , 60 ] and digital health care and telemedicine [ 61 , 62 ]. From 2019 onwards, however, there was an increasing trend in the number of published articles on MI training for medical students. This could be attributed to the MI Network of Trainers (MINT) making it mandatory to attend MI training during the COVID-19 pandemic to provide virtual training in 2020 and 2021 [ 52 ], which has facilitated remote participation.

Types of MI education programs in medical schools

All the studies included in this review did not use conventional MI but utilized a variety of MI adaptation techniques. Most studies [ 4 , 36 , 38 , 42 , 44 , 46 , 47 , 49 ] have used adapted MI to conduct their MI training, possibly because of the need to tailor MI programs to fit medical school curricula. Medical students have been linked to extensive academic responsibilities and clinical rotations [ 63 ], contributing to this adaptation of MI. In fact, the lack of harmonization of training methods among medical schools has led to challenges in understanding the optimal approach to teach MI among medical students [ 31 ]. Furthermore, there is no consensus on the standard dose of training in MI that is adequate or mandatory for learners to acquire sufficient skilfulness in the practice of MI [ 9 ]. Moreover, medical schools have time constraints and limited MI teaching opportunities because of their hectic medical curriculum schedules [ 41 ]. This may lead to a variety of adaptations of MI, as noted in this review. Future research can focus on addressing the lack of harmonization in MI training methods and emphasize building and employing standardized MI training with adequate dosing across medical schools.

Delivery modalities and teaching methods used

In the present review, the delivery modalities used to train medical students in MI varied across the studies. Most studies [ 26 , 29 , 35 , 36 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 47 , 48 , 51 ] have focused on delivering face-to-face training on MI to clinical medical students. This aligns with the current literature, which suggests that MI is a complex communication skill [ 57 ] and is reported to be taught more effectively in face-to-face sessions [ 64 ]. In this review, only one study [ 50 ] used a fully online approach to teach MI to medical students. A systemic study suggested that for an online MI intervention to be effective, it requires significant emphasis on fidelity and training procedures [ 65 ]. In a recent comparative study, Schaper et al. [ 66 ] reported similar effects of training MI among general practitioners in both online and face-to-face training in MI skills and spirit. Future studies could focus on the implementation of online versus face-to-face training for medical students with an emphasis on fidelity and training procedures for MI.

A large proportion of the studies in this review report the use of a variety of teaching approaches (e.g., workshops, role-play, standardized patients, and small and large group sessions) to teach MI. This aligns with Kolb’s experiential learning cycle [ 67 ], where the process of learning occurs when knowledge is formed via the transformation of experience. This model is guided by four phases of the learning process: concrete experience (having an experience), reflective observation (reflecting on an experience), abstract conceptualization (learning from the experience), and active experimentation (experimenting what you have learned). Medical students who are given the opportunity to engage in Kolb’s learning cycle [ 67 ] via interactive activities, reflection and simulated or real-life settings are likely to develop good MI skills. Future research should underpin educational theories into MI training by implementing structured reflective exercises in MI education.

Educational outcomes based on Kirkpatrick’s hierarchy

Our review shows that all studies reported outcomes at Kirkpatrick’s Level 2, suggesting that medical students have acquired the intended knowledge, skills, and attitudes. There are only 4 studies that reported outcomes at Kirkpatrick’s Level 3, which evaluates the degree to which the students apply their learning to simulated or real-world settings. The first 3 studies [ 38 , 41 , 47 ] showed their improvement in behaviour by showing their learned skills in realistic settings, which included observing students’ behaviour in standardized patients or real patients. The last study [ 35 ] revealed improvements in the MI skills of real patients in diverse settings, such as traditional health behaviour interventions, such as alcohol, tobacco and weight loss interventions. Future studies should include longitudinal evaluations of the effectiveness of MI skills.

Key elements of MI education covered via the FRAMES model

According to the FRAMES model [ 68 ], all included studies reported the elements of responsibility and advice ( n =19; 100%) in the training of MI. The element responsibility is the shared responsibility of the learner’s growth by the learner and teacher. This could be attributed to the move towards competency-based medical education, which emphasizes shared responsibility among students while incorporating student-centric learning techniques and formative assessment as a vital element of the learning process [ 69 ]. In other words, the high reporting of ‘responsibility’ and ‘advice’ suggest that the present MI training significantly emphasizes medical students taking ownership of their learning and decision-making processes (‘responsibility’). Moreover, from a patient education perspective, empowering patients to take ownership of their health [ 70 ] and effectively guiding patients toward positive behavioural changes through good advice in a nonconfrontational approach is a basic tenet of MI (‘advice’).

The least reported element found in training for MI in our included studies [ 4 , 29 , 35 , 36 , 39 , 40 , 41 , 44 , 45 , 46 , 48 , 51 ] was self-efficacy. This may be due to MI training focusing less on self-efficacy and instead emphasizing other elements, such as empathy, open-ended questioning and reflective listening. An educational theory that is linked to the element of self-efficacy is social cognitive theory. Social cognitive theory can be defined as a person’s belief in their ability to determine the behaviours required to reach their desired goals and their perceptions of their ability and skills to manage their environment [ 71 , 72 ]. Continued research into integrating social cognitive theory into MI training could assist practitioners in comprehending the role and importance of self-efficacy in behaviour change and reflective practice. The lower reporting of ‘self-efficacy’ might also indicate a potential gap in MI training. Self-efficacy is essential because it relates to the practitioner’s confidence in their ability to effectively implement MI techniques and facilitate behaviour change in patients. Addressing this gap in future research could lead to more competent and confident practitioners who are better equipped to address challenging patient interactions and support positive health outcomes. Future studies can also utilize FRAMES to guide research design and interventions and investigate which aspects of FRAMES in the training of MI are most effective within the limited time frame of medical curricula.

Limitations

This scoping review is subject to several limitations. We included only English-language studies in which medical students were the target participants. We did not include articles that are categorized as grey literature or other forms of nonpeer review articles, which might have resulted in biased outcomes. Most of the studies focused on evaluating learner knowledge and skills in MI, which might have limited the practical applications of MI to real patients. The first author conducted the search and screening of the articles. This may lead to selection bias and reduce the reliability of the study selection process. The protocol for this review was developed before the search was initiated but was not registered or published online, which increases the risk of selective reporting. The database search was limited to MEDLINE Complete and CINAHL Complete, which were accessed via EBSCOhost and the search engine Google Scholar. Although a comprehensive search was conducted, other databases that were relevant to the review, such as the PsycINFO and ERIC databases, were not included, potentially resulting in missing relevant articles. Kirkpatrick’s hierarchy was utilized to assess educational outcomes in this review. This approach may neglect other core aspects of educational interventions. Furthermore, although we have extensively searched various countries, most of the studies reported are from the USA ( n =11; 57.8%) or Germany ( n =4; 21.0%). A lack of diversity among studies in other regions may lead to biased outcomes.

Based on our review, the findings suggest that motivational interviewing can be taught effectively in medical schools via adaptations of MI and a variety of teaching approaches. However, there is a need for further research investigating standardized MI training across medical schools, the adequate dose for training in MI and the implementation of reflective practices that are supported by educational learning theories. Furthermore, longitudinal studies can assess the effectiveness of MI. Future studies may benefit from exploring and better understanding the relationship between MI and self-efficacy in their MI interventions. The FRAMES model can be used to guide research and explore which aspects of FRAMES are optimally delivered within the limited time frame of medical curricula.

Availability of data and materials

All data generated or analysed during this study are included in this published article.

Abbreviations

Attention-deficit/hyperactivity disorder

Behaviour Change Counselling Index

Brief motivational interviewing

Course Experience Questionnaire

Calgary-Cambridge Observation Guide

Cumulative Index of Nursing and Allied Health Literature

Feedback, Responsibility, Advice, Menu of Options, Empathy, Self-Efficacy

Human immunodeficiency virus

Helpful response questionnaire

Jefferson Scale of Physician Empathy

Large Group Activities

Learning Outcomes Questionnaire

Medical Literature Analysis and Retrieval System Online

• Motivational interviewing

Motivational Interviewing Confidence Scale

Motivational Interviewing Knowledge and Attitudes Test

Motivational interviewing network of trainers

Motivational interviewing skill code

Motivational interviewing treatment integrity

Mechanisms of Motivational Interview

O = open-ended questions, A = affirmations, R = reflections, and S = summaries to promote active listening

Objective Structured Clinical Examination

Preferred Reporting Items for Systematic reviews and Meta-analysis extension for Scoping Reviews guidelines

Small Group Activities

Simulated Patient

Tabacco Intervention Basic Skills

Theory of Planned Behaviour

Video Assessment of the Simulated Encounter

Spencer JC, Wheeler SB. A systematic review of Motivational Interviewing interventions in cancer patients and survivors. Patient Educ Couns. 2016;99(7):1099–105.

Article   Google Scholar  

Gabbay RA, Kaul S, Ulbrecht J, Scheffler NM, Armstrong DG. Motivational interviewing by podiatric physicians: a method for improving patient self-care of the diabetic foot. J Am Podiatr Med Assoc. 2011;101(1):78–84.

Miller W, Rollnick S. Motivational interviewing: preparing people for change. New York: 2nd The Guilford Press; 2002.

Google Scholar  

Erschens R, Fahse B, Festl-Wietek T, Herrmann-Werner A, Keifenheim KE, Zipfel S, et al. Training medical students in motivational interviewing using a blended learning approach: a proof-of-concept study. Front Psychol. 2023;14:1204810.

Searight HR. Counseling patients in primary care: evidence-based strategies. Am Fam Phys. 2018;98(12):719–28.

Miller ET, Spilker J. Readiness to change and brief educational interventions: successful strategies to reduce stroke risk. J Neurosci Nurs. 2003;35(4):215–22.

Jaguga F, Ott MA, Kwobah EK, Apondi E, Giusto A, Barasa J, et al. Adapting a substance use screening and brief intervention for peer-delivery and for youth in Kenya. SSM Ment Health. 2023;4:100254.

Frey AJ, Lee J, Small JW, Sibley M, Owens JS, Skidmore B, et al. Mechanisms of motivational interviewing: a conceptual framework to guide practice and research. Prev Sci. 2021;22(6):689–700.

Miller WR, Rollnick S. Motivational interviewing: Helping people change and grow. New York: Guilford Publications; 2023.

Rimayanti MU, O’Halloran PD, Shields N, Morris R, Taylor NF. Comparing process evaluations of motivational interviewing interventions for managing health conditions and health promotions: a scoping review. Patient Educ Counsel. 2022;105(5):1170–80.

Centis E, Petroni ML, Ghirelli V, Cioni M, Navacchia P, Guberti E, et al. Motivational interviewing adapted to group setting for the treatment of relapse in the behavioral therapy of obesity. A clinical audit. Nutrients. 2020;12(12):3881.

Ford ES, Bergmann MM, Kröger J, Schienkiewitz A, Weikert C, Boeing H. Healthy living is the best revenge: findings from the European Prospective Investigation Into Cancer and Nutrition-Potsdam study. Arch Intern Med. 2009;169(15):1355–62.

Lundahl BW, Kunz C, Brownell C, Tollefson D, Burke BL. A meta-analysis of motivational interviewing: twenty-five years of empirical studies. Res Soc Work Pract. 2010;20:137–60.

Carroll KM, Ball SA, Nich C, Martino S, Frankforter TL, Farentinos C, et al. Motivational interviewing to improve treatment engagement and outcome in individuals seeking treatment for substance abuse: a multisite effectiveness study. Drug Alcohol Depend. 2006;81(3):301–12.

Rongkavilit C, Wang B, Naar-King S, Bunupuradah T, Parsons JT, Panthong A, et al. Motivational interviewing targeting risky sex in HIV-positive young Thai men who have sex with men. Arch Sex Behav. 2015;44(2):329–40.

Sönmez Sari E, Kitiş Y. The effect of nurse-led motivational interviewing based on the trans-theoretical model on promoting physical activity in healthy older adults: a randomized controlled trial. Int J Nurs Pract. 2024;30(2):e13252.

Polcin D, Witbrodt J, Nayak MB, Korcha R, Pugh S, Salinardi M. Characteristics of women with alcohol use disorders who benefit from intensive motivational interviewing. Subst Abus. 2022;43(1):23–31.

Lovejoy TI. Telephone-delivered motivational interviewing targeting sexual risk behavior reduces depression, anxiety, and stress in HIV-positive older adults. Ann Behav Med. 2012;44(3):416–21.

BlansonHenkemans OA, van der Boog PJ, Lindenberg J, van der Mast CA, Neerincx MA, Zwetsloot-Schonk BJ. An online lifestyle diary with a persuasive computer assistant providing feedback on self-management. Technol Health Care. 2009;17(3):253–67.

Meinzer MC, Oddo LE, Vasko JM, Murphy JG, Iwamoto D, Lejuez CW, et al. Motivational interviewing plus behavioral activation for alcohol misuse in college students with ADHD. Psychol Addict Behav. 2021;35(7):803–16.

Waite I, Grant D, Mayes J, Greenwood S. Can a brief behavioural change intervention encourage hospital patients with low physical activity levels to engage and initiate a change in physical activity behaviour? Physiotherapy. 2020;108:22–8.

Oveisi S, Stein L, Babaeepour E, Araban M. The impact of motivational interviewing on relapse to substance use among women in Iran: a randomized clinical trial. BMC psychiatry. 2020;20:1–7.

Schmiege SJ, Magnan RE, Yeater EA, Ewing SWF, Bryan AD. Randomized trial to reduce risky sexual behavior among justice-involved adolescents. Am J Prev Med. 2021;60(1):47–56.

Saffari M, Sanaeinasab H, Mobini M, Sepandi M, Rashidi-Jahan H, Sehlo MG, et al. Effect of a health-education program using motivational interviewing on oral health behavior and self-efficacy in pregnant women: a randomized controlled trial. Eur J Oral Sci. 2020;128(4):308–16.

Riley L, Guthold R, Cowan M, Savin S, Bhatti L, Armstrong T, et al. The World Health Organization STEPwise approach to noncommunicable disease risk-factor surveillance: methods, challenges, and opportunities. Am J Public Health. 2016;106(1):74–8.

D’Urzo KA, Flood SM, Baillie C, Skelding S, Dobrowolski S, Houlden RL, et al. Evaluating the implementation and impact of a motivational interviewing workshop on medical student knowledge and social cognitions towards counseling patients on lifestyle behaviors. Teach Learn Med. 2020;32(2):218–30.

Levensky ER, Forcehimes A, O’Donohue WT, Beitz K. Motivational interviewing: an evidence-based approach to counseling helps patients follow treatment recommendations. Am J Nurs. 2007;107(10):50–8.

Berger DJ, Nickolich S, Nasir M. Introduction to tobacco cessation and motivational interviewing: evaluation of a lecture and case-based learning activity for medical students. Cureus. 2024;16(2):e53704.

Edwards EJ, Arora B, Green P, Bannatyne AJ, Nielson T. Teaching brief motivational interviewing to medical students using a pedagogical framework. Patient Educ Counsel. 2022;105(7):2315–9.

Manojna GS, Madhavi BD. Effectiveness of teaching motivational interview technique to third professional year medical students to improve counseling skills–An interventional study. MRIMS J Health Sci. 2024:10.4103. [Epub ahead of print] June 19, 2024.

Kaltman S, Tankersley A. Teaching motivational interviewing to medical students: a systematic review. Acad Med. 2020;95(3):458–69.

Kirkpatrick JD, Kirkpatrick WK. Kirkpatrick’s four levels of training evaluation. Alexandria, VA: Association for Talent Development; 2016.

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Tricco A, Lillie E, Zarin W, O’Brien K, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169:467–73.

Bell K, Cole BA. Improving medical students’ success in promoting health behavior change: a curriculum evaluation. J Gen Intern Med. 2008;23(9):1503–6.

Brown RL, Pfeifer JM, Gjerde CL, Seibert CS, Haq CL. Teaching patient-centered tobacco intervention to first-year medical students. J Gen Intern Med. 2004;19(5):534–9.

Martino S, Haeseler F, Belitsky R, Pantalon M, Fortin AI. Teaching brief motivational interviewing to Year three medical students. Med Educ. 2007;41(2):160–7.

Mounsey AL, Bovbjerg V, White L, Gazewood J. Do students develop better motivational interviewing skills through role-play with standardised patients or with student colleagues? Med Educ. 2006;40(8):775–80.

Poirier MK, Clark MM, Cerhan JH, Pruthi S, Geda YE, Dale LC. Teaching motivational interviewing to first-year medical students to improve counseling skills in health behavior change. Mayo Clinic Proc. 2004;79:327–31 Elsevier.

White LL, Gazewood JD, Mounsey AL. Teaching students behavior change skills: description and assessment of a new Motivational interviewing curriculum. Med Teach. 2007;29(4):e67-71.

Haeseler F, Fortin AHT, Pfeiffer C, Walters C, Martino S, Haeseler F, et al. Assessment of a motivational interviewing curriculum for year 3 medical students using a standardized patient case. Patient Educ Counsel. 2011;84(1):27–30.

Lim BT, Moriarty H, Huthwaite M. 'Being-in-role’: a teaching innovation to enhance empathic communication skills in medical students. Med Teach. 2011;33(12):e663-9.

Opheim A, Andreasson S, Eklund AB, Prescott P. The effects of training medical students in motivational interviewing. Health Educ J. 2009;68(3):170–8.

Brogan Hartlieb K, Engle B, Obeso V, Pedoussaut MA, Merlo LJ, Brown DR. Advanced patient-centered communication for health behavior change: motivational interviewing workshops for medical learners. MedEdPORTAL. 2016;12:10455.

Gecht-Silver M, Lee D, Ehrlich-Jones L, Bristow M. Evaluation of a motivational interviewing training for third-year medical students. Fam Med. 2016;48(2):132–5.

Kaltman S, WinklerPrins V, Serrano A, Talisman N. Enhancing motivational interviewing training in a family medicine clerkship. Teach Learn Med. 2015;27(1):80–4.

Purkabiri K, Steppacher V, Bernardy K, Karl N, Vedder V, Borgmann M, et al. Outcome of a four-hour smoking cessation counselling workshop for medical students. Tob Induc Dis. 2016;14:37.

Jacobs NN, Calvo L, Dieringer A, Hall A, Danko R. Motivational interviewing training: a case-based curriculum for preclinical medical students. MedEdPORTAL. 2021;17:11104.

Keifenheim KE, Velten-Schurian K, Fahse B, Erschens R, Loda T, Wiesner L, et al. “A change would do you good”: Training medical students in Motivational Interviewing using a blended-learning approach - a pilot evaluation. Patient Educ Counsel. 2019;102(4):663–9.

Plass AM, Covic A, Lohrberg L, Albright G, Goldman R, Von Steinbüchel N. Effectiveness of a minimal virtual motivational interviewing training for first years medical students: differentiating between pre-test and then-test. Patient Educ Counsel. 2022;105(6):1457–62.

Martino S, Haeseler F, Belitsky R, Pantalon M, Fortin AHT. Teaching brief motivational interviewing to Year three medical students. Med Educ. 2007;41(2):160–7.

Miller WR. The evolution of motivational interviewing. Behav Cogn Psychother. 2023:1–17.

Small JW, Frey A, Lee J, Seeley JR, Scott TM, Sibley MH. Fidelity of motivational interviewing in school-based intervention and research. Prev Sci. 2021;22(6):712–21.

Rollnick S, Kaplan SG, Rutschman R. Motivational interviewing in schools: conversations to improve behavior and learning. Guilford Publications; 2016.

Pincus R, Bridges CW, Remley TP. School counselors using motivational interviewing. J Prof Counsel Pract Theory Res. 2018;45(2):82–94.

Lin CH, Chiang SL, Heitkemper MM, Hung YJ, Lee MS, Tzeng WC, et al. Effects of telephone-based motivational interviewing in lifestyle modification program on reducing metabolic risks in middle-aged and older women with metabolic syndrome: a randomized controlled trial. Int J Nurs Stud. 2016;60:12–23.

Dobber J, Latour C, Snaterse M, van Meijel B, ter Riet G, Scholte op Reimer W, et al. Developing nurses’ skills in motivational interviewing to promote a healthy lifestyle in patients with coronary artery disease. Eur J Cardiovasc Nurs. 2019;18(1):28–37.

Almansour M, AlQurmalah SI, Abdul Razack HI. Motivational interviewing-an evidence-based, collaborative, goal-oriented communication approach in lifestyle medicine: a comprehensive review of the literature. J Taibah Univ Med Sci. 2023;18(5):1170–8.

Polcin DL, Korcha R, Witbrodt J, Mericle AA, Mahoney E. Motivational Interviewing Case Management (MICM) for persons on probation or parole entering sober living houses. Crim Justice Behav. 2018;45(11):1634–59.

Sarpavaara H. The causes of change and no-change in substance users’ talk during motivational interviewing in the probation service in Finland. Int J Offender Ther Comp Criminol. 2017;61(4):430–44.

Bottel L, te Wildt BT, Brand M, Pape M, Herpertz S, Dieris-Hirche J. Telemedicine as bridge to the offline world for person affected with problematic internet use or internet use disorder and concerned significant others. Digit Health. 2023;9:20552076221144184.

Creech SK, Pulverman CS, Kahler CW, Orchowski LM, Shea MT, Wernette GT, et al. Computerized intervention in primary care for women veterans with sexual assault histories and psychosocial health risks: a randomized clinical trial. J Gen Intern Med. 2022;37(5):1097–107.

Duthie CJ, Cameron C, Smith-Han K, Beckert L, Delpachitra S, Garland SN, et al. Reasons for why medical students prefer specific sleep management strategies. Behav Sleep Med. 2024;22(4):516–29.

Clancy R, Taylor A. Engaging clinicians in motivational interviewing: comparing online with face-to-face post-training consolidation. Int J Ment Health Nurs. 2016;25(1):51–61.

Frost H, Campbell P, Maxwell M, O’Carroll RE, Dombrowski SU, Williams B, et al. Effectiveness of Motivational Interviewing on adult behaviour change in health and social care settings: a systematic review of reviews. PloS one. 2018;13(10):e0204890.

Schaper K, Woelber JP, Jaehne A. Can the spirit of motivational interviewing be taught online? A comparative study in general practitioners. Patient Educ Counsel. 2024;125:108297.

Kolb DA. Experience as the source of learning and development. Upper Sadle River: Prentice Hall; 1984.

Miller WR, Sanchez VC. Motivating young adults for treatment and lifestyle change. 1994.

Frank JR, Mungroo R, Ahmad Y, Wang M, De Rossi S, Horsley T. Toward a definition of competency-based education in medicine: a systematic review of published definitions. Med Teach. 2010;32(8):631–7.

Lim E, Wynaden D, Heslop K. Using Q-methodology to explore mental health nurses’ knowledge and skills to use recovery-focused care to reduce aggression in acute mental health settings. Int J Ment Health Nurs. 2021;30(2):413–26.

Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191.

Lönnfjord V, Hagquist C. The psychometric properties of the Swedish version of the general self-efficacy scale: a Rasch analysis based on adolescent data. Curr Psychol. 2018;37:703–15.

Download references

Acknowledgements

Additionally, the authors would like to thank Universiti Malaysia Sarawak for the support provided for this publication.

Open Access funding provided by Universiti Malaysia Sarawak.

Author information

Authors and affiliations.

Faculty of Medicine and Health Sciences, Universiti Malaysia Sarawak (UNIMAS), Kota Samarahan, Sarawak, 94300, Malaysia

Leonard Yik Chuan Lei, Keng Sheng Chew, Chee Shee Chai & Yoke Yong Chen

You can also search for this author in PubMed   Google Scholar

Contributions

LLYC, the first author, made significant contributions to developing the idea, the searches, conducting analysis and was responsible for drafting the manuscript. KSC and CYY contributed significantly to the conceptualization, alignment and reviewing of the manuscript. KSC and CYY and CCS participated in the analysis and writing of the manuscript. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Leonard Yik Chuan Lei .

Ethics declarations

Ethics approval and consent to participate.

This study did not require ethical approval or consent to participate.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Lei, L., Chew, K., Chai, C. et al. Evidence for motivational interviewing in educational settings among medical schools: a scoping review. BMC Med Educ 24 , 856 (2024). https://doi.org/10.1186/s12909-024-05845-w

Download citation

Received : 11 April 2024

Accepted : 30 July 2024

Published : 08 August 2024

DOI : https://doi.org/10.1186/s12909-024-05845-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Scoping review
• Motivational behaviour
• Motivational change
• Motivational enhancement
• Medical education
• Medical teaching

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• News & Events

Attorneys Author Peer-Reviewed Article Proposing Changes to Research Misconduct Regulatory Framework in Light of PubPeer Comments

Online platform PubPeer was established to encourage discourse on scientific research. But for institutions that publish research, investigating comments that appear on PubPeer and other internet sources can be burdensome.

In an Accountability in Research article , health care and educational institutions counsel Minal Caron , associate Carolyn Lye and partner Mark Barnes , and Dr. Barbara Bierer of Harvard Medical School and Brigham and Women’s Hospital, describe the institutional challenges of handling PubPeer comments within the existing regulatory framework for reviewing allegations of research misconduct relating to federally funding research.

The authors propose new regulatory interpretations and provisions to ensure that specific, credible allegations are rigorously assessed, while reducing burden on institutions tasked with investigating the multitude of comments on PubPeer and other internet sources.

Accountability in Research is a peer-reviewed interdisciplinary journal focused on ethics, standards and policy issues in science, medicine, law, management and public policy.

Related Services

Stay Up To Date with Ropes & Gray

Ropes & Gray attorneys provide timely analysis on legal developments, court decisions and changes in legislation and regulations.

Stay in the loop with all things Ropes & Gray, and find out more about our people, culture, initiatives and everything that’s happening.

We regularly notify our clients and contacts of significant legal developments, news, webinars and teleconferences that affect their industries.

• Open access
• Published: 07 August 2024

Impact of neonatal sepsis on neurocognitive outcomes: a systematic review and meta-analysis

• Wei Jie Ong   ORCID: orcid.org/0000-0001-8244-2977 1   na1 ,
• Jun Jie Benjamin Seng   ORCID: orcid.org/0000-0002-3039-3816 1 , 2 , 3   na1 ,
• Beijun Yap 1 ,
• George He 4 ,
• Nooriyah Aliasgar Moochhala 4 ,
• Chen Lin Ng 1 ,
• Rehena Ganguly   ORCID: orcid.org/0000-0001-9347-5571 5 ,
• Jan Hau Lee   ORCID: orcid.org/0000-0002-8430-4217 6 &
• Shu-Ling Chong   ORCID: orcid.org/0000-0003-4647-0019 7  

BMC Pediatrics volume  24 , Article number:  505 ( 2024 ) Cite this article

311 Accesses

1 Altmetric

Metrics details

Introduction

Sepsis is associated with neurocognitive impairment among preterm neonates but less is known about term neonates with sepsis. This systematic review and meta-analysis aims to provide an update of neurocognitive outcomes including cognitive delay, visual impairment, auditory impairment, and cerebral palsy, among neonates with sepsis.

We performed a systematic review of PubMed, Embase, CENTRAL and Web of Science for eligible studies published between January 2011 and March 2023. We included case–control, cohort studies and cross-sectional studies. Case reports and articles not in English language were excluded. Using the adjusted estimates, we performed random effects model meta-analysis to evaluate the risk of developing neurocognitive impairment among neonates with sepsis.

Of 7,909 studies, 24 studies ( n  = 121,645) were included. Majority of studies were conducted in the United States ( n  = 7, 29.2%), and all studies were performed among neonates. 17 (70.8%) studies provided follow-up till 30 months. Sepsis was associated with increased risk of cognitive delay [adjusted odds ratio, aOR 1.14 (95% CI: 1.01—1.28)], visual impairment [aOR 2.57 (95%CI: 1.14- 5.82)], hearing impairment [aOR 1.70 (95% CI: 1.02–2.81)] and cerebral palsy [aOR 2.48 (95% CI: 1.03–5.99)].

Neonates surviving sepsis are at a higher risk of poorer neurodevelopment. Current evidence is limited by significant heterogeneity across studies, lack of data related to long-term neurodevelopmental outcomes and term infants.

Peer Review reports

Sepsis is a major cause of mortality and morbidity among neonates [ 1 , 2 , 3 , 4 ]. Young infants especially neonates, defined by age < 28 days old, have a relatively immature immune system and are susceptible to sepsis [ 5 , 6 ]. Annually, there are an estimated 1.3 to 3.9 million cases of infantile sepsis worldwide and up to 700,000 deaths [ 7 ]. Low-income and middle-income countries bear a disproportionate burden of neonatal sepsis cases and deaths [ 7 , 8 ]. While advances in medical care over the past decade have reduced mortality, neonates who survive sepsis are at risk of developing neurocognitive complications, which affect the quality of life for these children and their caregivers [ 9 ].

Previous reviews evaluating neurocognitive outcomes in neonates with infections or sepsis have focused on specific types of pathogens (e.g., Group B streptococcus or nosocomial infections [ 10 ]), or are limited to specific populations such as very low birth weight or very preterm neonates [ 11 ], and there remains paucity of data regarding neurocognitive outcomes among term and post-term neonates. There remains a gap for an updated comprehensive review which is not limited by type of pathogen or gestation. In this systematic review, we aim to provide a comprehensive update to the current literature on the association between sepsis and the following adverse neurocognitive outcomes (1) mental and psychomotor delay (cognitive delay (CD)), (2) visual impairment, (3) auditory impairment and (4) cerebral palsy (CP) among neonates [ 11 ].

We performed a systematic review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines [ 12 ]. This study protocol was registered with Open Science Framework ( https://doi.org/10.17605/OSF.IO/B54SE ).

Eligibility criteria

We identified studies which evaluated neurocognitive outcomes in neonates less than 90 days old (regardless of gestational age) with sepsis. While the neonatal period is traditionally defined to be either the first 28 days postnatally for term and post-term infants, or 27 days after the expected date of delivery for preterm infants [ 13 ], serious late onset infections in the young infant population can present beyond the neonatal period [ 14 ], hence we defined the upper age limit as 90 days old to obtain a more complete picture of the burden of young infantile sepsis [ 15 ]. Post-term neonates was defined as a neonate delivered at >  = 42 weeks of gestational age in this study [ 16 ]. We included studies that either follow international sepsis definitions such as Surviving Sepsis Campaign guidelines definitions [ 17 ], or if they fulfilled clinical, microbiological and/or biochemical criteria for sepsis as defined by study authors. The primary outcome of interest was impaired neurocognitive outcome defined by the following domains of neurodevelopmental impairment (NDI) [ 11 ]: (1) CD, (2) visual impairment, (3) auditory impairment and (4) CP. We selected these domains because they were highlighted as key neurocognitive sequelae after intrauterine insults in a landmark review by Mwaniki et al. [ 18 ]. The authors’ definitions of these outcomes and their assessment tools were captured, including the use of common validated instruments (e.g., a common scale used for CD is the Bayley Scales of Infant Development (BSID) [ 19 ] while a common instrument used for CP was the Gross Motor Function Classification System (GMFCS) [ 20 ]. Specifically for BSID, its two summative indices score – Mental Development Index (MDI) and Psychomotor Development Index (PDI) were collected. The MDI assesses both the non-verbal cognitive and language skills, while PDI assess the combination of fine and gross motor skills. The cut-off points for mild, moderate and severe delay for MDI and PDI were < 85 or < 80, < 70 and < 55 respectively [ 21 ]. There were no restrictions on duration of follow-up or time of assessment of neurocognitive outcomes to allow capturing of both short- and long-term neurocognitive outcomes.

Case–control, cohort studies and cross-sectional studies published between January 2011 and March 2023 were included. Because the definition and management of sepsis has evolved over the years [ 22 ], we chose to include studies published from 2011 onwards. Case reports, animal studies, laboratory studies and publications that were not in English language were excluded. Hand-searching of previous systematic reviews were performed to ensure all relevant articles were included. To avoid small study effects, we also excluded studies with a sample size of less than 50 [ 23 ].

Information sources and search strategy

Four databases (PubMed, Cochrane Central, Embase and Web of Science) were used to identify eligible studies. The search strategy was developed in consultation with a research librarian. The first search was conducted on 4 December 2021 and an updated search was conducted on 3 April 2023. The detailed search strategy can be found in Supplementary Tables 1A and B.

Study selection process

Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia) [ 24 ] was utilized during this review. Five reviewers (WJO, BJY, NM, CLN and GH) independently conducted the database search and screened the title and abstracts for relevance. Following training on inclusion and exclusion eligibility, 4 reviewers (WJO, NM, CLN and GH) subsequently assessed the full text of shortlisted articles for eligibility. All full texts were independently assessed by at least 2 reviewers. Any conflict related to study eligibility were resolved in discussion with the senior author (S-LC). We recorded the reason(s) for exclusion of each non-eligible article.

Data collection process and data items

Four reviewers (WJO, NM, CLN and GH) independently carried out the data extraction using a standardized data collection form, and any conflict was resolved by discussion, or with input from the senior author (S-LC). A pilot search was performed for the first 200 citations to evaluate concordance among reviewers and showed good concordance among reviewers of 94%. For studies with missing data required for data collection or meta-analyses, we contacted the corresponding authors of articles to seek related information. If there was no reply from the authors, the data were labelled as missing.

Study risk of bias assessment

Three reviewers (BJY, GH and WJO) independently carried out the assessment of risk of bias using the Newcastle–Ottawa Scale (NOS) for all observational studies [ 25 ]. Studies were graded based on three domains namely, selection, comparability and outcomes. Studies were assigned as low, moderate and high risk of bias if they were rated 0–2 points, 3–5 points and 6–9 points respectively. Any conflict was resolved by discussion or with input from the senior author (S-LC).

Statistical analysis

All outcomes (i.e. CD, visual impairment, auditory impairment and CP) were analysed as categorical data. Analyses were done for each NDI domain separately. To ensure comparability across scales, results from different studies were only pooled if the same measurement tools were used to assess the outcomes and hence sub-group analyses were based on different scales and/or different definitions of neurocognitive outcomes used by authors. Both unadjusted and adjusted odds ratios (aOR) and/or relative risk (RR) for each NDI domain were recorded. Where source data were present, we calculated the unadjusted OR if the authors did not report one, together with the 95% confidence interval (CI). For adjusted odds ratio, these were extracted from individual studies and variables used for adjustment were determined at the individual study level.

Meta-analysis was conducted for all outcomes that were reported by at least 2 independent studies or cohorts. Studies were included in the meta-analysis only if they reported outcomes for individual NDI domains within 30 months from sepsis occurrence. For each domain, all selected studies were pooled using DerSimonian-Laird random effects model due to expected heterogeneity. Studies were pooled based on adjusted and unadjusted analyses. Case–control and cohort studies were pooled separately. The pooled results were expressed as unadjusted odds ratio (OR) or adjusted odds ratio (aOR) with corresponding 95% confidence interval (95% CI). If there was more than 1 study that utilized the same population, we only analysed data from the most recent publication or from the larger sample size, to avoid double counting. Standard error (SE) from studies with multiple arms with same control group were adjusted using SE = √(K/2), where K refers to number of treatment arms including control [ 26 ]. Heterogeneity across studies was evaluated using the I^2 statistic, for which ≥ 50% is indicative of significant heterogeneity. With regards to publication bias, this was performed using Egger’s test and funnel plots only if the number of studies pooled were 10 or more for each outcome.

For neurocognitive related outcomes, subgroup analyses were performed based on the severity of the NDI domain outcomes and distinct, non-overlapping populations of septic infants (such as late onset vs early onset sepsis, culture positive sepsis vs clinically diagnosed sepsis, term and post term patients).

All analyses were done using ‘meta’ library from R software (version 4.2.2) [ 27 ]. The statistical significance threshold was a two tailed P- value < 0.05.

Certainty of evidence

The certainty of evidence for outcomes in this review was performed during the GRADE criteria [ 28 ] which is centred on the study design, risk of bias, inconsistency, indirectness, imprecision, and other considerations.

Study selection

From 7,909 studies identified, a total of 24 articles were included (Fig.  1 ) [ 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 ]. A total of 101,657 and 19,988 preterm and term infants were included in this review.

PRISMA flowchart of the study selection process for search

Study characteristics

There were 2 case–control studies and 22 cohort studies, with a total of 121,645 infants (Table  1 ). Studies were conducted in 16 different countries (Fig.  2 ), with the most studies conducted in the United States of America (USA) (7 studies, n  = 92,358 patients) [ 30 , 33 , 37 , 41 , 42 , 47 , 52 ]. There were no studies that were conducted solely on term infants. 5 studies reported data specifically on ELBW infants (27,078 infants) and 6 studies on VLBW infants (3,322 infants). All studies were performed among neonates.

World map depicting distribution of studies that evaluate neurocognitive outcomes in infantile and neonatal sepsis

Risk of bias 

Overall, all 24 studies were classified as low risk (Supplementary Table 2). 5 papers scored high risk for outcome bias for having greater than 10% of initial population being lost to follow-up [ 29 , 32 , 40 , 41 , 42 ].

Outcome measures reported by domain

As the number of studies pooled for each outcome was less than 10, publication bias was not analysed in the meta-analyses.

Cognitive delay (CD)

Among 24 studies that assessed for CD, 16 studies reported either the incidence of CD among young infants with sepsis compared to those without, and/or the odds ratio (adjusted and/or unadjusted) comparing the two populations [ 29 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 40 , 41 , 42 , 45 , 46 , 48 , 49 ]. The scales used, authors’ definition of CD, incidence of CD among those with sepsis and those without are described in Table  2 . The most common tools used for assessment of CD were the Bayley Scales of Infant Development (BSID) ( n  = 13) and Denver Development Screening Test II ( n  = 2).

Infantile sepsis was associated with increased risk of overall CD delays [aOR 1.14 (95%CI: 1.01, 1.28)], overall PDI delay (aOR 1.73 (95%CI: 1.16, 2.58)) and moderate PDI delay [aOR 1.85 (95%CI: 1.01, 3.36)]. Conversely, infantile sepsis was not associated with increased risk for severe PDI delay nor overall MDI delay [aOR 1.30 (95%CI: 0.99, 1.71)] or its subgroups. There were no significant differences in outcomes between different subgroups of infections as well as culture-proven or clinically defined sepsis for either MDI or PDI (Table  8 , Fig.  3 A and B).

A Forest plot on adjusted odds ratios for neurocognitive outcomes related to MDI, PDI, visual impairment, hearing impairment and cerebral palsy. B Forest plot on unadjusted odds ratios for neurocognitive outcomes related to MDI, PDI, visual impairment, hearing impairment and cerebral palsy. Legend: MDI: Mental Developmental Index; PDI: Psychomotor Developmental Index. Foot note: Mild MDI or PDI: < 85 or < 80; Moderate MDI or PDI < 70; Severe MDI or PDI < 55

Visual impairment

Seven studies reported data on visual impairment (Table  3 ) [ 31 , 33 , 41 , 42 , 47 , 49 ]. The most common definition of visual impairment utilized was “visual acuity of < 20/200” ( n  = 4, 66.7%).

In the meta-analysis, infantile sepsis was associated with significantly increased risk of visual impairment [aOR 2.57 (95%CI: 1.14, 5.82)] but there were no statistically significant differences in visual impairment between subgroups of early or late onset sepsis, and blood culture negative conditions as compared to the non-septic population (Table  8 , Fig.  3 A and B).

Hearing impairment

Seven studies reported data on hearing impairment (Table  4 ) [ 31 , 33 , 41 , 42 , 47 , 49 ]. Two studies defined hearing impairment as permanent hearing loss affecting communication with or without amplification [ 42 , 47 ]. Other definitions included “sensorineural hearing loss requiring amplification” ( n  = 1), “bilateral hearing impairment with no functional hearing (with or without amplification)” ( n  = 1), “clinical hearing loss” ( n  = 1).

In the meta-analysis, sepsis was associated with increased risk of hearing impairment [aOR 1.70 (95% CI: 1.02–2.81)]. However, in the subgroup analyses, there were no differences in risk of hearing impairment between patients with late onset sepsis as compared to the non-septic population (Table  8 , Fig.  3 A and B).

Cerebral palsy

Nine studies [ 29 , 32 , 33 , 41 , 42 , 47 , 48 , 49 , 50 ] reported data on CP (Table  5 ), of which 5 studies [ 41 , 42 , 45 , 49 , 50 ] used the GMFCS scale. In the meta-analysis, infantile sepsis was associated with significantly increased risk of CP [aOR 2.48 (95%CI: 1.03; 5.99)]. There was no difference in rates of CP among patients with proven or suspected sepsis, as compared with infants with no sepsis (Table  8 , Fig.  3 A and B).

Differences in neurocognitive outcomes between neonates with culture-proven or clinically diagnosed sepsis as well as early or late onset sepsis

Tables 6 and 7 showed data related to differences in neurocognitive outcomes between neonates with culture-proven or clinically diagnosed sepsis as well as early or late onset sepsis. Meta-analyses were not be performed due to significant heterogeneity in definitions of sepsis, time of assessment of outcomes.

Differences in neurocognitive outcomes between term and post-term neonates

There were no studies which evaluated neurocognitive outcomes between term and post-term neonates and infants.

We found that the certainty of evidence to be very low to low for the four main neurocognitive outcomes selected. (Supplementary File 3).

In this review involving more than 121,000 infants, we provide an update to the literature regarding young infant sepsis and neurocognitive impairment. Current collective evidence demonstrate that young infant sepsis was associated with increased risk of developing neurocognitive impairment in all domains of CD, visual impairment, auditory impairment and cerebral palsy.

Cognitive delay

In this review, higher rates of cognitive delay were noted among infants with sepsis [ 29 , 31 , 33 , 34 , 35 , 36 , 37 , 38 , 40 , 41 , 42 , 45 , 46 , 48 , 49 , 52 ]. We found that infants with sepsis reported lower PDI scores (Table  8 ), which measures mainly neuromotor development. On the other hand, young infant sepsis was not associated with lower MDI scores (Table  8 ), which assesses cognitive and language development. The pathophysiological mechanism of young infant sepsis and its preferential impact on PDI remains unclear. Postulated mechanisms include development of white matter lesions which may arise from the susceptibility of oligodendrocyte precursors to inflammatory processes such as hypoxia and ischemia [ 53 ]. Future studies should look into evaluating the causes of the above findings. A majority of included studies focused on early CD outcomes while no studies evaluated long-term outcomes into adulthood. CD is known to involve complex genetic and experiential interactions [ 54 ] and may evolve overtime with brain maturation. Delays in speech and language, intellectual delay and borderline intellectual functioning are shown to be associated with poorer academic or employment outcomes in adulthood [ 55 , 56 ], and early assessment of CD may not fully reveal the extent of delays. The only study with follow-up to the adolescent phase showed a progressive increase in NDI rate as the participants aged, which provides evidence of incremental long-term negative outcomes associated with infantile sepsis [ 44 ]. Moving forward, studies with longer follow-up may allow for further examination of the long-term effects of neonatal sepsis on CD.

There were different versions of the BSID instrument (BSID-II and BSID-III) [ 19 , 57 , 58 ]. BSID-II lacked subscales in PDI and MDI scores, leading to the development of BSID-III with the segregation of PDI into fine and gross motor scales and MDI into cognitive, receptive language, and expressive language scales [ 59 ]. Although we pooled results of both BSID-II and BSID-III in our study, we recognize that comparisons between BSID-II and BSID-III are technically challenging due to differences in standardised scores [ 59 , 60 ]. In addition, the BSID-IV was created in 2019 which has fewer items, However, none of our studies utilized this instrument. Future studies should consider this instrument, as well as standardising the timepoints for assessment of CD.

Young infant sepsis was associated with increased risk of developing visual impairment. This was similar to results noted by a previous systematic review published in 2014 [ 61 ] and 2019 [ 62 ] which showed that neonatal sepsis was associated with twofold risk of developing retinopathy of prematurity in preterm infants. Specifically, meningitis was associated with a greater risk of visual impairment compared to just sepsis alone [ 47 ]. The mechanism of visual impairment has not been fully described although various theories have been suggested, including sepsis mediated vascular endothelial damage, increased body oxidative stress response as well as involvement of inflammatory cytokines and mediators [ 63 , 64 ].

Our meta-analysis showed an increased risk of hearing impairment for young infants with young infants with sepsis. This is consistent with a previous report that found an association between neonatal meningitis and sensorineural hearing loss [ 65 ]. One potential confounder which we were unable to account for may have been the use of ototoxic antimicrobial agents such as aminoglycosides. Additional confounders include very low birth weight, patient’s clinical states (e.g. hyperbilirubinemia requiring exchange transfusion) and use of mechanical ventilation or extracorporeal membrane support. To allow for meaningful comparisons of results across different study populations, it is imperative that a standardised definition of hearing impairment post neonatal sepsis be established for future studies.

Our meta-analysis found an association between neonatal sepsis and an increased risk of developing CP. This is also consistent with previous systematic reviews which had found a significant association of sepsis and CP in VLBW and early preterm infants [ 11 ]. One study found that infants born at full term and who experienced neonatal infections were at a higher risk of developing a spastic triplegia or quadriplegia phenotype of CP [ 66 ]. The pathophysiology and mechanism of injury to white matter resulting in increased motor dysfunction remains unclear and more research is required in this area.

Limitations and recommendations for future research

The main limitation of this review lies in the heterogeneity in the definitions of sepsis, exposures and assessment of outcomes across studies. This is likely attributed to the varying definition of sepsis used in different countries as well as lack of gold standard definitions or instruments for assessment of each component of NDI. A recent review of RCTs [ 67 ] also reported similar limitations where 128 different varying definitions of neonatal sepsis were used in literature. Notably, there is a critical need for developing international standardized guidelines for defining neonatal sepsis as well as assessment of NDI such as hearing and visual impairment. Another important limitation relates to the inability to assess quality of neonatal care delivered as well as temporal changes in medical practices which could have affected neurocognitive outcomes for neonates with sepsis. Improving quality of neonatal care has been shown to significantly reduce mortality risk among neonates with sepsis, especially in resource-poor countries [ 68 ]. We performed a comprehensive search strategy (PubMed, Embase, Web of Science and CENTRAL) coupled with hand searching of references within included systematic reviews, but did not evaluate grey literature. Future studies should include additional literature databases and grey literature. Another area of research gap lies in the paucity of data related to differences in neurocognitive outcomes between term and post-term neonates with sepsis and future research is required to bridge this area of research gap. Likewise, there are few studies which evaluated differences in neurocognitive outcomes between early or late onset sepsis and outcomes assessed were significantly heterogenous which limits meaningful meta-analyses. Similarly, there was significant heterogeneity in study outcomes, causative organisms and severity of disease.

We found a lack of long-term outcomes and recommend that future prospective cohorts include a longer follow-up duration as part of the study design. This is important given the implication of NDI on development into adulthood. Most data were reported for preterm infants with low birth weight, and there was a paucity of data for term infants in our literature review. Since prematurity itself is a significant cause of NDI [ 69 ], future studies should consider how gestational age and/or birth weight can be adequately adjusted for in the analysis.

Apart from the domains of NDI we chose to focus on in this review, there are other cognitive domains classified by the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-V) [ 70 ] and/or recommended by the Common Data Elements (CDE) workgroup [ 71 ]. Future studies may wish to look into the implications of sepsis on other neuro-cognitive domains related to executive function, complex attention and societal cognition which are studied for other types of acquired brain injury [ 71 , 72 ].

Our systematic review and meta-analysis found that neonates surviving sepsis are at a higher risk of poorer neurodevelopment. However, the evidence is limited by significant heterogeneity and selection bias due to differing definitions used for NDI and for sepsis. There is also a lack of long-term follow-up data, as well as data specific for term and post-term infants. Future prospective studies should be conducted with long-term follow-up to assess the impact of neurodevelopmental impairment among all populations of neonates with sepsis.

Availability of data and materials

All data generated or analyzed in the study are found in the tables and supplementary materials.

Liu L, Johnson HL, Cousens S, Perin J, Scott S, Lawn JE, et al. Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet. 2012;379:2151–61.

Article   PubMed   Google Scholar  

WHO. Newborns: improving survival and well-being. Geneve: World Health Organisation; 2020.

Chiesa C, Panero A, Osborn JF, Simonetti AF, Pacifico L. Diagnosis of neonatal sepsis: a clinical and laboratory challenge. Clin Chem. 2004;50:279–87.

Article   CAS   PubMed   Google Scholar  

Ramaswamy VV, Abiramalatha T, Bandyopadhyay T, Shaik NB, Bandiya P, Nanda D, et al. ELBW and ELGAN outcomes in developing nations-Systematic review and meta-analysis. PLoS ONE. 2021;16:e0255352.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Zhang X, Zhivaki D, Lo-Man R. Unique aspects of the perinatal immune system. Nat Rev Immunol. 2017;17:495–507.

Prabhudas M, Adkins B, Gans H, King C, Levy O, Ramilo O, et al. Challenges in infant immunity: Implications for responses to infection and vaccines. Nat Immunol. 2011;12:189–94.

World Health Organization. Global report on the epidemiology and burden of sepsis. 2020. Available from: https://www.who.int/publications/i/item/9789240010789 .

Milton R, Gillespie D, Dyer C, Taiyari K, Carvalho MJ, Thomson K, et al. Neonatal sepsis and mortality in low-income and middle-income countries from a facility-based birth cohort: an international multisite prospective observational study. Lancet Glob Health. 2022May 1;10(5):e661-72. https://doi.org/10.1016/S2214-109X(22)00043-2 .

Li Y, Ji M, Yang J. Current understanding of long-term cognitive impairment after sepsis. Front Immunol. 2022;13:855006.

Haller S, Deindl P, Cassini A, Suetens C, Zingg W, Abu Sin M, et al. Neurological sequelae of healthcare-associated sepsis in very-low-birthweight infants: Umbrella review and evidence-based outcome tree. Euro Surveill. 2016;21:30143.

Alshaikh B, Yusuf K, Sauve R. Neurodevelopmental outcomes of very low birth weight infants with neonatal sepsis: Systematic review and meta-analysis. J Perinatol. 2013;33:558–64.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. https://doi.org/10.1136/bmj.n71 .

U.S. Department of Health and Human Services F and D, Administration C for DE and R (CDER), (CBER) C for BE and R. General Clinical Pharmacology Considerations for Neonatal Studies for Drugs and Biological Products Guidance for Industry. 2019. Available from: https://www.fda.gov/media/129532/download . [cited 2022 Aug 9].

Bizzarro MJ, Shabanova V, Baltimore RS, Dembry LM, Ehrenkranz RA, Gallagher PG. Neonatal sepsis 2004–2013: the rise and fall of coagulase-negative staphylococci. J Pediatr. 2015;166:1193–9.

Article   PubMed   PubMed Central   Google Scholar  

Goddard B, Chang J, Sarkar IN. Using self organizing maps to compare sepsis patients from the neonatal and adult intensive care unit. AMIA Jt Summits Transl Sci Proc. 2019;2019:127–35.

PubMed   PubMed Central   Google Scholar  

Galal M, Symonds I, Murray H, Petraglia F, Smith R. Postterm pregnancy. Facts Views Vis Obgyn. 2012;4(3):175–87. Available from: https://www.ncbi.nlm.nih.gov/pubmed/24753906 .

Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intensive Care Med. 2021;47:1181–247.

Mwaniki MK, Atieno M, Lawn JE, Newton CRJC. Long-term neurodevelopmental outcomes after intrauterine and neonatal insults: a systematic review. Lancet. 2012;379:445–52.

Bayley N. Bayley scales of infant and toddler development, Third edition: screening test manual. San Antonio, Texas: Pearson Clinical Assessment PsychCorp; 2006.

Google Scholar  

Palisano R, Rosenbaum P, Walter S, Russell D, Wood E, Galuppi B. Development and reliability of a system to classify gross motor function in children with cerebral palsy. Dev Med Child Neurol. 1997;39:214–23.

Spencer-Smith MM, Spittle AJ, Lee KJ, Doyle LW, Anderson PJ. Bayley-III cognitive and language scales in preterm children. Pediatrics. 2015;135(5):e1258-65.

Survival Sepsis Campaign. https://www.sccm.org/SurvivingSepsisCampaign/About-SSC/History . History of Surviving Sepsis Campaign | SCCM.

Tan B, Wong JJM, Sultana R, Koh JCJW, Jit M, Mok YH, et al. Global Case-Fatality Rates in Pediatric Severe Sepsis and Septic Shock: A Systematic Review and Meta-analysis. JAMA Pediatr. 2019;173:352–62.

Covidence systematic review software. Melbourne, Australia: Veritas Health Innovation; Available from: www.covidence.org .

GA Wells, B Shea, D O’Connell, J Peterson, V Welch, M Losos PT. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Available from: https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp . [cited 2022 Aug 9].

Rücker G, Cates CJ, Schwarzer G. Methods for including information from multi-arm trials in pairwise meta-analysis. Res Synth Methods. 2017;8:392–403.

R Core Team. R core team (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org . 2021.

Schünemann H, Oxman JBGGA. GRADE Handbook. עלון הנוטע. 2013;66(1997).

Schlapbach LJ, Aebischer M, Adams M, Natalucci G, Bonhoeffer J, Latzin P, et al. Impact of sepsis on neurodevelopmental outcome in a swiss national cohort of extremely premature infants. Pediatrics. 2011;128:e348-57.

Adams-Chapman I, Bann CM, Das A, Goldberg RN, Stoll BJ, Walsh MC, et al. Neurodevelopmental outcome of extremely low birth weight infants with Candida infection. J Pediatr. 2013;163(4):961-7.e3.

de Haan TR, Beckers L, de Jonge RCJ, Spanjaard L, van Toledo L, Pajkrt D, et al. Neonatal gram negative and candida sepsis survival and neurodevelopmental outcome at the corrected age of 24 months. PLoS One. 2013;8:e59214.

Mitha A, Foix-L’Hélias L, Arnaud C, Marret S, Vieux R, Aujard Y, et al. Neonatal infection and 5-year neurodevelopmental outcome of very preterm infants. Pediatrics. 2013;132:e372-80.

Alshaikh B, Yee W, Lodha A, Henderson E, Yusuf K, Sauve R. Coagulase-negative staphylococcus sepsis in preterm infants and long-term neurodevelopmental outcome. J Perinatol. 2014;34:125–9.

Ferreira RC, Mello RR, Silva KS. Neonatal sepsis as a risk factor for neurodevelopmental changes in preterm infants with very low birth weight. J Pediatr (Rio J). 2014;90:293–9.

Hentges CR, Silveira RC, Procianoy RS, Carvalho CG, Filipouski GR, Fuentefria RN, et al. Association of late-onset neonatal sepsis with late neurodevelopment in the first two years of life of preterm infants with very low birth weight. J Pediatr (Rio J). 2014;90:50–7.

Dangor Z, Lala SG, Cutland CL, Koen A, Jose L, Nakwa F, et al. Burden of invasive group B Streptococcus disease and early neurological sequelae in South African infants. PLoS One. 2015;10:e0123014.

Savioli K, Rouse C, Susi A, Gorman G, Hisle-Gorman E. Suspected or known neonatal sepsis and neurodevelopmental delay by 5 years. J Perinatol. 2018;38:1573–80.

Singh L, Das S, Bhat VB, Plakkal N. Early neurodevelopmental outcome of very low birthweight neonates with culture-positive blood stream infection: a prospective cohort study. Cureus. 2018;10:e3492.

Zonnenberg IA, van Dijk-Lokkart EM, van den Dungen FAM, Vermeulen RJ, van Weissenbruch MM. Eur J Pediatr. 2019;178:673–80.

Nakwa FL, Lala SG, Madhi SA, Dangor Z. Neurodevelopmental impairment at 1 year of age in infants with previous invasive group B streptococcal sepsis and meningitis. Pediatric Infect Dis J. 2020;39:794–8.

Article   Google Scholar  

Mukhopadhyay S, Puopolo KM, Hansen NI, Lorch SA, Demauro SB, Greenberg RG, et al. Neurodevelopmental outcomes following neonatal late-onset sepsis and blood culture-negative conditions. Arch Dis Child Fetal Neonatal Ed. 2021;106:467–73.

Mukhopadhyay S, Puopolo KM, Hansen NI, Lorch SA, DeMauro SB, Greenberg RG, et al. Impact of early-onset sepsis and antibiotic use on death or survival with neurodevelopmental impairment at 2 years of age among extremely preterm infants. J Pediatr. 2020;221:39-46.e5.

Horváth-Puhó E, Snoek L, van Kassel MN, Gonçalves BP, Chandna J, Procter SR, et al. Prematurity modifies the risk of long-term neurodevelopmental impairments after invasive group B streptococcus infections during infancy in Denmark and the Netherlands. Clin Infect Dis. 2021;74:S44–53.

Article   PubMed Central   Google Scholar  

Horváth-Puhó E, van Kassel MN, Gonçalves BP, de Gier B, Procter SR, Paul P, et al. Mortality, neurodevelopmental impairments, and economic outcomes after invasive group B streptococcal disease in early infancy in Denmark and the Netherlands: a national matched cohort study. Lancet Child Adolesc Health. 2021;5:398–407.

Ortgies T, Rullmann M, Ziegelhöfer D, Bläser A, Thome UH. The role of early-onset-sepsis in the neurodevelopment of very low birth weight infants. BMC Pediatr. 2021;21:289.

Shim SY, Cho SJ, Park EA. Neurodevelopmental outcomes at 18–24 months of corrected age in very low birth weight infants with late-onset sepsis. J Korean Med Sci. 2021;36:e205.

Brumbaugh JE, Bell EF, Do BT, Greenberg RG, Stoll BJ, Demauro SB, et al. Incidence of and neurodevelopmental outcomes after late-onset meningitis among children born extremely preterm. JAMA Netw Open. 2022;5(12):e2245826.

Golin MO, Souza FIS, Paiva L da S, Sarni ROS. The value of clinical examination in preterm newborns after neonatal sepsis: a cross-sectional observational study. Dev Neurorehabil. 2022;25(2):80–6.

Humberg A, Fortmann MI, Spiegler J, Rausch TK, Siller B, Silwedel C, et al. Recurrent late-onset sepsis in extremely low birth weight infants is associated with motor deficits in early school age. Neonatology. 2022;119(6):695–702.

Kartam M, Embaireeg A, Albalool S, Almesafer A, Hammoud M, Al-Hathal M, et al. Late-onset sepsis in preterm neonates is associated with higher risks of cerebellar hemorrhage and lower motor scores at three years of age. Oman Med J. 2022;37(2):e368.

Paul P, Chandna J, Procter SR, Dangor Z, Leahy S, Santhanam S, et al. Neurodevelopmental and growth outcomes after invasive Group B Streptococcus in early infancy: a multi-country matched cohort study in South Africa, Mozambique, India, Kenya, and Argentina. EClinicalMedicine. 2022;47:101358.

Bright HR, Babata K, Allred EN, Erdei C, Kuban KCK, Joseph RM, et al. Neurocognitive outcomes at 10 years of age in extremely preterm newborns with late-onset bacteremia. Journal of Pediatrics. 2017;187:43-49. e1.

Romanelli RMC, Anchieta LM, Mourão MVA, Campos FA, Loyola FC, Mourão PHO, et al. Fatores de risco e letalidade de infecção da corrente sanguínea laboratorialmente confirmada, causada por patógenos não contaminantes da pele em recém-nascidos. J pediatr (Rio J). 2013;89(2):189–96.

Burgaleta M, Johnson W, Waber DP, Colom R, Karama S. Cognitive ability changes and dynamics of cortical thickness development in healthy children and adolescents. Neuroimage. 2014;84:810–9.

Peltopuro M, Ahonen T, Kaartinen J, Seppälä H, Närhi V. Borderline intellectual functioning: a systematic literature review. Intellect Dev Disabil. 2014;52:419–43.

Conti-Ramsden G, Durkin K, Toseeb U, Botting N, Pickles A. Education and employment outcomes of young adults with a history of developmental language disorder. Int J Lang Commun Disord. 2018;53:237–55.

Czeizel AE, Dudas I;, Murphy MM, Fernandez-Ballart JD, Arija V. Bayley scales of infant development-administration manual. Paediatr Perinat Epidemiol. 2019.

Bayley N. Manual for the Bayley Scales of Infant Development (2nd ed.). San Antonio: TX: The Psychological Corporation; 1993.

Bos AF. Bayley-Ii Or Bayley-Iii: what do the scores tell us? Dev Med Child Neurol. 2013;55:978–9.

Johnson S, Marlow N. Developmental screen or developmental testing? Early Hum Dev. 2006;82(3):173–83.

Bakhuizen SE, De Haan TR, Teune MJ, Van Wassenaer-Leemhuis AG, Van Der Heyden JL, Van Der Ham DP, et al. Meta-analysis shows that infants who have suffered neonatal sepsis face an increased risk of mortality and severe complications. Acta Paediatr Int J Paediatr. 2014;103:1211–8.

Cai S, Thompson DK, Yang JYM, Anderson PJ. Short-and long-term neurodevelopmental outcomes of very preterm infants with neonatal sepsis: a systematic review and meta-analysis. Children. 2019;6:131.

Joussen AM, Poulaki V, Le ML, Koizumi K, Esser C, Janicki H, Schraermeyer U, Kociok N, Fauser S, Kirchhof B, Kern TS, Adamis AP. A central role for inflammation in the pathogenesis of diabetic retinopathy. FASEB J. 2004;18(12):1450–2.

Ushio-Fukai M. VEGF signaling through NADPH oxidase-derived ROS. In: Antioxidants and Redox Signaling. 2007.

Sharma A, Leaf JM, Thomas S, Cane C, Stuart C, Tremlett C, et al. Sensorineural hearing loss after neonatal meningitis: a single-centre retrospective study. BMJ Paediatr Open. 2022;6(1):e001601.

Smilga AS, Garfinkle J, Ng P, Andersen J, Buckley D, Fehlings D, et al. Neonatal infection in children with cerebral palsy: a registry-based cohort study. Pediatr Neurol. 2018;80:77–83.

Hayes R, Hartnett J, Semova G, Murray C, Murphy K, Carroll L, et al. Neonatal sepsis definitions from randomised clinical trials. Pediatr Res. 2023;93:1141–8.

Rahman AE, Iqbal A, Hoque DME, Moinuddin M, Zaman SB, Rahman QSU, et al. Managing neonatal and early childhood syndromic sepsis in sub-district hospitals in resource poor settings: Improvement in quality of care through introduction of a package of interventions in rural Bangladesh. PLoS One. 2017;12(1):e0170267.

Singh M, Alsaleem M GCP. StatPearls. Treasure Island (FL): StatPearls Publishing. 2022. Neonatal Sepsis. [Updated 2022 Sep 29]. Available from: https://www-ncbi-nlm-nih-gov.libproxy1.nus.edu.sg/books/NBK531478/ .

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. Diagnostic and Statistical Manual of Mental Disorders. 2022.

Book   Google Scholar  

McCauley SR, Wilde EA, Anderson VA, Bedell G, Beers SR, Campbell TF, et al. Recommendations for the Use of Common Outcome measures in pediatric traumatic brain injury research. J Neurotrauma. 2012;29:678–705.

Goh MSL, Looi DSH, Goh JL, Sultana R, Goh SSM, Lee JH, Chong SL. The Impact of Traumatic Brain Injury on Neurocognitive Outcomes in Children: a Systematic Review and Meta-Analysis. J Neurol Neurosurg Psychiatry. 2021:jnnp-2020-325066.

Download references

Acknowledgements

We would like to thank Ms. Wong Suei Nee, senior librarian from the National University of Singapore for helping us with the search strategy. We will also like to thank Dr Ming Ying Gan, Dr Shu Ting Tammie Seethor, Dr Jen Heng Pek, Dr Rachel Greenberg, Dr Christoph Hornik and Dr Bobby Tan, for their inputs in the initial design of this study.

Conflict of interest

No financial or non-financial benefits have been received or will be received from any party related directly or indirectly to the subject of this article.

Author information

Wei Jie Ong and Jun Jie Benjamin Seng are co-first authors.

Authors and Affiliations

MOH Holdings, Singapore, 1 Maritime Square, Singapore, 099253, Singapore

Wei Jie Ong, Jun Jie Benjamin Seng, Beijun Yap & Chen Lin Ng

SingHealth Regional Health System PULSES Centre, Singapore Health Services, Outram Rd, Singapore, 169608, Singapore

Jun Jie Benjamin Seng

SingHealth Duke-NUS Family Medicine Academic Clinical Programme, Singapore, Singapore

Yong Loo Lin School of Medicine, 10 Medical Dr, Yong Loo Lin School of Medicine, Singapore, Singapore

George He & Nooriyah Aliasgar Moochhala

Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore

Rehena Ganguly

Children’s Intensive Care Unit, KK Women’s and Children’s Hospital, SingHealth Paediatrics Academic Clinical Programme, 100 Bukit Timah Rd, Singapore, 229899, Singapore

Jan Hau Lee

Department of Emergency Medicine, KK Women’s and Children’s Hospital, SingHealth Paediatrics Academic Clinical Programme, SingHealth Emergency Medicine Academic Clinical Programme, 100 Bukit Timah Rd, Singapore, 229899, Singapore

Shu-Ling Chong

You can also search for this author in PubMed   Google Scholar

Contributions

SLC and JHL were the study’s principal investigators and were responsible for the conception and design of the study. WJO, JJBS, BY, GE, NAM and CLN were the co-investigators. WJO, JJBS, BY, GE, NAM and CLN were responsible for the screening and inclusion of articles and data extraction. All authors contributed to the data analyses and interpretation of data. WJO, JJBS, BY, GE, NAM and CLN prepared the initial draft of the manuscript. All authors revised the draft critically for important intellectual content and agreed to the final submission. All authors had access to all study data, revised the draft critically for important intellectual content and agreed to the final submission.

Corresponding author

Correspondence to Jun Jie Benjamin Seng .

Ethics declarations

Ethics approval and consent to participate.

As this was a systematic review with no access to patient data, ethical approval from the institutional review board was exempted.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Ong, W.J., Seng, J.J.B., Yap, B. et al. Impact of neonatal sepsis on neurocognitive outcomes: a systematic review and meta-analysis. BMC Pediatr 24 , 505 (2024). https://doi.org/10.1186/s12887-024-04977-8

Download citation

Received : 11 May 2024

Accepted : 26 July 2024

Published : 07 August 2024

DOI : https://doi.org/10.1186/s12887-024-04977-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Neonatal sepsis
• Infantile sepsis
• Neurocognitive outcomes
• Systematic review

BMC Pediatrics

ISSN: 1471-2431

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Science journal retracts peer-reviewed article containing AI generated ‘nonsensical’ images

• Share on Facebook
• Share on LinkedIn

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More

An open access scientific journal, Frontiers in Cell and Developmental Biology , was openly criticized and mocked by researchers on social media this week after they observed the publication had recently put up an article including imagery with gibberish descriptions and diagrams of anatomically incorrect mammalian testicles and sperm cells, which bore signs of being created by an AI image generator.

We thank the readers for their scrutiny of our articles: when we get it wrong, the crowdsourcing dynamic of open science means that community feedback helps us to quickly correct the record. — Frontiers (@FrontiersIn) February 15, 2024

The publication has since responded to one of its critics on the social network X, posting from its verified account: “We thank the readers for their scrutiny of our articles: when we get it wrong, the crowdsourcing dynamic of open science means that community feedback helps us to quickly correct the record.” It has also removed the article , entitled “ Cellular functions of spermatogonial stem cells in relation to JAK/STAT signaling pathway ” from its website and issued a retraction notice, stating:

“Following publication, concerns were raised regarding the nature of its AI-generated figures. The article does not meet the standards of editorial and scientific rigor for Frontiers in Cell and Development Biology; therefore, the article has been retracted.

This retraction was approved by the Chief Executive Editor of Frontiers. Frontiers would like to thank the concerned readers who contacted us regarding the published article. “

Misspelled words and anatomically incorrect illustrations

However, VentureBeat has obtained a copy and republished the original article below in the interest of maintaining the public record of it.

As you can observe in it, it contains several graphics and illustrations rendered in a seemingly clear and colorful scientific style, but zooming in, there are many misspelled words and misshapen letters, such as “protemns” instead of “proteins,” for example, and a word spelled “zxpens.”

Perhaps most problematic is the image of “rat” (spelled correctly) which appears first in the paper, and shows a massive growth in its groin region.

Blasted on X

Shortly after the paper’s publication on Feb. 13, 2024, researchers took to X to call it out and question how it made it through peer review.

Someone used DALL-E to create gobbledygook scientific figures and submitted them to Frontiers Journal. And guess what? The editor published it. LOL https://t.co/hjQkRQDkal https://t.co/aV1USo6Vt2 pic.twitter.com/VAkjJkY4dR — Veera Rajagopal  (@doctorveera) February 15, 2024

It’s finally happened. A peer-reviewed journal article with what appear to be nonsensical AI generated images. This is dangerous. pic.twitter.com/Ez54H6l7iZ — ?Kareem Carr | Statistician ? (@kareem_carr) February 15, 2024

The paper is authored by Xinyu Guo and Dingjun Hao of the Department of Spine Surgery, Hong Hui Hospital at Xi’an Jiaotong University; as well as Liang Dong of the Department of Spine Surgery, Xi’an Honghui Hospital in Xi’an, China.

It was reviewed by Binsila B. Krishnan of the National Institute of Animal Nutrition and Physiology (ICAR) in India and Jingbo Dai of Northwestern Medicine in the United States. It was edited by Arumugam Kumaresan at the National Dairy Research Institute (ICAR) in India.

VentureBeat reached out to all the authors and editors of the paper, as well as Amanda Gay Fisher, the journal’s Field Chief Editor, and a professor of biochemistry at the prestigious Oxford University in the U.K., to ask further questions about how the article was published, and will update when we hear back.

Troubling wider implications for AI’s impact on science, research, and medicine

AI has been touted as a valuable tool for advancing scientific research and discovery by some of its makers, including Google with its AlphaFold protein structure predictor and materials science AI GNoME, recently covered positively by the press (including VentureBeat) for discovering 2 million new materials .

However, those tools are focused on the research side. When it comes to publishing that research, it is clear that AI image generators could pose a major threat to scientific accuracy, especially if researchers are using them indiscriminately, to cut corners and publish faster, or because they are malicious or simply don’t care.

The move to use AI to create scientific illustrations or diagrams is troubling because it undermines the accuracy and trust among the scientific community and wider public that the work going into important fields that impact our lives and health — such as medicine and biology — is accurate, safe and screened.

Yet it may also be the product of the wider “publish or perish” climate that has arisen in science over the last several decades, in which researchers have attested they feel the need to rush out papers of little value to show they are contributing something, anything , to their field, and bolster the number of citations attributed to them by others, padding their resumes for future jobs.

But also, let’s be honest — some of these researchers on this paperwork in spine surgery at a human hospital: would you trust them to operate on your spine or help with your back health?

With more than 114,000 citations to its name, the journal Frontiers in Cell and Developmental Biology has now had its integrity called into question by this lapse: how many more papers published by it have AI-illustrated diagrams that have slipped through the review process?

Intriguingly, Frontiers in Cell and Developmental Biology is part of the wider Frontiers company of more than 230 different scientific publications founded in 2007 by neuroscientists Kamila Markram and Henry Markram , the former of whom is still listed as CEO.

The company says its “vision [is] to make science open, peer-review rigorous, transparent, and efficient and harness the power of technology to truly serve researchers’ needs,” and in fact, some of the tech it uses is AI for peer review.

As Frontiers proclaimed in a 2020 press release :

“ In an industry first, Artificial Intelligence (AI) is being deployed to help review research papers and assist in the peer-review process. The state-of-the-art Artificial Intelligence Review Assistant (AIRA), developed by open-access publisher Frontiers, helps editors, reviewers and authors evaluate the quality of manuscripts. AIRA reads each paper and can currently make up to 20 recommendations in just seconds, including the assessment of language quality, the integrity of the figures, the detection of plagiarism, as well as identifying potential conflicts of interest. “

The company’s website notes that AIRA debuted in 2018 as “ The next generation of peer review in which AI and machine learning enable more rigorous quality control and efficiency in the peer review .”

And just last summer, an article and video featuring Mirjam Eckert , chief publishing officer at Frontiers , stated:

“ At Frontiers, we apply AI to help build that trust. Our Artificial Intelligence Review Assistant (AIRA) verifies that scientific knowledge is accurately and honestly presented even before our people decide whether to review, endorse, or publish the research paper that contains it.

AIRA reads every research manuscript we receive and makes up to 20 checks a second. These checks cover, among other things, language quality, the integrity of figures and images, plagiarism, and conflicts of interest. The results give editors and reviewers another perspective as they decide whether to put a research paper through our rigorous and transparent peer review. “

Frontiers has also received favorably coverage of its AI article review assistant AIRA in such notable publications as The New York Times and Financial Times .

Clearly, the tool wasn’t able to effectively catch these nonsensical images in the article, leading to its retraction (if it was used at all in this case). But it also raises questions about the ability of such AI tools to detect, flag, and ultimately stop the publication of inaccurate scientific information — and the growing prevalence of its use at Frontiers and elsewhere across the publishing ecosystem. Perhaps that is the danger of being on the “frontier” of a new technology movement such as AI — the risk of it going wrong is higher than with the “tried and true,” human-only or analog approach.

VentureBeat also relies on AI tools for image generation and some text, but all articles are reviewed by human journalists prior to publication. AI was not used by VentureBeat in the writing, reporting, illustrating or publishing of this article.

Stay in the know! Get the latest news in your inbox daily

By subscribing, you agree to VentureBeat's Terms of Service.

Thanks for subscribing. Check out more VB newsletters here .

An error occured.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.43(12); 2018 Dec

The Extraordinary Importance of Sleep

New sleep deprivation studies confirm the relationship between inadequate sleep and a wide range of disorders, such as hypertension, obesity and type-2 diabetes, cardiovascular disease, impaired immune functioning, and more.

In the inaugural issue of the Journal of Clinical Sleep Medicine (2005), a feature article 1 traced early milestones in the developing field of sleep medicine, which slowly emerged from the older field of sleep research during the 1970s and 1980s. Sleep medicine, the article noted, was closely linked with and made possible by the discovery of electrical activity in the brain. The examination of electroencephalogram (EEG) patterns that occur during sleep led to the classification of stages of sleep, which in turn created an important foundation for probing human sleep, discerning abnormalities, and discovering significant relationships between sleep and health. By 2005, scientists and clinicians had not only identified and clearly defined a large number of sleep disorders but had discovered that many of them were highly prevalent.

The pace of research and discovery has only accelerated since 2005, and the number of peer-reviewed sleep journals has more than tripled. Today, researchers are more deeply probing the cellular and subcellular effects of disrupted sleep, as well as the effects of sleep deprivation on metabolism, hormone regulation, and gene expression. Newer studies are strengthening known and suspected relationships between inadequate sleep and a wide range of disorders, including hypertension, 2 obesity and type-2 diabetes, 3 impaired immune functioning, 4 cardiovascular disease and arrhythmias, 5 , 6 mood disorders, 7 neurodegeneration and dementia, 8 , 9 and even loneliness. 10

Research findings continue to underscore early concerns about public safety that were first raised when major industrial disasters such as the Exxon Valdez oil spill were linked to inadequate sleep. 11 Related research sponsored by major organizations, including the U.S. Department of Transportation, the U.S. Department of Defense, the National Institutes of Health, and the National Aeronautics and Space Administration (NASA), has helped to inspire national initiatives aimed at improving public safety and health. However, despite the astounding acceleration in research during the past few decades, inadequate sleep due to sleep disorders, work schedules, and chaotic lifestyles continues to threaten both health and safety.

“Pushing against the wave of accelerated growth in the field has been a shoreline of indifference,” says David F. Dinges, PhD, Professor and Chief of the Division of Sleep and Chronobiology in the Department of Psychiatry at the University of Pennsylvania Perelman School of Medicine. “Modern industrial pressures to use time 24 hours a day have led to shiftwork and a world in which virtually everything—law Susan L. Worley is a freelance medical writer who resides in Pennsylvania. enforcement, airports and all kinds of transportation, industrial operations, and hospitals—operates 24/7. People have come to value time so much that sleep is often regarded as an annoying interference, a wasteful state that you enter into when you do not have enough willpower to work harder and longer.”

David F. Dinges, PhD

It has become increasingly clear, however, that no matter how hectic our lives may be, we can no longer afford to ignore what research is telling us about the importance of sleep for our safety and mental and physical well-being.

Impact on Attention, Cognition, and Mood

While scientists are still working to identify and clarify all of the functions of sleep, 12 decades of studies—many of which have used the method of disrupting sleep and examining the consequences—have confirmed that sleep is necessary for our healthy functioning and even survival.

“We know for sure that sleep serves multiple functions,” says Dr. Dinges. “Nature tends to be very parsimonious in that it often uses a single system or biology in multiple ways to optimize the functioning of an organism. We know, for example, that sleep is critical for waking cognition—that is, for the ability to think clearly, to be vigilant and alert, and sustain attention. We also know that memories are consolidated during sleep, and that sleep serves a key role in emotional regulation.”

Studies conducted by Dr. Dinges and other scientists have shown that cognitive performance and vigilant attention begin to decline fairly quickly after more than 16 hours of continuous wakefulness, and that sleep deficits from partial sleep deprivation can accumulate over time, resulting in a steady deterioration in alertness. The widely used psychomotor vigilance test (PVT), a simple neurocognitive test developed by Dr. Dinges and colleagues that assesses an individual’s ability to sustain attention and respond to signals in a timely manner, has proven to be an exceptionally sensitive tool for capturing dose–response effects of sleep loss on neurobehavioral functioning. 13 The PVT also reliably detects sleep deficits caused by disrupted or fragmented sleep, and/or poorly timed sleep, which is important because a growing body of evidence suggests that the continuity and timing (or circadian alignment) of sleep may be as important as the total amount of time spent sleeping.

“We know that sleep is much more restorative of waking functions and health when it is consolidated and not fragmented,” explains Dr. Dinges. “That is, when sleep goes through the appropriate physiological sequences of non-REM (rapid eye movement) and REM states at night, and occurs when human sleep is temporally programmed by our circadian clock to occur. Such consolidated sleep is typically of a longer duration and better sleep quality than sleep taken at other times of the day, such as that which occurs with nightshift work, jet lag, and other conditions of circadian misalignment.”

Dr. Dinges and his colleagues have found that people whose daily sleep duration is inadequate, or repeatedly disrupted (e.g., by obstructive sleep apnea, restless legs syndrome, pain or stress, or shiftwork or jet lag), often are not aware of their accumulating sleep deficits or the toll that these deficits can take on their waking cognitive functions, including their performance, working memory, cognitive speed, and accuracy. Inadequate sleep also can take a toll on psychological well-being, significantly affecting our emotional and psychosocial interpretation of events and exacerbating our stress levels. Studies have indicated that changes in mood may be due in part to the effects of sleep deprivation on the processing of emotional memory—in other words, our tendency to select and remember negative memories after inadequate sleep. 14

In one study conducted by Dr. Dinges and colleagues, participants’ mood was observed after they were confronted with “high” and “low” performance demands, following varying degrees of sleep deprivation. 15

“To our surprise, those who were sleep-deprived responded to low stressors in much the same way that people without any sleep deprivation tended to respond to high stressors,” said Dr. Dinges. “In other words, we tend to become much more sensitive emotionally and socially when we are sleep-deprived. That is what I like to call the ‘who was at my desk or who touched my coffee cup?’ phenomenon. I think we all have experienced having an extreme reaction or a very negative emotional response to a mild stressor when we have not had enough sleep.”

Aiming for the Sweet Spot

How much sleep is enough? After decades of investigation, it appears that scientists have gathered enough evidence to begin to answer that question. 16

“When duration of sleep drops below seven hours, and especially when it starts to move toward six and half hours or less, a number of different disorders begin to increase in prevalence,” says Dr. Dinges. “Most experts would agree that there is a kind of sweet spot that most people should aim for, and for the average healthy adult that zone is ideally somewhere between 7 and 7 and a half hours. That is what the consensus evaluations of more than a thousand scientific articles have yielded—the consensus of evaluations conducted by the AASM (American Academy of Sleep Medicine) and Sleep Research Society jointly.”

Numerous large U.S. surveys—beginning with a 1982 survey by the American Cancer Society—have been used to estimate the number of hours that most people spend sleeping. Many surveys have identified a worrisome prevalence of “short” sleepers (people who sleep 6 hours or less) among respondents, and a general trend toward decreasing sleep duration between 1975 and 2006. More recently, however, an analysis of the American Time Use Survey (ATUS), spearheaded by Mathias Basner, MD, PhD, at the University of Pennsylvania 17 , has suggested that there may be cause for optimism.

“The analysis shows that there is a slight but steady increase in sleep time that stretches back to about 2003 or 2004,” says Dr. Dinges. “We think this increase, which is modest—at most a minute or two more per year—is due in part to the development of the field of sleep medicine, and public and scientific reports in the media about sleep loss contributing to accidents and catastrophes, and so forth. Ever so slowly, the message that it is important not to get sleep deprived, and to get help if you have a sleep disorder, has begun to penetrate to the public.”

The analysis notes that one sign of greater interest in sleep on the part of the public has been a significant increase in Google searches containing the word “sleep” since 2004. Data from the ATUS also suggest that over time, people have been willing to trade some of their daily activities in exchange for more sleep. It is important to note, says Dr. Dinges, that self-reports of time spent sleeping are not always accurate—they can be off by a half an hour or more, usually with people tending to estimate that they slept more than they did. He also notes that there is still a fairly large population sleeping 6 hours or less.

“Although there are signs that sleep time is increasing, it is not happening at nearly the dramatic rate that most experts would like to see,” says Dr. Dinges. “This is especially true for vulnerable populations. There is concern about school start times and bus times affecting the sleep of children and adolescents, and about extracurricular activities at the end of the school day sometimes leading to a delay in bed times for teenagers. All of this is an ongoing, evolving picture, with more research results coming out all the time, and with consequent changes in recommendations, to make sure that at least our most vulnerable populations are getting adequate sleep.”

Interindividual Differences in Vulnerability to Sleep Loss

While it is well established that the effects of sleep loss accumulate over time, with repeated exposure to inadequate, fragmented, or disrupted sleep, the degree to which individuals demonstrate adverse effects of inadequate sleep can vary considerably. 18

“We have learned that there are astonishingly mysterious phenotypes, or trait-like differences, in how vulnerable people are to sleep loss,” says Dr. Dinges. “This is still a relatively new area of research, and it has only been in the past few years that scientists have begun to replicate early findings regarding these phenotypic differences in vulnerability to the negative neurobehavioral effects of sleep loss. The interindividual differences that have been observed so far raise some extremely provocative scientific questions. We may find that there is something in waking biology that can substitute for, or somehow reduce, the impact of sleep loss on waking functioning, but thus far there is no evidence as to what that might be.”

Differences among individuals exist with regard to both the effects of sleep loss and the ability to recover from the effects of sleep loss. Differences in performance also have been shown to be task-dependent, suggesting that people who are vulnerable to the effects of sleep loss in one or more cognitive or neurobehavioral domains may be resistant to the effects of sleep loss in others. To better understand interindividual variability, scientists are investigating possible genetic mechanisms that may underlie complex interactions between circadian and sleep homeostatic systems—the systems that affect our drive for sleep as well as our alertness and performance during waking hours. A current goal is to discover biomarkers that may help predict individual performance after varying degrees of sleep loss. 19 And one hope is that biomarkers—ideally in the form of a simple “roadside” test such as a breathalyzer—may eventually be used to detect sleep loss-related impairment in drivers or in individuals responsible for operating sophisticated equipment or machinery. To date, no viable candidates have been found.

Investigators also are shedding light on the role that age may play in resilience to sleep loss. The results of one recent study indicate that younger adults are more vulnerable to the adverse effects of chronic sleep loss and recurring circadian disruption than older adults. 20 Although the neurobiological basis for these age-related differences is not yet understood, such findings may help to inform new approaches to the prevention of drowsy driving and related motor-vehicle accidents among young drivers.

Dr. Dinges emphasizes that findings regarding interindividual differences in response to sleep loss and in recovery from sleep loss should not diminish the message that adequate sleep is critical for everyone.

“Research has shown us that sleep is not an optional activity,” says Dr. Dinges. “There is no question that sleep is fundamentally conserved across species and across lifespans, and that any effort to eliminate it has been unsuccessful. We must plan our lives in the time domain with a serious consideration for sleep—planning when to sleep, ensuring that we get adequate sleep, and making sure that our sleep is not disturbed by disorders or diseases, whether or not they are sleep-related.”

Addressing Sleep Disorders

As connections between sleep disruption and both disease and mortality have become more firmly established, accurate and efficient diagnosis and management of sleep disorders (see Table 1 ) have become increasingly critical. Recent directions in the field of sleep medicine include a move toward patient-centered care, greater collaboration between specialists and primary care physicians, and the incorporation of new tools—including home-based diagnostic tests and novel electronic questionnaires—in the effort to create a comprehensive yet more personalized approach to assessment and treatment.

ICSD-3 Major Diagnostic Sections *

A chief goal is to improve the diagnosis of sleep disorders. Although approximately 70 million people in the U.S. have at least one sleep disorder, experts estimate that up to 80% of sleep disorders may go undetected or undiagnosed. One major challenge that clinicians face during the initial assessment of people with sleep disorders is the process of identifying and sorting out comorbidities. Untangling the causes and effects in bidirectional comorbidities can be particularly difficult. For example, insomnia—by far the most common sleep disorder—often is complicated by the presence of another sleep disorder, such as sleep apnea or restless legs syndrome.

“Some experts have even suggested that all cases of insomnia coexist with, or are caused by, another sleep disorder, most commonly sleep apnea,” says Clete A. Kushida, MD, PhD, Professor of Psychiatry and Behavioral Sciences at Stanford, and Division Chief and Medical Director of Stanford Sleep Medicine. “I’m not sure I would go quite that far, but certainly bidirectional comorbidities among individuals who experience sleep disorders are common. For example, pain syndromes—including back pain and limb pain, especially among older patients—are common comorbidities in patients with insomnia. Mood disorders also frequently occur in patients who experience insomnia.”

Comorbidities can complicate treatment and often require sleep specialists to collaborate with not only primary care physicians but also specialists in other therapeutic areas.

“If, for example, a person with insomnia also has been diagnosed with depression by a psychiatrist,” says Dr. Kushida, “our goal is to work hand in hand with the psychiatrist to find the right medication. There are both sedating and alerting antidepressants, and a patient may need to try one medication for a couple of weeks to months, slowly increasing the dose to a therapeutic level, until the effect on both the depression and the patient’s sleep can be determined. For some individuals, an alerting antidepressant can cause poor sleep, which in turn can exacerbate the depression. The process of achieving the right dose of the right medication can be complex, and benefits from a collaboration between specialists.”

Undetected obstructive sleep apnea (OSA) in patients with chronic pain, or other serious illnesses, can result in potentially dangerous comorbidities. Opioids, for example, are known to have adverse effects on respiration, and can lead to central sleep apnea (CSA)—shallow and irregular or interrupted breathing and sustained hypoventilation—a potentially lethal condition that can intensify the consequences of OSA. These risks underscore the need to improve methods for identifying and properly diagnosing the estimated 23.5 million U.S. adults with OSA. Public education and advocacy efforts are already helping to improve detection—in part by helping to address misconceptions about OSA.

“One of the biggest misconceptions is that only people who are significantly overweight experience sleep apnea,” says Dr. Kushida. “In fact, only up to 67% of people who have OSA are overweight, the rest are of normal weight. OSA also can be caused by craniofacial dysmorphism, or a defect of the airway that occurs during development. A narrow airway caused by deficient growth of the craniofacial skeleton, particularly the jaws, can become narrower and more prone to collapse with age, leading to sleep apnea.”

Treating Insomnia: The Value of Cognitive Behavioral Therapy

Insomnia, the most prevalent sleep disorder, affects approximately one third of all adults and is the most common condition that family and primary-care physicians encounter. According to the International Classification of Sleep Disorders (ICSD-3), chronic insomnia is the inability to attain sufficient sleep (despite adequate opportunity) for at least three nights per week for three months or longer, with negative daytime consequences. For most people, the disorder is transient, but for approximately 10% to 15% of those who experience insomnia (around 30 million people) it becomes chronic. Although pharmacologic treatments for insomnia ( Table 2 ) can be effective, most experts now recommend against the long-term use of pharmacotherapy.

Selected Pharmaceutical Treatments for Insomnia 21 , 27

“If a person has been diagnosed with chronic insomnia, the only treatment that has been shown to have long-term benefit is cognitive behavioral therapy, “says Dr. Kushida. “Medications really should be considered short-term treatments, because patients tend to develop dependence on, or tolerance to, hypnotic drugs. In our clinic, we commonly see that, over time, medications stop having an effect, and that means that patients may try higher doses of a medication, or keep switching to different medications. So, medications are a temporary solution—they just put a Band-Aid on the problem of insomnia, whereas cognitive behavioral therapy targets one of the pathways toward success.”

Cognitive behavioral therapy (CBT), which involves techniques that work in part by reducing cognitive and somatic arousal, is estimated to be effective in approximately 70% to 80% of people who experience chronic insomnia. Dr. Kushida notes that while drugs can sometimes be useful in the treatment of acute insomnia, they become problematic after acute insomnia transitions to chronic insomnia.

“A person might be an OK sleeper for several years, and then suddenly experience a traumatic event, such as the loss of a job, a divorce, or the death of a loved one, resulting in very poor sleep,” says Dr. Kushida. “Down the road, that person might obtain a better job, overcome grief, or find a new relationship, but continue to experience insomnia. We think in some cases the transition from acute insomnia to chronic insomnia occurs because the behavioral event triggers something in the person’s physiology that may lead to long-term changes. Once they are in a chronic insomnia phase, we tell patients that CBT is the only truly effective intervention.”

If a patient is already taking hypnotics, Dr. Kushida says that he will gradually wean the patient off medications while introducing CBT. He notes that often it is necessary for sleep specialists to manage the expectations of chronic sufferers.

Clete A. Kushida, MD, PhD

“We sometimes have to let patients with chronic insomnia know that we may never get them back to where they were when they had optimal sleep,” Dr. Kushida explains. “The behavioral methods we use work well, and usually we can get patients to the point where the insomnia is having less of an impact on their quality of life. Our inability to completely restore the patient’s ability to sleep well may partly be explained by as yet unidentified changes in his or her neurophysiology or neurochemistry. Some patients with chronic insomnia can begin to sleep normally again, but for the vast majority, we aim to make insomnia less of a burden on a patient’s daily life.”

Improving Clinical Research

In the field of sleep medicine, as in many other therapeutic areas, future directions in clinical trial research will place an emphasis on patient engagement and patient-centered outcomes.

“Perhaps the most important aim these days when developing and implementing any type of large-scale clinical research study is to incorporate the patient’s perspective,” says Dr. Kushida, who is currently analyzing the results of a comparative effectiveness sleep study sponsored by the Patient-Centered Outcomes Research Institute (PCORI). 22 The study, designed and conducted by a team at Stanford, introduced a new model of patient-centered, coordinated care and tested it against conventional outpatient treatment for sleep disorders.

“The patient’s perspective is so invaluable in guiding the success of a study that ideally it should be incorporated right at the inception of a research question or idea,” says Dr. Kushida. “When you are designing an especially complicated trial, for example, it is easy to incorporate a lot of tests and measures without being aware of the burden these can place on the participants. It’s critical to learn from patients whether they are overwhelmed by the number of tests, or whether travel time or the amount of time they need to take off from work may be impractical.”

Other efforts to improve clinical research include those focused on correcting for and/or eliminating several confounding variables that tend to plague sleep research. The surprising power of the placebo effect, 23 the related disconnect between objective and subjective evaluations of sleep loss and recovery from sleep loss, variable adherence to treatments, and, more recently, deceptive practices among clinical trial participants, are a few examples.

The placebo effect, which refers to any outcome that may be attributable to the expectations of clinical trial participants rather than to the drug or device being tested, can be especially problematic in experimental protocols that involve self-reports of sleep quality.

“Clinical trials involving patients with disorders such as insomnia or RLS that rely solely on subjective measures, or ratings of severity based on patient report, are particularly vulnerable to the placebo effect,” says Dr. Kushida. “It has been demonstrated that when these patients believe that they are receiving the study drug or device the likelihood of their experiencing a positive effect can increase significantly. There have been efforts to develop or introduce new objective endpoints in these studies, which may help with this problem.”

Achieving the right balance of subjective and objective measures of sleep is an important goal in both research and clinical practice. The current gold standard for objective assessment of sleep is polysomnography (PSG), which includes electrophysiological recordings of brain activity (EEG), muscle activity (EMG), and eye movements (EOG). A valuable, non-invasive method for determining sleep continuity and sleep architecture, PSG has been an indispensable objective endpoint in clinical trials, but it is expensive and not always practical. Novel approaches to objective measurement, including actigraphy, which may be used to help minimize recall bias and complement subjective measures of sleep (e.g., sleep logs or diaries), still have drawbacks. 24

“The problem with wearable devices right now,” says Dr. Kushida, “is that they tend to overestimate sleep, sometimes by as much as an hour. They also are not yet capable of accurately detecting different stages of sleep, such as non-REM and REM sleep. Because of our proximity to Silicon Valley, our laboratory tests a lot of these new devices, and often by the time we have finished testing one prototype, new ones have emerged. The product cycles are rapid, and the companies keep incorporating newer and newer technology. So, down the road, within about five to ten years, I think these devices will likely estimate sleep and detect sleep stages with precision.”

Also, objective tools are needed for addressing problems with adherence to treatment. One important current aim is to detect and correct for non-obvious factors that result in failure to adhere to treatment, whether unintended or deliberate, to ensure that trial outcomes accurately reflect the efficacy of a drug, medical device, or behavioral intervention. 25 A related problem is deliberate deception by trial participants. As part of a National Heart, Lung, and Blood Institute (NHLBI)-supported study focused on detecting and correcting for adherence problems, Dr. Kushida and colleagues began to explore the prevalence of deceptive practices among clinical trial participants. 26

“We found that deception among clinical trial participants is pretty common and that there is quite a range of deceptive practices, “says Dr. Kushida. “They include underreported drug holidays, fabrication or withholding of medical histories, pill dumping, exaggerated symptoms, and falsification of current health status. It’s important that we find a way to address these deceptive practices because both the integrity of research data and the safety of participants are at risk.”

Dr. Kushida adds that newer tools, such as electronic monitoring of pill dispensing and statistical predictive adherence models, may uncover and remedy pressing problems related to adherence and deceptive practices. “It already takes about 12 years for a new drug to be approved, and about three to five years for a new device to be approved. When deceptive practices are discovered too late, it can lead to the invalidation of research findings and further delays in approving much-needed treatments.”

Enhancing clinical research in the field will require a cooperative, international effort focused on advancing knowledge about sleep, circadian rhythms, and sleep disorders worldwide. During Dr. Kushida’s tenure as inaugural president of the World Sleep Society (WSS), he led an initiative to create international sleep fellowships to prepare physicians and scientists from various countries for future leadership roles in basic and/or clinical sleep research. He also oversaw the development of an International Sleep Research Network, designed to help sleep scientists and clinicians find collaborators with similar clinical/research interests. As the WSS continues to offer new services and expand its programs, it will be with an awareness of the needs of disadvantaged populations and the importance of access to appropriate treatment.

“One initiative of the WSS involves reviewing current published guidelines in various countries, to determine whether they meet international standards,” says Dr. Kushida. “Many guidelines are region-specific and list only medications approved in specific countries or regions. As we review the guidelines, we endorse them with caveats; we may note that particular treatments for insomnia are recommended, and when these are not available we recommend acceptable substitutes. The goal is to ensure that specialists can use practice guidelines in whichever country they practice sleep medicine, and that patients are receiving the best possible treatment available.”

Maintenance work is planned from 21:00 BST on Sunday 18th August 2024 to 21:00 BST on Monday 19th August 2024, and on Thursday 29th August 2024 from 11:00 to 12:00 BST.

During this time the performance of our website may be affected - searches may run slowly, some pages may be temporarily unavailable, and you may be unable to log in or to access content. If this happens, please try refreshing your web browser or try waiting two to three minutes before trying again.

We apologise for any inconvenience this might cause and thank you for your patience.

Journal of Materials Chemistry C

Hybridized local and charge-transfer excited states of blue oleds based on phenanthroimidazole derivates with a narrow fwhm of 24 nm †.

* Corresponding authors

a Shaanxi Key Laboratory of Photoelectric Functional Materials and Devices, School of Materials and Chemical Engineering, Xi’an Technological University, Xi’an, China E-mail: [email protected]

b Shaanxi Lighte Optoelectronics Material Co., Ltd., Xi’an, China E-mail: [email protected]

Designing OLEDs with high efficiency deep-blue emission has always been a research topic that researchers are committed to pursuing. It is a long-standing challenge to realize HLCT materials with a Commission Internationale de L’éclairage (CIE y ) < 0.08 and narrow width at half maximum (FWHM). Here, six phenanthroimidazole derivatives ( PT series) incorporating 3 a ,11 b -dihydro-1 H -phenanthroimidazole (PI) as the acceptor and triphenylamine (TPA) as the donor have been designed. The impact of introducing various functional groups in the minor axis of the entire molecule is investigated. The molecules were successfully synthesized and systematically investigated. Me-PT and TfMe-PT demonstrate exceptional photoluminescence quantum yields (PLQYs) of 80.10% and 87.64% and a narrow FWHM in neat films, respectively. Therefore, doped devices based on Me-PT and TfMe-PT were fabricated, with maximum external quantum efficiency (EQE max ) values of 5.52% and 5.47%, respectively. Both devices exhibit blue emission centered around a peak wavelength ( λ PL ) of 465 nm and 466 nm, accompanied by a narrow FWHM of 24 nm and 25 nm, respectively. These findings underscore a streamlined and effective approach in HLCT materials, providing significant guidance for the development of high purity blue emission OLEDs.

Supplementary files

• Supplementary information ZIP (157570K)

Transparent peer review

To support increased transparency, we offer authors the option to publish the peer review history alongside their article.

View this article’s peer review history

Article information

Download citation, permissions.

Hybridized local and charge-transfer excited states of blue OLEDs based on phenanthroimidazole derivates with a narrow FWHM of 24 nm

X. Song, S. Zhu, Y. Liu, T. Shi, L. Yang, Y. He, X. Niu, Z. Yang, J. Yuan and Z. Feng, J. Mater. Chem. C , 2024, Advance Article , DOI: 10.1039/D4TC02261K

To request permission to reproduce material from this article, please go to the Copyright Clearance Center request page .

If you are an author contributing to an RSC publication, you do not need to request permission provided correct acknowledgement is given.

If you are the author of this article, you do not need to request permission to reproduce figures and diagrams provided correct acknowledgement is given. If you want to reproduce the whole article in a third-party publication (excluding your thesis/dissertation for which permission is not required) please go to the Copyright Clearance Center request page .

Read more about how to correctly acknowledge RSC content .

Social activity

Search articles by author.

This article has not yet been cited.

Advertisements

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Machine learning methods in weather and climate applications: a survey.

1. Introduction

• Limited Scope: Existing surveys predominantly focus either on short-term weather forecasting or medium-to-long-term climate predictions. There is a notable absence of comprehensive surveys that endeavour to bridge these two-time scales. In addition, current investigations tend to focus narrowly on specific methods, such as simple neural networks, thereby neglecting some combination of methods.
• Lack of model details: Many extisting studies offer only generalized viewpoints and lack a systematic analysis of the specific model employed in weather and climate prediction. This absence creates a barrier for researchers aiming to understand the intricacies and efficacy of individual methods.
• Neglect of Recent Advances: Despite rapid developments in machine learning and computational techniques, existing surveys have not kept pace with these advancements. The paucity of information on cutting-edge technologies stymies the progression of research in this interdisciplinary field.
• Comprehensive scope: Unlike research endeavors that restrict their inquiry to a singular temporal scale, our survey provides a comprehensive analysis that amalgamates short-term weather forecasting with medium- and long-term climate predictions. In total, 20 models were surveyed, of which a select subset of eight were chosen for in-depth scrutiny. These models are discerned as the industry’s avant-garde, thereby serving as invaluable references for researchers. For instance, the PanGu model exhibits remarkable congruence with actual observational results, thereby illustrating the caliber of the models included in our analysis.
• In-Depth Analysis: Breaking new ground, this study delves into the intricate operational mechanisms of the eight focal models. We have dissected the operating mechanisms of these eight models, distinguishing the differences in their approaches and summarizing the commonalities in their methods through comparison. This comparison helps readers gain a deeper understanding of the efficacy and applicability of each model and provides a reference for choosing the most appropriate model for a given scenario.
• Identification of Contemporary Challenges and Future Work: The survey identifies pressing challenges currently facing the field, such as the limited dataset of chronological seasons and complex climate change effects, and suggests directions for future work, including simulating datasets and physics-based constraint models. These recommendations not only add a forward-looking dimension to our research but also act as a catalyst for further research and development in climate prediction.

2. Background

3. related work, 3.1. statistical method, 3.2. physical models, 4. taxonomy of climate prediction applications, 4.1. climate prediction milestone based on machine-learning, 4.2. classification of climate prediction methods, 5. short-term weather forecast, 5.1. model design.

• The Navier-Stokes Equations [ 73 ]: Serving as the quintessential descriptors of fluid motion, these equations delineate the fundamental mechanics underlying atmospheric flow. ∇ · v = 0 (3) ρ ∂ v ∂ t + v · ∇ v = − ∇ p + μ ∇ 2 v + ρ g (4)
• The Thermodynamic Equations [ 74 ]: These equations intricately interrelate the temperature, pressure, and humidity within the atmospheric matrix, offering insights into the state and transitions of atmospheric energy. ∂ ρ ∂ t + ∇ · ( ρ v ) = 0 ( Continuity equation ) (5) ∂ T ∂ t + v · ∇ T = q c p ( Energy equation ) (6) D p D t = − ρ c p ∇ · v ( Pressure equation ) (7)
• The Cloud Microphysics Parameterization Scheme is instrumental for simulating the life cycles of cloud droplets and ice crystals, thereby affecting [ 75 , 76 ] and atmospheric energy balance.
• Shortwave and Longwave Radiation Transfer Equations elucidate the absorption, scattering, and emission of both solar and terrestrial radiation, which in turn influence atmospheric temperature and dynamics.
• Empirical or Semi-Empirical Convection Parameterization Schemes simulate vertical atmospheric motions initiated by local instabilities, facilitating the capture of weather phenomena like thunderstorms.
• Boundary-Layer Dynamics concentrates on the exchanges of momentum, energy, and matter between the Earth’s surface and the atmosphere which are crucial for the accurate representation of surface conditions in the model.
• Land Surface and Soil/Ocean Interaction Modules simulate the exchange of energy, moisture, and momentum between the surface and the atmosphere, while also accounting for terrestrial and aquatic influences on atmospheric conditions.
• Encoder: The encoder component maps the local region of the input data (on the original latitude-longitude grid) onto the nodes of the multigrid graphical representation. It maps two consecutive input frames of the latitude-longitude input grid, with numerous variables per grid point, into a multi-scale internal mesh representation. This mapping process helps the model better capture and understand spatial dependencies in the data, allowing for more accurate predictions of future weather conditions.
• Processor: This part performs several rounds of message-passing on the multi-mesh, where the edges can span short or long ranges, facilitating efficient communication without necessitating an explicit hierarchy. More specifically, the section uses a multi-mesh graph representation. It refers to a special graph structure that is able to represent the spatial structure of the Earth’s surface in an efficient way. In a multi-mesh graph representation, nodes may represent specific regions of the Earth’s surface, while edges may represent spatial relationships between these regions. In this way, models can capture spatial dependencies on a global scale and are able to utilize the power of GNNs to analyze and predict weather changes.
• Decoder: It then maps the multi-mesh representation back to the latitude-longitude grid as a prediction for the next time step.

5.2. Result Analysis

6. medium-to-long-term climate prediction, 6.1. model design.

• Problem Definition: The goal is to approximate p ( Y ∣ X , M ) , a task challenged by high-dimensional geospatial data, data inhomogeneity, and a large dataset.
• Random Variable z : A latent variable with a fixed standard Gaussian distribution.
• Parametric Functions p θ , q ϕ , p ψ : Neural networks for transforming z and approximating target and posterior distributions.
• Objective Function: Maximization of the Evidence Lower Bound (ELBO).
• Initialize: Define random variable z ∼ N ( 0 , 1 ) [ 96 , 97 ] parametric functions p θ ( z , X , M ) , q ϕ ( z ∣ X , Y , M ) , p ψ ( Y ∣ X , M , z ) .
• Training Objective (Maximize ELBO) [ 98 ]: The ELBO is defined as: ELBO = E z ∼ q ϕ log p ψ ( Y ∣ X , M , z ) − D KL ( q ϕ ∥ p ( z ∣ X , M ) ) − D KL ( q ϕ ∥ p ( z ∣ X , Y , M ) ) (8) with terms for reconstruction, regularization, and residual error.
• Optimization: Utilize variational inference, Monte Carlo reparameterization, and Gaussian assumptions.
• Forecasting: Generate forecasts by sampling p ( z ∣ X , M ) , the likelihood of p ψ , and using the mean of p ψ for an average estimate.
• Two Generators : The CycleGAN model includes two generators. Generator G learns the mapping from the simulated domain to the real domain, and generator F learns the mapping from the real domain to the simulated domain [ 100 ].
• Two Discriminators : There are two discriminators, one for the real domain and one for the simulated domain. Discriminator D x encourages generator G to generate samples that look similar to samples in the real domain, and discriminator D y encourages generator F to generate samples that look similar to samples in the simulated domain.
• Cycle Consistency Loss : To ensure that the mappings are consistent, the model enforces the following condition through a cycle consistency loss: if a sample is mapped from the simulated domain to the real domain and then mapped back to the simulated domain, it should get a sample similar to the original simulated sample. Similarly, if a sample is mapped from the real domain to the simulated domain and then mapped back to the real domain, it should get a sample similar to the original real sample. L cyc ( G , F ) = E x ∼ p data ( x ) | | F ( G ( x ) ) − x | | 1 + E y ∼ p data ( y ) | | G ( F ( y ) ) − y | | 1 (10)
• Training Process : The model is trained to learn the mapping between these two domains by minimizing the adversarial loss and cycle consistency loss between the generators and discriminators. L Gen ( G , F ) = L GAN ( G , D y , X , Y ) + L GAN ( F , D x , Y , X ) + λ L cyc ( G , F ) (11)
• Application to Prediction : Once trained, these mappings can be used for various tasks, such as transforming simulated precipitation data into forecasts that resemble observed data.
• Reference Model: SPCAM. SPCAM serves as the foundational GCM and is embedded with Cloud-Resolving Models (CRMs) to simulate microscale atmospheric processes like cloud formation and convection. SPCAM is employed to generate “target simulation data”, which serves as the training baseline for the neural networks. The use of CRMs is inspired by recent advancements in data science, demonstrating that machine learning parameterizations can potentially outperform traditional methods in simulating convective and cloud processes.
• Neural Networks: ResDNNs, a specialized form of deep neural networks, are employed for their ability to approximate complex, nonlinear relationships. The network comprises multiple residual blocks, each containing two fully connected layers with Rectified Linear Unit (ReLU) activations. ResDNNs are designed to address the vanishing and exploding gradient problems in deep networks through residual connections, offering a stable and effective gradient propagation mechanism. This makes them well-suited for capturing the complex and nonlinear nature of atmospheric processes.
• Subgrid-Scale Physical Simulator. Traditional parameterizations often employ simplified equations to model subgrid-scale processes, which might lack accuracy. In contrast, the ResDNNs are organized into a subgrid-scale physical simulator that operates independently within each model grid cell. This simulator takes atmospheric states as inputs and outputs physical quantities at the subgrid scale, such as cloud fraction and precipitation rate.

6.2. Result Analysis

7. discussion, 7.1. overall comparison, 7.2. challenge, 7.3. future work.

• Simulate the dataset using statistical methods or physical methods.
• Combining statistical knowledge with machine learning methods to enhance the interpretability of patterns.
• Consider the introduction of physics-based constraints into deep learning models to produced more accurate and reliable results.
• Accelerating Physical Model Prediction with machine learning knowledge.

8. Conclusions

Author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest, abbreviations.

• Abbe, C. The physical basis of long-range weather. Mon. Weather Rev. 1901 , 29 , 551–561. [ Google Scholar ] [ CrossRef ]
• Zheng, Y.; Capra, L.; Wolfson, O.; Yang, H. Urban computing: Concepts, methodologies, and applications. Acm Trans. Intell. Syst. Technol. TIST 2014 , 5 , 1–55. [ Google Scholar ]
• Gneiting, T.; Raftery, A.E. Weather forecasting with ensemble methods. Science 2005 , 310 , 248–249. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Agapiou, A. Remote sensing heritage in a petabyte-scale: Satellite data and heritage Earth Engine applications. Int. J. Digit. Earth 2017 , 10 , 85–102. [ Google Scholar ] [ CrossRef ]
• Bendre, M.R.; Thool, R.C.; Thool, V.R. Big data in precision agriculture: Weather forecasting for future farming. In Proceedings of the 2015 1st International Conference on Next Generation Computing Technologies (NGCT), Dehradun, India, 4–5 September 2015; pp. 744–750. [ Google Scholar ]
• Zavala, V.M.; Constantinescu, E.M.; Krause, T. On-line economic optimization of energy systems using weather forecast information. J. Process Control 2009 , 19 , 1725–1736. [ Google Scholar ] [ CrossRef ]
• Nurmi, V.; Perrels, A.; Nurmi, P.; Michaelides, S.; Athanasatos, S.; Papadakis, M. Economic value of weather forecasts on transportation–Impacts of weather forecast quality developments to the economic effects of severe weather. EWENT FP7 Project . 2012, Volume 490. Available online: http://virtual.vtt.fi/virtual/ewent/Deliverables/D5/D5_2_16_02_2012_revised_final.pdf (accessed on 8 September 2023).
• Russo, J.A., Jr. The economic impact of weather on the construction industry of the United States. Bull. Am. Meteorol. Soc. 1966 , 47 , 967–972. [ Google Scholar ] [ CrossRef ]
• Badorf, F.; Hoberg, K. The impact of daily weather on retail sales: An empirical study in brick-and-mortar stores. J. Retail. Consum. Serv. 2020 , 52 , 101921. [ Google Scholar ] [ CrossRef ]
• De Freitas, C.R. Tourism climatology: Evaluating environmental information for decision making and business planning in the recreation and tourism sector. Int. J. Biometeorol. 2003 , 48 , 45–54. [ Google Scholar ] [ CrossRef ]
• Smith, K. Environmental Hazards: Assessing Risk and Reducing Disaster ; Routledge: London, UK, 2013. [ Google Scholar ]
• Hammer, G.L.; Hansen, J.W.; Phillips, J.G.; Mjelde, J.W.; Hill, H.; Love, A.; Potgieter, A. Advances in application of climate prediction in agriculture. Agric. Syst. 2001 , 70 , 515–553. [ Google Scholar ] [ CrossRef ]
• Guedes, G.; Raad, R.; Raad, L. Welfare consequences of persistent climate prediction errors on insurance markets against natural hazards. Estud. Econ. Sao Paulo 2019 , 49 , 235–264. [ Google Scholar ] [ CrossRef ]
• McNamara, D.E.; Keeler, A. A coupled physical and economic model of the response of coastal real estate to climate risk. Nat. Clim. Chang. 2013 , 3 , 559–562. [ Google Scholar ] [ CrossRef ]
• Kleerekoper, L.; Esch, M.V.; Salcedo, T.B. How to make a city climate-proof, addressing the urban heat island effect. Resour. Conserv. Recycl. 2012 , 64 , 30–38. [ Google Scholar ] [ CrossRef ]
• Kaján, E.; Saarinen, J. Tourism, climate change and adaptation: A review. Curr. Issues Tour. 2013 , 16 , 167–195. [ Google Scholar ]
• Dessai, S.; Hulme, M.; Lempert, R.; Pielke, R., Jr. Climate prediction: A limit to adaptation. Adapt. Clim. Chang. Threshold. Values Gov. 2009 , 64 , 78. [ Google Scholar ]
• Ham, Y.-G.; Kim, J.-H.; Luo, J.-J. Deep Learning for Multi-Year ENSO Forecasts. Nature 2019 , 573 , 568–572. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Howe, L.; Wain, A. Predicting the Future ; Cambridge University Press: Cambridge, UK, 1993; Volume V, pp. 1–195. [ Google Scholar ]
• Hantson, S.; Arneth, A.; Harrison, S.P.; Kelley, D.I.; Prentice, I.C.; Rabin, S.S.; Archibald, S.; Mouillot, F.; Arnold, S.R.; Artaxo, P.; et al. The status and challenge of global fire modelling. Biogeosciences 2016 , 13 , 3359–3375. [ Google Scholar ]
• Racah, E.; Beckham, C.; Maharaj, T.; Ebrahimi Kahou, S.; Prabhat, M.; Pal, C. ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events. Adv. Neural Inf. Process. Syst. 2017 , 30 , 3402–3413. [ Google Scholar ]
• Gao, S.; Zhao, P.; Pan, B.; Li, Y.; Zhou, M.; Xu, J.; Zhong, S.; Shi, Z. A nowcasting model for the prediction of typhoon tracks based on a long short term memory neural network. Acta Oceanol. Sin. 2018 , 37 , 8–12. [ Google Scholar ]
• Ren, X.; Li, X.; Ren, K.; Song, J.; Xu, Z.; Deng, K.; Wang, X. Deep Learning-Based Weather Prediction: A Survey. Big Data Res. 2021 , 23 , 100178. [ Google Scholar ]
• Reichstein, M.; Camps-Valls, G.; Stevens, B.; Jung, M.; Denzler, J.; Carvalhais, N.; Prabhat, F. Deep learning and process understanding for data-driven Earth system science. Nature 2019 , 566 , 195–204. [ Google Scholar ] [ CrossRef ]
• Stockhause, M.; Lautenschlager, M. CMIP6 data citation of evolving data. Data Sci. J. 2017 , 16 , 30. [ Google Scholar ] [ CrossRef ]
• Hsieh, W.W. Machine Learning Methods in the Environmental Sciences: Neural Networks and Kernels ; Cambridge University Press: Cambridge, UK, 2009. [ Google Scholar ]
• Krasnopolsky, V.M.; Fox-Rabinovitz, M.S.; Chalikov, D.V. New Approach to Calculation of Atmospheric Model Physics: Accurate and Fast Neural Network Emulation of Longwave Radiation in a Climate Model. Mon. Weather Rev. 2005 , 133 , 1370–1383. [ Google Scholar ] [ CrossRef ]
• Krasnopolsky, V.M.; Fox-Rabinovitz, M.S.; Belochitski, A.A. Using ensemble of neural networks to learn stochastic convection parameterizations for climate and numerical weather prediction models from data simulated by a cloud resolving model. Adv. Artif. Neural Syst. 2013 , 2013 , 485913. [ Google Scholar ] [ CrossRef ]
• Chevallier, F.; Morcrette, J.-J.; Chéruy, F.; Scott, N.A. Use of a neural-network-based long-wave radiative-transfer scheme in the ECMWF atmospheric model. Q. J. R. Meteorol. Soc. 2000 , 126 , 761–776. [ Google Scholar ]
• Krasnopolsky, V.M.; Fox-Rabinovitz, M.S.; Hou, Y.T.; Lord, S.J.; Belochitski, A.A. Accurate and fast neural network emulations of model radiation for the NCEP coupled climate forecast system: Climate simulations and seasonal predictions. Mon. Weather Rev. 2010 , 138 , 1822–1842. [ Google Scholar ] [ CrossRef ]
• Tolman, H.L.; Krasnopolsky, V.M.; Chalikov, D.V. Neural network approximations for nonlinear interactions in wind wave spectra: Direct mapping for wind seas in deep water. Ocean. Model. 2005 , 8 , 253–278. [ Google Scholar ] [ CrossRef ]
• Markakis, E.; Papadopoulos, A.; Perakakis, P. Spatiotemporal Forecasting: A Survey. arXiv 2018 , arXiv:1808.06571. [ Google Scholar ]
• Box, G.E.; Jenkins, G.M.; Reinsel, G.C.; Ljung, G.M. Time Series Analysis: Forecasting and Control ; John Wiley & Sons: Hoboken, NJ, USA, 2015. [ Google Scholar ]
• He, Y.; Kolovos, A. Spatial and Spatio-Temporal Geostatistical Modeling and Kriging. In Wiley StatsRef: Statistics Reference Online ; John Wiley & Sons: Hoboken, NJ, USA, 2015. [ Google Scholar ]
• Lu, H.; Fan, Z.; Zhu, H. Spatiotemporal Analysis of Air Quality and Its Application in LASG/IAP Climate System Model. Atmos. Ocean. Sci. Lett. 2011 , 4 , 204–210. [ Google Scholar ]
• Chatfield, C. The Analysis of Time Series: An Introduction , 7th ed.; CRC Press: Boca Raton, FL, USA, 2016. [ Google Scholar ]
• Stull, R. Meteorology for Scientists and Engineers , 3rd ed.; Brooks/Cole: Pacific Grove, CA, USA, 2015. [ Google Scholar ]
• Yuval, J.; O’Gorman, P.A. Machine Learning for Parameterization of Moist Convection in the Community Atmosphere Model. Proc. Natl. Acad. Sci. USA 2020 , 117 , 12–20. [ Google Scholar ]
• Gagne, D.J.; Haupt, S.E.; Nychka, D.W. Machine Learning for Spatial Environmental Data. Meteorol. Monogr. 2020 , 59 , 9.1–9.36. [ Google Scholar ]
• Xu, Z.; Li, Y.; Guo, Q.; Shi, X.; Zhu, Y. A Multi-Model Deep Learning Ensemble Method for Rainfall Prediction. J. Hydrol. 2020 , 584 , 124579. [ Google Scholar ]
• Kuligowski, R.J.; Barros, A.P. Localized precipitation forecasts from a numerical weather prediction model using artificial neural networks. Weather. Forecast. 1998 , 13 , 1194–1204. [ Google Scholar ] [ CrossRef ]
• Shi, X.; Chen, Z.; Wang, H.; Yeung, D.Y.; Wong, W.K.; Woo, W.C. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. arXiv 2015 , arXiv:1506.04214. [ Google Scholar ]
• Qiu, M.; Zhao, P.; Zhang, K.; Huang, J.; Shi, X.; Wang, X.; Chu, W. A short-term rainfall prediction model using multi-task convolutional neural networks. In Proceedings of the 2017 IEEE International Conference on Data Mining (ICDM), New Orleans, LA, USA, 18–21 November 2017; IEEE: New York, NY, USA, 2017; pp. 395–404. [ Google Scholar ]
• Karevan, Z.; Suykens, J.A. Spatio-temporal stacked lstm for temperature prediction in weather forecasting. arXiv 2018 , arXiv:1811.06341. [ Google Scholar ]
• Chattopadhyay, A.; Nabizadeh, E.; Hassanzadeh, P. Analog Forecasting of extreme-causing weather patterns using deep learning. J. Adv. Model. Earth Syst. 2020 , 12 , e2019MS001958. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Sønderby, C.K.; Espeholt, L.; Heek, J.; Dehghani, M.; Oliver, A.; Salimans, T.; Alchbrenner, N. MetNet: A Neural Weather Model for Precipitation Forecasting. arXiv 2020 , arXiv:2003.12140. [ Google Scholar ]
• Pathak, J.; Subramanian, S.; Harrington, P.; Raja, S.; Chattopadhyay, A.; Mardani, M.; Anandkumar, A. FourCastNet: A Global Data-Driven High-Resolution Weather Model Using Adaptive Fourier Neural Operators. arXiv 2022 , arXiv:2202.11214. [ Google Scholar ]
• Lam, R.; Sanchez-Gonzalez, A.; Willson, M.; Wirnsberger, P.; Fortunato, M.; Pritzel, A.; Battaglia, P. GraphCast: Learning skillful medium-range global weather forecasting. arXiv 2022 , arXiv:2212.12794. [ Google Scholar ]
• Bi, K.; Xie, L.; Zhang, H.; Chen, X.; Gu, X.; Tian, Q. Accurate Medium-Range Global Weather Forecasting with 3D Neural Networks. Nature 2023 , 619 , 533–538. [ Google Scholar ] [ CrossRef ]
• Nguyen, T.; Brandstetter, J.; Kapoor, A.; Gupta, J.K.; Grover, A. ClimaX: A foundation model for weather and climate. arXiv 2023 , arXiv:2301.10343. [ Google Scholar ]
• Gangopadhyay, S.; Clark, M.; Rajagopalan, B. Statistical Down-scaling using K-nearest neighbors. In Water Resources Research ; Wiley Online Library: Hoboken, NJ, USA, 2005; Volume 41. [ Google Scholar ]
• Tripathi, S.; Srinivas, V.V.; Nanjundiah, R.S. Down-scaling of precipitation for climate change scenarios: A support vector machine approach. J. Hydrol. 2006 , 330 , 621–640. [ Google Scholar ] [ CrossRef ]
• Krasnopolsky, V.M.; Fox-Rabinovitz, M.S. Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction. Neural Netw. 2006 , 19 , 122–134. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Raje, D.; Mujumdar, P.P. A conditional random field–based Down-scaling method for assessment of climate change impact on multisite daily precipitation in the Mahanadi basin. In Water Resources Research ; Wiley Online Library: Hoboken, NJ, USA, 2009; Volume 45. [ Google Scholar ]
• Zarei, M.; Najarchi, M.; Mastouri, R. Bias correction of global ensemble precipitation forecasts by Random Forest method. Earth Sci. Inform. 2021 , 14 , 677–689. [ Google Scholar ] [ CrossRef ]
• Andersson, T.R.; Hosking, J.S.; Pérez-Ortiz, M.; Paige, B.; Elliott, A.; Russell, C.; Law, S.; Jones, D.C.; Wilkinson, J.; Phillips, T.; et al. Seasonal Arctic Sea Ice Forecasting with Probabilistic Deep Learning. Nat. Commun. 2021 , 12 , 5124. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Wang, X.; Han, Y.; Xue, W.; Yang, G.; Zhang, G. Stable climate simulations using a realistic general circulation model with neural network parameterizations for atmospheric moist physics and radiation processes. Geosci. Model Dev. 2022 , 15 , 3923–3940. [ Google Scholar ] [ CrossRef ]
• Baño-Medina, J.; Manzanas, R.; Cimadevilla, E.; Fernández, J.; González-Abad, J.; Cofiño, A.S.; Gutiérrez, J.M. Down-scaling Multi-Model Climate Projection Ensembles with Deep Learning (DeepESD): Contribution to CORDEX EUR-44. Geosci. Model Dev. 2022 , 15 , 6747–6758. [ Google Scholar ] [ CrossRef ]
• Hess, P.; Lange, S.; Boers, N. Deep Learning for bias-correcting comprehensive high-resolution Earth system models. arXiv 2022 , arXiv:2301.01253. [ Google Scholar ]
• Wang, F.; Tian, D. On deep learning-based bias correction and Down-scaling of multiple climate models simulations. Clim. Dyn. 2022 , 59 , 3451–3468. [ Google Scholar ] [ CrossRef ]
• Pan, B.; Anderson, G.J.; Goncalves, A.; Lucas, D.D.; Bonfils, C.J.W.; Lee, J. Improving Seasonal Forecast Using Probabilistic Deep Learning. J. Adv. Model. Earth Syst. 2022 , 14 , e2021MS002766. [ Google Scholar ] [ CrossRef ]
• Hu, Y.; Chen, L.; Wang, Z.; Li, H. SwinVRNN: A Data-Driven Ensemble Forecasting Model via Learned Distribution Perturbation. J. Adv. Model. Earth Syst. 2023 , 15 , e2022MS003211. [ Google Scholar ] [ CrossRef ]
• Chen, L.; Zhong, X.; Zhang, F.; Cheng, Y.; Xu, Y.; Qi, Y.; Li, H. FuXi: A cascade machine learning forecasting system for 15-day global weather forecast. arXiv 2023 , arXiv:2306.12873. [ Google Scholar ]
• Lin, H.; Gao, Z.; Xu, Y.; Wu, L.; Li, L.; Li, S.Z. Conditional local convolution for spatio-temporal meteorological forecasting. Proc. Aaai Conf. Artif. Intell. 2022 , 36 , 7470–7478. [ Google Scholar ] [ CrossRef ]
• Chen, K.; Han, T.; Gong, J.; Bai, L.; Ling, F.; Luo, J.J.; Chen, X.; Ma, L.; Zhang, T.; Su, R.; et al. FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead. arXiv 2023 , arXiv:2304.02948. [ Google Scholar ]
• De Burgh-Day, C.O.; Leeuwenburg, T. Machine Learning for numerical weather and climate modelling: A review. EGUsphere 2023 , 2023 , 1–48. [ Google Scholar ]
• LeCun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 1998 , 86 , 2278–2324. [ Google Scholar ] [ CrossRef ]
• Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2012 , 25 , 1097–1105. [ Google Scholar ] [ CrossRef ]
• Scherer, D.; Müller, A.; Behnke, S. Evaluation of pooling operations in convolutional architectures for object recognition. In Proceedings of the International Conference on Artificial Neural Networks 2010, Thessaloniki, Greece, 15–18 September 2010; pp. 92–101. [ Google Scholar ]
• LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015 , 521 , 436–444. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Liu, Y.; Racah, E.; Correa, J.; Khosrowshahi, A.; Lavers, D.; Kunkel, K.; Wehner, M.; Collins, W. Application of deep convolutional neural networks for detecting extreme weather in climate datasets. arXiv 2016 , arXiv:1605.01156. [ Google Scholar ]
• Goodfellow, I.; Warde-Farley, D.; Mirza, M.; Courville, A.; Bengio, Y. Maxout networks. In Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA, 16–21 June 2013; pp. 1319–1327. [ Google Scholar ]
• Marion, M.; Roger, T. Navier-Stokes equations: Theory and approximation. Handb. Numer. Anal. 1998 , 6 , 503–689. [ Google Scholar ]
• Iacono, M.J.; Mlawer, E.J.; Clough, S.A.; Morcrette, J.-J. Impact of an improved longwave radiation model, RRTM, on the energy budget and thermodynamic properties of the NCAR community climate model, CCM3. J. Geophys. Res. Atmos. 2000 , 105 , 14873–14890. [ Google Scholar ] [ CrossRef ]
• Guo, Y.; Shao, C.; Su, A. Comparative Evaluation of Rainfall Forecasts during the Summer of 2020 over Central East China. Atmosphere 2023 , 14 , 992. [ Google Scholar ] [ CrossRef ]
• Guo, Y.; Shao, C.; Su, A. Investigation of Land–Atmosphere Coupling during the Extreme Rainstorm of 20 July 2021 over Central East China. Atmosphere 2023 , 14 , 1474. [ Google Scholar ] [ CrossRef ]
• Bauer, P.; Thorpe, A.; Brunet, G. The Quiet Revolution of Numerical Weather Prediction. Nature 2015 , 525 , 47–55. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is All You Need. In Proceedings of the NeurIPS, Long Beach, CA, USA, 4–9 December 2017. [ Google Scholar ]
• Wang, H.; Zhu, Y.; Green, B.; Adam, H.; Yuille, A.; Chen, L.C. Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation. arXiv 2019 , arXiv:2003.07853. [ Google Scholar ]
• Schmit, T.J.; Griffith, P.; Gunshor, M.M.; Daniels, J.M.; Goodman, S.J.; Lebair, W.J. A closer look at the ABI on the GOES-R series. Bull. Am. Meteorol. Soc. 2017 , 98 , 681–698. [ Google Scholar ] [ CrossRef ]
• Li, Z.; Kovachki, N.; Azizzadenesheli, K.; Liu, B.; Bhattacharya, K.; Stuart, A.; Anandkumar, A. Fourier Neural Operator for Parametric Partial Differential Equations. In Proceedings of the International Conference on Learning Representations (ICLR), Virtual Event. 3–7 May 2021. [ Google Scholar ]
• Guibas, J.; Mardani, M.; Li, Z.; Tao, A.; Anandkumar, A.; Catanzaro, B. Adaptive Fourier Neural Operators: Efficient token mixers for transformers. In Proceedings of the International Conference on Representation Learning, Virtual Event. 25–29 April 2022. [ Google Scholar ]
• Rasp, S.; Thuerey, N. Purely data-driven medium-range weather forecasting achieves comparable skill to physical models at similar resolution. arXiv 2020 , arXiv:2008.08626. [ Google Scholar ]
• Weyn, J.A.; Durran, D.R.; Caruana, R.; Cresswell-Clay, N. Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models. arXiv 2021 , arXiv:2102.05107. [ Google Scholar ] [ CrossRef ]
• Rasp, S.; Dueben, P.D.; Scher, S.; Weyn, J.A.; Mouatadid, S.; Thuerey, N. Weatherbench: A benchmark data set for data-driven weather forecasting. J. Adv. Model. Earth Syst. 2020 , 12 , e2020MS002203. [ Google Scholar ] [ CrossRef ]
• Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the International Conference on Computer Vision, Virtual. 11–17 October 2021; IEEE: New York, NY, USA, 2021; pp. 10012–10022. [ Google Scholar ]
• Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv 2020 , arXiv:2010.11929. [ Google Scholar ]
• Váňa, F.; Düben, P.; Lang, S.; Palmer, T.; Leutbecher, M.; Salmond, D.; Carver, G. Single precision in weather forecasting models: An evaluation with the IFS. Mon. Weather Rev. 2017 , 145 , 495–502. [ Google Scholar ] [ CrossRef ]
• IPCC. Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change ; Cambridge University Press: Cambridge, UK; New York, NY, USA, 2013. [ Google Scholar ]
• Flato, G.; Marotzke, J.; Abiodun, B.; Braconnot, P.; Chou, S.C.; Collins, W.; Cox, P.; Driouech, F.; Emori, S.; Eyring, V.; et al. Evaluation of Climate Models. In Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change ; Cambridge University Press: Cambridge, UK; New York, NY, USA, 2013. [ Google Scholar ]
• Washington, W.M.; Parkinson, C.L. An Introduction to Three-Dimensional Climate Modeling ; University Science Books: Beijing, China, 2005. [ Google Scholar ]
• Giorgi, F.; Gutowski, W.J. Regional Dynamical Down-scaling and the CORDEX Initiative. Annu. Rev. Environ. Resour. 2015 , 40 , 467–490. [ Google Scholar ] [ CrossRef ]
• Randall, D.A.; Wood, R.A.; Bony, S.; Colman, R.; Fichefet, T.; Fyfe, J.; Kattsov, V.; Pitman, A.; Shukla, J.; Srinivasan, J.; et al. Climate Models and Their Evaluation. In Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change ; Cambridge University Press: Cambridge, UK; New York, NY, USA, 2007. [ Google Scholar ]
• Taylor, K.E.; Stouffer, R.J.; Meehl, G.A. An overview of CMIP5 and the experiment design. Bull. Am. Meteorol. Soc. 2012 , 93 , 485–498. [ Google Scholar ] [ CrossRef ]
• Miao, C.; Shen, Y.; Sun, J. Spatial–temporal ensemble forecasting (STEFS) of high-resolution temperature using machine learning models. J. Adv. Model. Earth Syst. 2019 , 11 , 2961–2973. [ Google Scholar ]
• Mukkavilli, S.; Perone, C.S.; Rangapuram, S.S.; Müller, K.R. Distribution regression forests for probabilistic spatio-temporal forecasting. In Proceedings of the International Conference on Machine Learning (ICML), Vienna, Austria, 12–18 July 2020. [ Google Scholar ]
• Walker, G.; Charlton-Perez, A.; Lee, R.; Inness, P. Challenges and progress in probabilistic forecasting of convective phenomena: The 2016 GFE/EUMETSAT/NCEP/SPC severe convective weather workshop. Bull. Am. Meteorol. Soc. 2016 , 97 , 1829–1835. [ Google Scholar ]
• Kingma, D.P.; Welling, M. Auto-encoding variational bayes. arXiv 2013 , arXiv:1312.6114. [ Google Scholar ]
• Krasting, J.P.; John, J.G.; Blanton, C.; McHugh, C.; Nikonov, S.; Radhakrishnan, A.; Zhao, M. NOAA-GFDL GFDL-ESM4 model output prepared for CMIP6 CMIP. Earth Syst. Grid Fed. 2018 , 10 . [ Google Scholar ] [ CrossRef ]
• Zhu, J.-Y.; Park, T.; Isola, P.; Efros, A.A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2223–2232. [ Google Scholar ]
• Brands, S.; Herrera, S.; Fernández, J.; Gutiérrez, J.M. How well do CMIP5 Earth System Models simulate present climate conditions in Europe and Africa? Clim. Dynam. 2013 , 41 , 803–817. [ Google Scholar ] [ CrossRef ]
• Vautard, R.; Kadygrov, N.; Iles, C. Evaluation of the large EURO-CORDEX regional climate model ensemble. J. Geophys. Res.-Atmos. 2021 , 126 , e2019JD032344. [ Google Scholar ] [ CrossRef ]
• Boé, J.; Somot, S.; Corre, L.; Nabat, P. Large discrepancies in summer climate change over Europe as projected by global and regional climate models: Causes and consequences. Clim. Dynam. 2020 , 54 , 2981–3002. [ Google Scholar ] [ CrossRef ]
• Baño-Medina, J.; Manzanas, R.; Gutiérrez, J.M. Configuration and intercomparison of deep learning neural models for statistical Down-scaling. Geosci. Model Dev. 2020 , 13 , 2109–2124. [ Google Scholar ] [ CrossRef ]
• Lecun, Y.; Bengio, Y. Convolutional Networks for Images, Speech, and Time-Series. Handb. Brain Theory Neural Netw. 1995 , 336 , 1995. [ Google Scholar ]
• Dee, D.P.; Uppala, S.M.; Simmons, A.J.; Berrisford, P.; Poli, P.; Kobayashi, S.; Andrae, U.; Balmaseda, M.A.; Balsamo, G.; Bauer, D.P.; et al. The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Q. J. Roy Meteor. Soc. 2011 , 137 , 553–597. [ Google Scholar ] [ CrossRef ]
• Cornes, R.C.; van der Schrier, G.; van den Besselaar, E.J.M.; Jones, P.D. An Ensemble Version of the E-OBS Temperature and Precipitation Data Sets. J. Geophys. Res.-Atmos. 2018 , 123 , 9391–9409. [ Google Scholar ] [ CrossRef ]
• Baño-Medina, J.; Manzanas, R.; Gutiérrez, J.M. On the suitability of deep convolutional neural networks for continentalwide Down-scaling of climate change projections. Clim. Dynam. 2021 , 57 , 1–11. [ Google Scholar ] [ CrossRef ]
• Maraun, D.; Widmann, M.; Gutiérrez, J.M.; Kotlarski, S.; Chandler, R.E.; Hertig, E.; Wibig, J.; Huth, R.; Wilcke, R.A. VALUE: A framework to validate Down-scaling approaches for climate change studies. Earths Future 2015 , 3 , 1–14. [ Google Scholar ] [ CrossRef ]
• Vrac, M.; Ayar, P. Influence of Bias Correcting Predictors on Statistical Down-scaling Models. J. Appl. Meteorol. Clim. 2016 , 56 , 5–26. [ Google Scholar ] [ CrossRef ]
• Williams, P.M. Modelling Seasonality and Trends in Daily Rainfall Data. In Advances in Neural Information Processing Systems 10, Proceedings of the Neural Information Processing Systems (NIPS): Denver, Colorado, USA, 1997 ; MIT Press: Cambridge, MA, USA, 1998; pp. 985–991. ISBN 0-262-10076-2. [ Google Scholar ]
• Cannon, A.J. Probabilistic Multisite Precipitation Down-scaling by an Expanded Bernoulli–Gamma Density Network. J. Hydrometeorol. 2008 , 9 , 1284–1300. [ Google Scholar ] [ CrossRef ]
• Schoof, J.T. and Pryor, S.C. Down-scaling temperature and precipitation: A comparison of regression-based methods and artificial neural networks. Int. J. Climatol. 2001 , 21 , 773–790. [ Google Scholar ] [ CrossRef ]
• Maraun, D.; Widmann, M. Statistical Down-Scaling and Bias Correction for Climate Research ; Cambridge University Press: Cambridge, UK, 2018; ISBN 9781107588783. [ Google Scholar ]
• Vrac, M.; Stein, M.; Hayhoe, K.; Liang, X.-Z. A general method for validating statistical Down-scaling methods under future climate change. Geophys. Res. Lett. 2007 , 34 , L18701. [ Google Scholar ] [ CrossRef ]
• San-Martín, D.; Manzanas, R.; Brands, S.; Herrera, S.; Gutiérrez, J.M. Reassessing Model Uncertainty for Regional Projections of Precipitation with an Ensemble of Statistical Down-scaling Methods. J. Clim. 2017 , 30 , 203–223. [ Google Scholar ] [ CrossRef ]
• Quesada-Chacón, D.; Barfus, K.; Bernhofer, C. Climate change projections and extremes for Costa Rica using tailored predictors from CORDEX model output through statistical Down-scaling with artificial neural networks. Int. J. Climatol. 2021 , 41 , 211–232. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Chen, L.; Han, B.; Wang, X.; Zhao, J.; Yang, W.; Yang, Z. Machine Learning Methods in Weather and Climate Applications: A Survey. Appl. Sci. 2023 , 13 , 12019. https://doi.org/10.3390/app132112019

Chen L, Han B, Wang X, Zhao J, Yang W, Yang Z. Machine Learning Methods in Weather and Climate Applications: A Survey. Applied Sciences . 2023; 13(21):12019. https://doi.org/10.3390/app132112019

Chen, Liuyi, Bocheng Han, Xuesong Wang, Jiazhen Zhao, Wenke Yang, and Zhengyi Yang. 2023. "Machine Learning Methods in Weather and Climate Applications: A Survey" Applied Sciences 13, no. 21: 12019. https://doi.org/10.3390/app132112019

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals