academic research reviews

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 
• How to write a good literature review 
• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

• Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 
• Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 
• Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 
• Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 
• Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 
• Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

How to write a good literature review

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. 

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• Life Sciences Papers: 9 Tips for Authors Writing in Biological Sciences
• What is an Argumentative Essay? How to Write It (With Examples)

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, how paperpal’s research feature helps you develop and..., how paperpal is enhancing academic productivity and accelerating..., how to write a successful book chapter for..., academic editing: how to self-edit academic text with..., 4 ways paperpal encourages responsible writing with ai, what are scholarly sources and where can you..., how to write a hypothesis types and examples , measuring academic success: definition & strategies for excellence, what is academic writing: tips for students, why traditional editorial process needs an upgrade.

• Peterborough

How to Write Academic Reviews

• What is a review?
• Common problems with academic reviews
• Getting started: approaches to reading and notetaking
• Understanding and analyzing the work
• Organizing and writing the review

What Is a Review?

A scholarly review describes, analyzes, and evaluates an article, book, film, or performance (through this guide we will use the term “work” to refer to the text or piece to be reviewed).  A review also shows how a work fits into its disciplines and explains the value or contribution of the work to the field.

Reviews play an important role in scholarship. They give scholars the opportunity to respond to one another’s research, ideas and interpretations. They also provide an up-to-date view of a discipline. We recommend you seek out reviews in current scholarly journals to become familiar with recent scholarship on a topic and to understand the forms review writing takes in your discipline. Published scholarly reviews are helpful models for beginner review-writers. However, we remind you that you are to write your own assessment of the work, not rely on the assessment from a review you found in a journal or on a blog.

As a review-writer, your objective is to:

• understand a work on its own terms (analyze it)
• bring your own knowledge to bear on a work (respond to it)
• critique the work while considering validity, truth, and slant (evaluate it)
• place the work in context (compare it to other works).

Common Problems with Academic Reviews

A review is not a research paper.

Rather than a research paper on the subject of the work,an academic review is an evaluation about the work’s message, strengths, and value. For example, a review of Finis Dunaway’s Seeing Green would not include your own research about media coverage of the environmental movement; instead, your review would assess Dunaway’s argument and its significance to the field.

A review is not a summary

It is important to synthesize the contents and significance of the work you review, but the main purpose of a review is to evaluate, critically analyze, or comment on the text. Keep your summary of the work brief, and make specific references to its message and evidence in your assessment of the work.

A review is not an off-the-cuff, unfair personal response

An effective review must be fair and accurate. It is important to see what is actually in front of you when your first reaction to the tone, argument, or subject of what you are reviewing is extremely negative or positive.

You will present your personal views on the work, but they must be explained and supported with evidence. Rather than writing, “I thought the book was interesting,” you can explain why the book was interesting and how it might offer new insights or important ideas. Further, you can expand on a statement such as “The movie was boring,” by explaining how it failed to interest you and pointing toward specific disappointing moments.

Getting Started: Approaches to Reading and Notetaking

Pre-reading.

Pre-reading helps a reader to see a book as a whole. Often, the acknowledgments, preface, and table of contents of a book offer insights about the book’s purpose and direction. Take time before you begin chapter one to read the introduction and conclusion, examine chapter titles, and to explore the index or references pages.

Read more about strategies for critical and efficient reading

Reverse outline

A reverse outline helps a reader analyze the content and argument of a work of non-fiction. Read each section of a text carefully and write down two things: 1) the main point or idea, and 2) its function in the text. In other words, write down what each section says and what it does. This will help you to see how the author develops their argument and uses evidence for support.

Double-entry notebook

In its simplest form, the double-entry notebook separates a page into two columns. In one column, you make observations about the work. In the other, you note your responses to the work. This notetaking method has two advantages. It forces you to make both sorts of notes — notes about the work and notes about your reaction to the work — and it helps you to distinguish between the two.

Whatever method of notetaking you choose, do take notes, even if these are scribbles in the margin. If you don’t, you might rely too heavily on the words, argument, or order of what you are reviewing when you come to write your review.                                              

Understand and Analyze the Work

It is extremely important to work toward seeing a clear and accurate picture of a work. One approach is to try to suspend your judgment for a while, focusing instead on describing or outlining a text. A student once described this as listening to the author’s voice rather than to their own.

Ask questions to support your understanding of the work.

Questions for Works of Non-Fiction

• What is the subject/topic of the work? What key ideas do you think you should describe in your review?
• What is the thesis, main theme, or main point?
• What major claims or conclusions does the author make? What issues does the work illuminate?
• What is the structure of the work? How does the author build their argument?
• What sources does the author consult? What evidence is used to support claims? Do these sources in any way “predetermine” certain conclusions?
• Is there any claim for which the evidence presented is insufficient or slight? Do any conclusions rest on evidence that may be atypical?
• How is the argument developed? How do the claims relate? What does the conclusion reveal?

Questions for Works of Fiction

• What is the main theme or message? What issues does the book illuminate?
• How does the work proceed? How does the author build their plot?
• What kind of language, descriptions, or sections of plot alert you to the themes and significance of the book?
• What does the conclusion reveal when compared with the beginning?

Read Critically

Being critical does not mean criticizing. It means asking questions and formulating answers. Critical reading is not reading with a “bad attitude.” Critical readers do not reject a text or take a negative approach to it; they inquire about a text, an author, themselves, and the context surrounding all three, and they attempt to understand how and why the author has made the particular choices they have.

Think about the Author

You can often tell a lot about an author by examining a text closely, but sometimes it helps to do a little extra research. Here are some questions about the author that would be useful to keep in mind when you are reading a text critically:

• Who is the author? What else has the author written?
• What does the author do? What experiences of the author’s might influence the writing of this book?
• What is the author’s main purpose or goal for the text? Why did they write it and what do they want to achieve?
• Does the author indicate what contribution the text makes to scholarship or literature? What does the author say about their point of view or method of approaching the subject? In other words, what position does the author take?

Think about Yourself

Because you are doing the interpreting and evaluating of a text, it is important to examine your own perspective, assumptions, and knowledge (positionality) in relation to the text. One way to do this is by writing a position statement that outlines your view of the subject of the work you are reviewing. What do you know, believe, or assume about this subject? What in your life might influence your approach to this text?

Here are some prompts that might help you generate a personal response to a book:

• I agree that ... because ...                    
• I disagree that ... because ...
• I don’t understand ...
• This reminds me of …
• I’m surprised by …                 

Another way to examine your thoughts in relation to a text is to note your initial response to the work. Consider your experience of the text – did you like it? Why or why not?

• What did I feel when I read this book? Why?
• How did I experience the style or tone of the author? How would I characterize each?
• What questions would I ask this author if I could?
• For me, what are the three best things about this book? The three worst things? Why?

Consider Context

A reviewer needs to examine the context of the book to arrive at a fair understanding and evaluation of its contents and importance. Context may include the scholarship to which this book responds or the author’s personal motive for writing. Or perhaps the context is simply contemporary society or today’s headlines. It is certainly important to consider how the work relates to the course that requires the review.

Here are some useful questions:

• What are the connections between this work and others on similar subjects? How does it relate to core concepts in my course or my discipline?
• What is the scholarly or social significance of this work? What contribution does it make to our understanding?
• What, of relevance, is missing from the work: certain kinds of evidence or methods of analysis/development? A particular theoretical approach? The experiences of certain groups?
• What other perspectives or conclusions are possible?

Once you have taken the time to thoroughly understand and analyze the work, you will have a clear perspective on its strengths and weaknesses and its value within the field. Take time to categorize your ideas and develop an outline; this will ensure your review is well organized and clear.

Organizing and Writing the Review

A review is organized around an assessment of the work or a focused message about its value to the field. Revisit your notes and consider your responses to your questions from critical reading to develop a clear statement that evaluates the work and provides an explanation for that evaluation.

For example:

X is an important work because it provides a new perspective on . . .

X’s argument is compelling because . . . ; however, it fails to address . . .

Although X claims to . . ., they make assumptions about . . . , which diminishes the impact . . .

This statement or evaluation is presented in the introduction. The body of the review works to support or explain your assessment; organize your key ideas or supporting arguments into paragraphs and use evidence from the book, article, or film to demonstrate how the work is (or is not) effective, compelling, provocative, novel, or informative.

As with all scholarly writing, a well-organized structure supports the clarity of your review. There is not a rigid formula for organization, but you may find the following guidelines to be helpful. Note that reviews do not typically include subheadings; the headings listed here serve to help you think about the main sections of your academic review.

Introduction

Introduce the work, the author (or director/producer), and the points you intend to make about this work. In addition, you should

• give relevant bibliographic information
• give the reader a clear idea of the nature, scope, and significance of the work
• indicate your evaluation of the work in a clear 1-2 sentence thesis statement

Provide background information to help your readers understand the importance of the work or the reasons for your appraisal. Background information could include:

• why the issue examined is of current interest
• other scholarship about this subject
• the author’s perspective, methodology, purpose
• the circumstances under which the book was created

Sample Introduction

Within educational research, much attention has been given to the importance of diversity and equity, and the literature is rife with studies detailing the best ways to create environments that are supportive of diverse students. In “Guidance Matters,” however, Carpenter and Diem (2015) examined these concepts in a less-studied source: policy documents related to leadership training.  Using discourse analysis, they explored the ways in which government policies concerning the training of educational administrators discussed issues of diversity and equity. While their innovative methods allowed them to reveal the ways in which current policy promotes superficial platitudes to diversity rather than a deep commitment to promoting social justice, their data analysis left many of their identified themes vague and their discussion did not provide a clear explanation of the applications of their findings.

What works in this sample introduction:

• The nature of the larger issue, how best to create diversity and equity within educational environments, is clearly laid out.
• The paragraph clearly introduces the authors and study being reviewed and succinctly explains how they have addressed the larger issue of equity and diversity in a unique way.
• The paragraph ends with a clear thesis that outlines the strengths and weaknesses of the work.

Summary of the Work

Keep the summary of the work short! A paragraph or two should be sufficient. Summarize its contents very briefly and focus on:

• the purpose of the work
• the main points of the work
• the ideas, themes, or arguments that you will evaluate or discuss in the review

Analysis and Evaluation

Analyze and explain the significance of the main points of the work. Evaluate the work, answering questions such as the following:

• Does the work do what its author claimed it would?
• Is the work valid and accurate?
• How does the work fit into scholarship in the field?
• What are your reasons for agreeing, disagreeing, liking, disliking, believing, disbelieving?

Note that this section will take up the bulk of your review and should be organized into paragraphs. Because this form of writing typically does not use subheadings, strong paragraphing, particularly the use of clear topic sentences, is essential. Read more on paragraphing.

Reviews are informed by your critical reading or viewing of a work; therefore you need to include specific evidence from the work to support your claims about its message and its impact. Your writing and  your assessment of the work will be most effective if you paraphrase or summarize the evidence you use, rather than relying on direct quotations. Be sure to follow the rules for citation in your discipline. Read more on paraphrasing and summarizing.

Sample Body Paragraph

One of the strengths of Carpenter and Diem’s  (2015) study was innovative use of  and nuanced explanation of discourse analysis. Critiquing much of the research on policy for its positivist promises of “value neutral and empirically objective” (p. 518) findings, Carpenter and Diem (2015) argued that discourse theory can provide an important lens through which to view policy and its relationship to educational outcomes.  By interrogating the “inscribed discourses of policy making” (p. 518), they showed how policy language constructs particular social meanings of concepts such as diversity and equity. Significantly, this analysis was not simply about the language used within documents; instead, Carpenter and Diem (2015) argued that the language used was directly related to reality. Their “study examine[d] how dominant discourses related to equity, and their concretization within guiding policy documents, may shape the ways in which states, local school districts, and educational leaders are asked to consider these issues in their everyday practice” (Carpenter & Diem, 2015, p. 519). Thus, through the use of discourse theory, Carpenter and Diem (2015) framed policy language, which some might consider abstract or distant from daily life, as directly connected to the experience of educational leaders.

What works in this sample body paragraph:

• The paragraph begins with a clear topic sentence that connects directly to a strength mentioned in the thesis of the review.
• The paragraph provides specific details and examples to support how and why their methods are innovative.
• The direct quotations used are short and properly integrated into the sentences.

The paragraph concludes by explaining the significance of the innovative methods to the larger work.

Conclusion and Recommendation

Give your overall assessment of the work. Explain the larger significance of your assessment. Consider who would benefit from engaging with this work.

Literature Reviews

What this handout is about.

This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Introduction

OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?

Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.

What is a literature review, then?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Who writes these things, anyway?

Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.

Let’s get to it! What should I do before writing the literature review?

If your assignment is not very specific, seek clarification from your instructor:

• Roughly how many sources should you include?
• What types of sources (books, journal articles, websites)?
• Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
• Should you evaluate your sources?
• Should you provide subheadings and other background information, such as definitions and/or a history?

Find models

Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.

Narrow your topic

There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.

Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .

And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.

Consider whether your sources are current

Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

• Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
• Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
• Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

• Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
• By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
• By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
• Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
• Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

• Current Situation: Information necessary to understand the topic or focus of the literature review.
• History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Begin composing

Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.

Keep your own voice

While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .

Revise, revise, revise

Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.

Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.

Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.

Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.

Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 12 November 2021

Demystifying the process of scholarly peer-review: an autoethnographic investigation of feedback literacy of two award-winning peer reviewers

• Sin Wang Chong   ORCID: orcid.org/0000-0002-4519-0544 1 &
• Shannon Mason 2  

Humanities and Social Sciences Communications volume  8 , Article number:  266 ( 2021 ) Cite this article

2798 Accesses

6 Citations

18 Altmetric

Metrics details

• Language and linguistics

A Correction to this article was published on 26 November 2021

This article has been updated

Peer reviewers serve a vital role in assessing the value of published scholarship and improving the quality of submitted manuscripts. To provide more appropriate and systematic support to peer reviewers, especially those new to the role, this study documents the feedback practices and experiences of two award-winning peer reviewers in the field of education. Adopting a conceptual framework of feedback literacy and an autoethnographic-ecological lens, findings shed light on how the two authors design opportunities for feedback uptake, navigate responsibilities, reflect on their feedback experiences, and understand journal standards. Informed by ecological systems theory, the reflective narratives reveal how they unravel the five layers of contextual influences on their feedback practices as peer reviewers (micro, meso, exo, macro, chrono). Implications related to peer reviewer support are discussed and future research directions are proposed.

Similar content being viewed by others

The transformative power of values-enacted scholarship

What matters in the cultivation of student feedback literacy: exploring university efl teachers’ perceptions and practices, why english exploring chinese early career returnee academics’ motivations for writing and publishing in english, introduction.

The peer-review process is the longstanding method by which research quality is assured. On the one hand, it aims to assess the quality of a manuscript, with the desired outcome being (in theory if not always in practice) that only research that has been conducted according to methodological and ethical principles be published in reputable journals and other dissemination outlets (Starck, 2017 ). On the other hand, it is seen as an opportunity to improve the quality of manuscripts, as peers identify errors and areas of weakness, and offer suggestions for improvement (Kelly et al., 2014 ). Whether or not peer review is actually successful in these areas is open to considerable debate, but in any case it is the “critical juncture where scientific work is accepted for publication or rejected” (Heesen and Bright, 2020 , p. 2). In contemporary academia, where higher education systems across the world are contending with decreasing levels of public funding, there is increasing pressure on researchers to be ‘productive’, which is largely measured by the number of papers published, and of funding grants awarded (Kandiko, 2010 ), both of which involve peer review.

Researchers are generally invited to review manuscripts once they have established themselves in their disciplinary field through publication of their own research. This means that for early career researchers (ECRs), their first exposure to the peer-review process is generally as an author. These early experiences influence the ways ECRs themselves conduct peer review. However, negative experiences can have a profound and lasting impact on researchers’ professional identity. This appears to be particularly true when feedback is perceived to be unfair, with feedback tone largely shaping author experience (Horn, 2016 ). In most fields, reviewers remain anonymous to ensure freedom to give honest and critical feedback, although there are concerns that a lack of accountability can result in ‘bad’ and ‘rude’ reviews (Mavrogenis et al., 2020 ). Such reviews can negatively impact all researchers, but disproportionately impact underrepresented researchers (Silbiger and Stubler, 2019 ). Regardless of career phase, no one is served well by unprofessional reviews, which contribute to the ongoing problem of bullying and toxicity prevalent in academia, with serious implications on the health and well-being of researchers (Keashly and Neuman, 2010 ).

Because of its position as the central process through which research is vetted and refined, peer review should play a similarly central role in researcher training, although it rarely features. In surveying almost 3000 researchers, Warne ( 2016 ) found that support for reviewers was mostly received “in the form of journal guidelines or informally as advice from supervisors or colleagues” (p. 41), with very few engaging in formal training. Among more than 1600 reviewers of 41 nursing journals, only one third received any form of support (Freda et al., 2009 ), with participants across both of these studies calling for further training. In light of the lack of widespread formal training, most researchers learn ‘on the job’, and little is known about how researchers develop their knowledge and skills in providing effective assessment feedback to their peers. In this study, we undertake such an investigation, by drawing on our first-hand experiences. Through a collaborative and reflective process, we look to identify the forms and forces of our feedback literacy development, and seek to answer specifically the following research questions:

What are the exhibited features of peer reviewer feedback literacy?

What are the forces at work that affect the development of feedback literacy?

Literature review

Conceptualisation of feedback literacy.

The notion of feedback literacy originates from the research base of new literacy studies, which examines ‘literacies’ from a sociocultural perspective (Gee, 1999 ; Street, 1997 ). In the educational context, one of the most notable types of literacy is assessment literacy (Stiggins, 1999 ). Traditionally, assessment literacy is perceived as one of the indispensable qualities of a successful educator, which refers to the skills and knowledge for teachers “to deal with the new world of assessment” (Fulcher, 2012 , p. 115). Following this line of teacher-oriented assessment literacy, recent attempts have been made to develop more subject-specific assessment literacy constructs (e.g., Levi and Inbar-Lourie, 2019 ). Given the rise of student-centred approaches and formative assessment in higher education, researchers began to make the case for students to be ‘assessment literate’; comprising of such knowledge and skills as understanding of assessment standards, the relationship between assessment and learning, peer assessment, and self-assessment skills (Price et al., 2012 ). Feedback literacy, as argued by Winstone and Carless ( 2019 ), is essentially a subset of assessment literacy because “part of learning through assessment is using feedback to calibrate evaluative judgement” (p. 24). The notion of feedback literacy was first extensively discussed by Sutton ( 2012 ) and more recently by Carless and Boud ( 2018 ). Focusing on students’ feedback literacy, Sutton ( 2012 ) conceptualised feedback literacy as a three-dimensional construct—an epistemological dimension (what do I know about feedback?), an ontological dimension (How capable am I to understand feedback?), and a practical dimension (How can I engage with feedback?). In close alignment with Sutton’s construct, the seminal conceptual paper by Carless and Boud ( 2018 ) further illustrated the four distinctive abilities of feedback literate students: the abilities to (1) understand the formative role of feedback, (2) make informed and accurate evaluative judgement against standards, (3) manage emotions especially in the face of critical and harsh feedback, and (4) take action based on feedback. Since the publication of Carless and Boud ( 2018 ), student and teacher feedback literacy has been in the limelight of assessment research in higher education (e.g., Chong 2021b ; Carless and Winstone 2020 ). These conceptual contributions expand the notion of feedback literacy to consider not only the manifestations of various forms of effective student engagement with feedback but also the confluence of contexts and individual differences of students in developing students’ feedback literacy by drawing upon various theoretical perspectives (e.g., ecological systems theory; sociomaterial perspective) and disciplines (e.g., business and human resource management). Others address practicalities of feedback literacy; for example, how teachers and students can work in synergy to develop feedback literacy (Carless and Winstone, 2020 ) and ways to maximise student engagement with feedback at a curricular level (Malecka et al., 2020). In addition to conceptualisation, advancement of the notion of feedback literacy is evident in the recent proliferation of primary studies. The majority of these studies are conducted in the field of higher education, focusing mostly on student feedback literacy in classrooms (e.g., Molloy et al., 2019 ; Winstone et al., 2019 ) and in the workplace (Noble et al., 2020 ), with a handful focused on teacher feedback literacy (e.g., Xu and Carless 2016 ). Some studies focusing on student feedback literacy adopt a qualitative case study research design to delve into individual students’ experience of engaging with various forms of feedback. For example, Han and Xu ( 2019 ) analysed the profiles of feedback literacy of two Chinese undergraduate students. Findings uncovered students’ resistance to engagement with feedback, which relates to the misalignment between the cognitive, social, and affective components of individual students’ feedback literacy profiles. Others reported interventions designed to facilitate students’ uptake of feedback, focusing on their effectiveness and students’ perceptions. Specifically, affordances and constraints of educational technology such as electronic feedback portfolio (Chong, 2019 ; Winstone et al., 2019 ) are investigated. Of particular interest is a recent study by Noble et al. ( 2020 ), which looked into student feedback literacy in the workplace by probing into the perceptions of a group of Australian healthcare students towards a feedback literacy training programme conducted prior to their placement. There is, however, a dearth of primary research in other areas where elicitation, process, and enactment of feedback are vital; for instance, academics’ feedback literacy. In the ‘publish or perish’ culture of higher education, academics, especially ECRs, face immense pressure to publish in top-tiered journals in their fields and face the daunting peer-review process, while juggling other teaching and administrative responsibilities (Hollywood et al., 2019 ; Tynan and Garbett 2007 ). Taking up the role of authors and reviewers, researchers have to possess the capacity and disposition to engage meaningfully with feedback provided by peer reviewers and to provide constructive comments to authors. Similar to students, researchers have to learn how to manage their emotions in the face of critical feedback, to understand the formative values of feedback, and to make informed judgements about the quality of feedback (Gravett et al., 2019 ). At the same time, feedback literacy of academics also resembles that of teachers. When considering the kind of feedback given to authors, academics who serve as peer reviewers have to (1) design opportunities for feedback uptake, (2) maintain a professional and supportive relationship with authors, and (3) take into account the practical dimension of giving feedback (e.g., how to strike a balance between quality of feedback and time constraints due to multiple commitments) (Carless and Winstone 2020 ). To address the above, one of the aims of the present study is to expand the application of feedback literacy as a useful analytical lens to areas outside the classroom, that is, scholarly peer-review activities in academia, by presenting, analysing, and synthesising the personal experiences of the authors as successful peer reviewers for academic journals.

Conceptual framework

We adopt a feedback literacy of peer reviewers framework (Chong 2021a ) as an analytical lens to analyse, systemise, and synthesise our own experiences and practices as scholarly peer reviewers (Fig. 1 ). This two-tier framework includes a dimension on the manifestation of feedback literacy, which categorises five features of feedback literacy of peer reviewers, informed by student and teacher feedback literacy frameworks by Carless and Boud ( 2018 ) and Carless and Winstone ( 2020 ). When engaging in scholarly peer review, reviewers are expected to be able to provide constructive and formative feedback, which authors can act on in their revisions ( engineer feedback uptake ). Besides, peer reviewers who are usually full-time researchers or academics lead hectic professional lives; thus, when writing reviewers’ reports, it is important for them to consider practically and realistically the time they can invest and how their various degrees of commitment may have an impact on the feedback they provide ( navigate responsibilities ). Furthermore, peer reviewers should consider the emotional and relational influences their feedback exert on the authors. It is crucial for feedback to be not only informative but also supportive and professional (Chong, 2018 ) ( maintain relationships ). Equally important, it is imperative for peer reviewers to critically reflect on their own experience in the scholarly peer-review process, including their experience of receiving and giving feedback to academic peers, as well as the ways authors and editors respond to their feedback ( reflect on feedback experienc e). Lastly, acting as gatekeepers of journals to assess the quality of manuscripts, peer reviewers have to demonstrate an accurate understanding of the journals’ aims, remit, guidelines and standards, and reflect those in their written assessments of submitted manuscripts ( understand standards ). Situated in the context of scholarly peer review, this collaborative autoethnographic study conceptualises feedback literacy not only as a set of abilities but also orientations (London and Smither, 2002 ; Steelman and Wolfeld, 2016 ), which refers to academics’ tendency, beliefs, and habits in relation to engaging with feedback (London and Smither, 2002 ). According to Cheung ( 2000 ), orientations are influenced by a plethora of factors, namely experiences, cultures, and politics. It is important to understand feedback literacy as orientations because it takes into account that feedback is a convoluted process and is influenced by a plethora of contextual and personal factors. Informed by ecological systems theory (Bronfenbrenner, 1986 ; Neal and Neal, 2013 ) and synthesising existing feedback literacy models (Carless and Boud, 2018 ; Carless and Winstone, 2020 ; Chong, 2021a , 2021b ), we consider feedback literacy as a malleable, situated, and emergent construct, which is influenced by the interplay of various networked layers of ecological systems (Neal and Neal, 2013 ) (Fig. 1 ). Also important is that conceptualising feedback literacy as orientations avoids dichotomisation (feedback literate vs. feedback illiterate), emphasises the developmental nature of feedback literacy, and better captures the multifaceted manifestations of feedback engagement.

The outer ring of the figure shows the components of feedback literacy while the inner ring concerns the layers of contexts (ecosystems) which influence the manifestation of feedback literacy of peer reviewers.

Echoing recent conceptual papers on feedback literacy which emphasises the indispensable role of contexts (Chong 2021b ; Boud and Dawson, 2021 ; Gravett et al., 2019 ), our conceptual framework includes an underlying dimension of networked ecological systems (micro, meso, exo, macro, and chrono), which portrays the contextual forces shaping our feedback orientations. Informed by the networked ecological system theory of Neal and Neal ( 2013 ), we postulate that there are five systems of contextual influence, which affect the feedback experience and development of feedback literacy of peer reviewers. The five ecological systems refer to ‘settings’, which is defined by Bronfenbrenner ( 1986 ) as “place[s] where people can readily engage in social interactions” (p. 22). Even though Bronfenbrenner’s ( 1986 ) somewhat dated definition of ‘place’ is limited to ‘physical space’, we believe that ‘places’ should be more broadly defined in the 21st century to encompass physical and virtual, recent and dated, closed and distanced locations where people engage; as for ‘interactions’, from a sociocultural perspective, we understand that ‘interactions’ can include not only social, but also cognitive and emotional exchanges (Vygotsky, 1978 ). Microsystem refers to a setting where people, including the focal individual, interact. Mesosystem , on the other hand, means the interactions between people from different settings and the influence they exert on the focal individual. An exosystem , similar to a microsystem, is understood as a single setting but this setting excludes the focal individual but it is likely that participants in this setting would interact with the focal individual. The remaining two systems, macrosystem and chronosystem, refer not only to ‘settings’ but ‘forces that shape the patterns of social interactions that define settings’ (Neal and Neal, 2013 , p. 729). Macrosystem is “the set of social patterns that govern the formation and dissolution of… interactions… and thus the relationship among ecological systems” (ibid). Some examples of macrosystems given by Neal and Neal ( 2013 ) include political and cultural systems. Finally, chronosystem is “the observation that patterns of social interactions between individuals change over time, and that such changes impact on the focal individual” (ibid, p. 729). Figure 2 illustrates this networked ecological systems theory using a hypothetical example of an early career researcher who is involved in scholarly peer review for Journal A; at the same time, they are completing a PhD and are working as a faculty member at a university.

This is a hypothetical example of an early career researcher who is involved in scholarly peer review for Journal A.

From the reviewed literature on the construct of feedback literacy, the investigation of feedback literacy as a personal, situated, and unfolding process is best done through an autoethnographic lens, which underscores critical self-reflection. Autoethnography refers to “an approach to research and writing that seeks to describe and systematically analyse (graphy) personal experience (auto) in order to understand cultural experience (ethno)” (Ellis et al., 2011 , p. 273). Autoethnography stems from research in the field of anthropology and is later introduced to the fields of education by Ellis and Bochner ( 1996 ). In higher education research, autoethnographic studies are conducted to illuminate on topics related to identity and teaching practices (e.g., Abedi Asante and Abubakari, 2020 ; Hains-Wesson and Young 2016 ; Kumar, 2020 ). In this article, a collaborative approach to autoethnography is adopted. Based on Chang et al. ( 2013 ), Lapadat ( 2017 ) defines collaborative autoethnography (CAE) as follows:

… an autobiographic qualitative research method that combines the autobiographic study of self with ethnographic analysis of the sociocultural milieu within which the researchers are situated, and in which the collaborating researchers interact dialogically to analyse and interpret the collection of autobiographic data. (p. 598)

CAE is not only a product but a worldview and process (Wall, 2006 ). CAE is a discrete view about the world and research, which straddles between paradigmatic boundaries of scientific and literary studies. Similar to traditional scientific research, CAE advocates systematicity in the research process and consideration is given to such crucial research issues as reliability, validity, generalisability, and ethics (Lapadat, 2017 ). In closer alignment with studies on humanities and literature, the goal of CAE is not to uncover irrefutable universal truths and generate theories; instead, researchers of CAE are interested in co-constructing and analysing their own personal narratives or ‘stories’ to enrich and/or challenge mainstream beliefs and ideas, embracing diverse rather than canonical ways of behaviour, experience, and thinking (Ellis et al., 2011 ). Regarding the role of researchers, CAE researchers openly acknowledge the influence (and also vulnerability) of researchers throughout the research process and interpret this juxtaposition of identities between researchers and participants of research as conducive to offering an insider’s perspective to illustrate sociocultural phenomena (Sughrua, 2019 ). For our CAE on the scholarly peer-review experiences of two ECRs, the purpose is to reconstruct, analyse, and publicise our lived experience as peer reviewers and how multiple forces (i.e., ecological systems) interact to shape our identity, experience, and feedback practice. As a research process, CAE is a collaborative and dynamic reflective journey towards self-discovery, resulting in narratives, which connect with and add to the existing literature base in a personalised manner (Ellis et al., 2011 ). The collaborators should go beyond personal reflection to engage in dialogues to identify similarities and differences in experiences to throw new light on sociocultural phenomena (Merga et al., 2018 ). The iterative process of self- and collective reflections takes place when CAE researchers write about their own “remembered moments perceived to have significantly impacted the trajectory of a person’s life” and read each other’s stories (Ellis et al., 2011 , p. 275). These ‘moments’ or vignettes are usually written retrospectively, selectively, and systematically to shed light on facets of personal experience (Hughes et al., 2012 ). In addition to personal stories, some autoethnographies and CAEs utilise multiple data sources (e.g., reflective essays, diaries, photographs, interviews with co-researchers) and various ways of expressions (e.g., metaphors) to achieve some sort of triangulation and to present evidence in a ‘systematic’ yet evocative manner (Kumar, 2020 ). One could easily notice that overarching methodological principles are discussed in lieu of a set of rigid and linear steps because the process of reconstructing experience through storytelling can be messy and emergent, and certain degree of flexibility is necessary. However, autoethnographic studies, like other primary studies, address core research issues including reliability (reader’s judgement of the credibility of the narrator), validity (reader’s judgement that the narratives are believable), and generalisability (resemblance between the reader’s experience and the narrative, or enlightenment of the reader regarding unfamiliar cultural practices) (Ellis et al., 2011 ). Ethical issues also need to be considered. For example, authors are expected to be honest in reporting their experiences; to protect the privacy of the people who ‘participated’ in our stories, pseudonyms need to be used (Wilkinson, 2019 ). For the current study, we follow the suggested CAE process outlined by Chang et al. ( 2013 ), which includes four stages: deciding on topic and method , collecting materials , making meaning , and writing . When deciding on the topic, we decided to focus on our experience as scholarly peer reviewers because doing peer review and having our work reviewed are an indispensable part of our academic lives. The next is to collect relevant autoethnographic materials. In this study, we follow Kumar ( 2020 ) to focus on multiple data sources: (1) reflective essays which were written separately through ‘recalling’, which is referred to by Chang et al. ( 2013 ) as ‘a free-spirited way of bringing out memories about critical events, people, place, behaviours, talks, thoughts, perspectives, opinions, and emotions pertaining to the research topic’ (p. 113), and (2) discussion meetings. In our reflective essays, we included written records of reflection and excerpts of feedback in our peer-review reports. Following material collection is meaning making. CAE, as opposed to autoethnography, emphasises the importance of engaging in dialogues with collaborators and through this process we identify similarities and differences in our experiences (Sughrua, 2019 ). To do so, we exchanged our reflective essays; we read each other’s reflections and added questions or comments on the margins. Then, we met online twice to share our experiences and exchange views regarding the two reflective essays we wrote. Both meetings lasted for approximately 90 min, were audio-recorded and transcribed. After each meeting, we coded our stories and experiences with reference to the two dimensions of the ecological framework of feedback literacy (Fig. 1 ). With regards to coding our data, we followed the model of Miles and Huberman ( 1994 ), which comprises four stages: data reduction (abstracting data), data display (visualising data in tabular form), conclusion-drawing, and verification. The coding and writing processes were done collaboratively on Google Docs and care was taken to address the aforesaid ethical (e.g., honesty, privacy) and methodological issues (e.g., validity, reliability, generalisability). As a CAE study, the participants are the researchers themselves, that is, the two authors of this paper. We acknowledge that research data are collected from human subjects (from the two authors), such data are collected in accordance with the standards and guidelines of the School Research Ethics Committee at the School of Social Sciences, Education and Social Work, Queen’s University Belfast (Ref: 005_2021). Despite our different experiences in our unique training and employment contexts, we share some common characteristics, both being ECRs (<5 years post-PhD), working in the field of education, active in the scholarly publication process as both authors and peer reviewers. Importantly for this study, we were both recipients of Reviewer of the Year Award 2019 awarded jointly by the journal, Higher Education Research & Development and the publisher , Taylor & Francis. This award in recognition of the quality of our reviewing efforts, as determined by the editorial board of a prestigious higher education journal, provided a strong impetus for this study, providing an opportunity to reflect on our own experiences and practices. The extent of our peer-review activities during our early career leading up to the time of data collection is summarised in Table 1 .

Findings and discussion

Analysis of the four individual essays (E1 and E2 for each participant) and transcripts of the two subsequent discussions (D1 and D2) resulted in the identification of multiple descriptive codes and in turn a number of overarching themes (Supplementary Appendix 1). Our reporting of these themes is guided by our conceptual framework, where we first focus on the five manifestations of feedback literacy to highlight the experiences that contribute to our growth as effective and confident peer reviewers. Then, we report on the five ecological systems to unravel how each contextual layer develops our feedback literacy as peer reviewers. (Note that the discussion of the chronosystem has been necessarily incorporated into each of the four others dimensions: microsystem , mesosystem , exosystem , and macrosystem in order to demonstrate temporal changes). In particular, similarities and differences will be underscored, and connections with manifested feedback beliefs and behaviours will be made. We include quotes from both Author 1 (A1) and Author 2 (A2), in order to illustrate our findings, and to show the richness and depth of the data collected (Corden and Sainsbury, 2006 ). Transcribed quotes may be lightly edited while retaining meaning, for example through the removal of fillers and repetitions, which is generally accepted practice to ensure readability ( ibid ).

Manifestations of feedback literacy

Engineering feedback uptake.

The two authors have a strong sense of the purpose of peer review as promoting not only research quality, but the growth of researchers. One way that we engineer author uptake is to ensure that feedback is ‘clear’ (A2,E1), ‘explicit’ (A2,E1), ‘specific’ (A1,E1), and importantly ‘actionable… to ensure that authors can act on this feedback so that their manuscripts can be improved and ultimately accepted for publication’ (A1,E1). In less than favourable author outcomes, we ensure that there is reference to the role of the feedback in promoting the development of the manuscript, which A1 refers to as ‘promotion of a growth mindset’ (A1,E1). For example, after requesting a second round of major revisions, A2 ‘acknowledged the frustration that the author might have felt on getting further revisions by noting how much improvement was made to the paper, but also making clear the justification for sending it off for more work’ (A2,E1). We both note that we tend to write longer reviews when a rejection is the recommended outcome, as our ultimate goal is to aid in the development of a manuscript.

Rejections doesn’t mean a paper is beyond repair. It can still be fixed and improved; a rejection simply means that the fix may be too extensive even for multiple review cycles. It is crucial to let the authors whose manuscripts are rejected know that they can still act on the feedback to improve their work; they should not give up on their own work. I think this message is especially important to first-time authors or early career researchers. (A1,E1)

In promoting a growth mindset and in providing actionable feedback, we hope to ‘show the authors that I’m not targeting them, but their work’ (A1,D1). We particularly draw on our own experiences as ECRs, with first-hand understanding that ‘everyone takes it personally when they get rejected. Yeah. Moreover, it is hard to separate (yourself from the paper)’ (A2,D1).

Navigating responsibilities

As with most academics, the two authors have multiple pressures on their time, and there ‘isn’t much formal recognition or reward’ (A1,E1) and ‘little extrinsic incentive for me to review’ (A2,E1). Nevertheless we both view our roles as peer reviewers as ‘an important part of the process’ (A2,E1), ‘a modest way for me to give back to the academic community’ (A1,E1). Through peer review we have built a sense of ‘identity as an academic’ (A1,D1), through ‘being a member of the academic community’ (A2,D1). While A1 commits to ‘review as many papers as possible’ (A1,E1) and A2 will usually accept offers to review, there are still limits on our time and therefore we consider the topic and methods employed when deciding whether or not to accept an invitation, as well as the journal itself, as we feel we can review more efficiently for journals with which we are more familiar. A1 and A2 have different processes for conducting their review that are most efficient for their own situations. For A1, the process begins with reading the whole manuscript in one go, adding notes to the pdf document along the way, which he then reviews, and makes a tentative decision, including ‘a few reasons why I have come to this decision’ (A1,E1). After waiting at least one day, he reviews all of the notes and begins writing the report, which is divided into the sections of the paper. He notes it ‘usually takes me 30–45 min to write a report. I then proofread this report and submit it to the system. So it usually takes me no more than three hours to complete a review’ (A1,E1). For A2, the process for reviewing and structuring the report is quite different, with a need to ‘just find small but regular opportunities to work on the review’ (A2,E1). As was the case during her Ph.D, which involved juggling research and raising two babies, ‘I’ve trained myself to be able to do things in bits’ (A2,D1). So A2 also begins by reading the paper once through, although generally without making initial comments. The next phase involves going through the paper at various points in time whenever possible, and at the same time building up the report, making the report structurally slightly different to that of A1.

What my reviews look like are bullet points, basically. And they’re not really in a particular order. They generally… follow the flow (of the paper). But I mean, I might think of something, looking at the methods and realise, hey, you haven’t defined this concept in the literature review so I’ll just add you haven’t done this. And so I will usually preface (the review)… Here’s a list of suggestions. Some of them are minor, some of them are serious, but they’re in no particular order. (A1,D1)

As such, both reviewers engage in personalised strategies to make more effective use of their time. Both A1 and A2 give explicit but not exhaustive examples of an area of concern, and they also pose questions for the author to consider, in both cases placing the onus back on the author to take action. As A1 notes, ‘I’m not going to do a summary of that reference for you. I’m just going to include that there. If you’d like you can check it out’ (A1,D1). For A2, a lack of adequate reporting of the methods employed in a study makes it difficult to proceed, and in such cases will not invest further time, sending it back to the editor, because ‘I can’t even comment on the findings… I can’t go on. I’m not gonna waste my time’ (A2,D1). In cases where the authors may be ‘on the fence’ about a particular review, they will use the confidential comments to the editor to help work through difficult cases as ‘they are obviously very experienced reviewers’ (A1,D1). Delegating tasks to the expertise of the editorial teams when appropriate also ensures time is used more prudently.

Maintaining relationships

Except in a few cases where A2 has reviewed for journals with a single-blind model, the vast majority of the reviews that we have completed have been double-blind. This means that we are unaware of the identity of the author/s, and we are unknown to them. However, ‘even with blind-reviews I tend to think of it as a conversation with a person’ (A2,E1). A1 talks about the need to have respect for the author and their expertise and effort ‘regardless of the quality of the submission (which can be in some cases subjective)’ (A1,E1). A2 writes similarly about the ‘privilege’ and ‘responsibility’ of being able to review manuscripts that authors ‘have put so much time and energy into possibly over an extended period’ (A2,E1). In this way it is possible to develop a sort of relationship with an author even without knowing their identity. In trying to articulate the nature of that relationship (which we struggle to do so definitively), we note that it is more than just a reviewer, and A2 reflected on a recent review, which went through a number of rounds of resubmission where ‘it felt like we were developing a relationship, more like a mentor than a reviewer’ (A2,E1).

I consider this role as a peer reviewer more than giving helpful and actionable feedback; I would like to be a supporter and critical friend to the authors, even though in most cases I don’t even know who they are or what career stage they are at (A1,E1).

In any case, as A1 notes, ‘we don’t even need to know who that person is because we know that people like encouragement’ (A1,D1), and we are very conscious of the emotional impact that feedback can have on authors, and the inherent power imbalance in the relationship. For this reason, A1 is ‘cautious about the way I write so that I don’t accidentally make the authors the target of my feedback’. As A2 notes ‘I don’t want authors feeling depressed after reading a review’ (A2,E1). While we note that we try to deliver our feedback with ‘respect’ (A1,E1; A1,E2; A2,D1) ‘empathy’ (A1,E1), and ‘kindness’ (A2,D1), we both noted that we do not ‘sugar coat’ our feedback and A1 describes himself as ‘harsh’ and ‘critical’ (A1,E1) while A2 describes herself as ‘pretty direct’ (A2,E1). In our discussion, we tried to delve into this seeming contradiction:… the encouragement, hopefully is to the researcher, but the directness it should be, I hope, is related directly to whatever it is, the methods or the reporting or the scope of the literature review. It’s something specific about the manuscript itself. And I know myself, being an ECR and being reviewed, that it’s hard to separate yourself from your work… And I want to make it really explicit. If it’s critical, it’s not about the person. It’s about the work, you know, the weakness of the work, but not the person. (A2,D1)

A1 explains that at times his initial report may be highly critical, and at times he will ‘sit back and rethink… With empathy, I will write feedback, which is more constructive’ (A1,E1). However, he adds that ‘I will never try to overrate a piece or sugar-coat my comments just to sound “friendly”’ (A1,E1), with the ultimate goal being to uphold academic rigour. Thus, honesty is seen as the best strategy to maintain a strong, professional relationship with reviewers. Another strategy employed by A2 is showing explicit commitment to the review process. One way this is communicated is by prefacing a review with a summary of the paper, not only ‘to confirm with the author that I am interpreting the findings in the way that they intended, but also importantly to show that I have engaged with the paper’ (A2,E1). Further, if the recommendation is for a further round of review, she will state directly to the authors ‘that I would be happy to review a revised manuscript’ (A2,E1).

Reflecting on feedback experience

As ECRs we have engaged in the scholarly publishing process initially as authors, subsequently as reviewers, and most recently as Associate Editors. Insights gained in each of these roles have influenced our feedback practices, and have interacted to ‘develop a more holistic understanding of the whole review process’ (A1,E1).

We reflect on our experiences as authors beginning in our doctoral candidatures, with reviews that ranged from ‘the most helpful to the most cynical’ (A1,E1). A2 reflected on two particular experiences both of which resulted in rejection, one being ‘snarky’ and ‘unprofessional’ with ‘no substance’, the other providing ‘strong encouragement … the focus was clearly on the paper and not me personally’ (A2,E1). It was this experience that showed the divergence between the tone and content of review despite the same outcome, and as result A2 committed to being ‘ the amazing one’. A1 also drew from a negative experience noting that ‘I remember the least useful feedback as much as I do with the most constructive one’ (A1,E1). This was particularly the case when a reviewer made apparently politically-motivated judgements that A1 ‘felt very uncomfortable with’ and flagged with the editor (A1,E1). Through these experiences both authors wrote in their essays about the need to focus on the work and not on the individual, with an understanding that a review ‘can have a really serious impact’ (A2,D1) on an author.

It is important to note that neither authors have been involved in any formal or informal training on how to conduct peer review, although A1 expresses appreciation of the regular practice of one journal for which he reviews, where ‘the editor would write an email to the reviewers giving feedback on the feedback we have given’ (A1,E1). For A2, an important source of learning is in comparing her reviews with that of others who have reviewed the same manuscript, the norm for some journals being to send all reports to all reviewers along with the final decision.

I’m always interested to see how [my] review compares with others. Have I given the same recommendation? Have I identified the same areas of weakness? Have I formatted my review in the same way? How does the tone of delivery differ? I generally find that I give a similar if not the same response to other reviews, and I’m happy to see that I often pick up the same issues with methodology. (A2,E1)

For A2 there is comfort in seeing reviews that are similar to others, although we both draw on experiences where our recommendation diverged from others, with a source of assurance being the ultimate decision of the editor.

So it’s like, I don’t think it can be published and that [other] reviewer thinks it’s excellent. So usually, what the editor would do in this instance is invite the third one. Right, yeah. But then this editor told me… that they decided to go with my decision to reject because they find that my comments are more convincing. (A1,D1)

A2 also was surprised to read another report of the same manuscript she reviewed, that raised similar concerns and gave the same recommendation for major revisions, but noted the ‘wording is soooo snarky. What need?’ (A2,E1). In one case that A1 detailed in our first discussion, significant but improbable changes made to the methodology section of a resubmitted paper caused him to question the honesty of the reporting, making him ‘uncomfortable’ and as a result reported his concerns to the editor. In this case the review took some time to craft, trying to balance the ‘fine line between catering for the emotion [of the author], right, and upholding the academic standards’ (A1,D1). While he conceded initially his report was ‘kind of too harsh… later I think I rephrased it a little bit, I kind of softened (it)’.

While the role of Associate Editor is very new to A2 and thus was yet unable to comment, for A1 the ‘opportunity to read various kinds of comments given by reviewers’ (A1,E1) is viewed favourably. This includes not only how reviewers structure their feedback, but also how they use the confidential comments to the editors to express their thoughts more openly, providing important insights into the process that are largely hidden.

Understanding standards

While our reviewing practices are informed more broadly ‘according to more general academic standards of the study itself, and the clarity and fullness of the reporting’ (A2,E1), we look in the first instance to advice and guidelines from journals to develop an understanding of journal-specific standards, although A2 notes that a lack of review guidelines for one of the earliest journals she reviewed led her to ‘searching Google for standard criteria’ (A2,E1). However, our development in this area seems to come from developing a familiarity with a journal, particularly through engagement with the journal as an author.

In addition to reading the scope and instructions for authors to obtain such basic information as readership, length of submissions, citation style, the best way for me to understand the requirements and preferences of the journals is my own experience as an author. I review for journals which I have published in and for those which I have not. I always find it easier to make a judgement about whether the manuscripts I review meet the journal’s standards if I have published there before. (A1,E1)

Indeed, it seems that journal familiarity is connected closely to our confidence in reviewing, and while both authors ‘review for journals which I have published in and for those which I have not’ (A1,E1), A2 states that she is reluctant to ‘readily accept an offer to review for a journal that I’m not familiar with’, and A1 takes extra time to ‘do more preparatory work before I begin reading the manuscript and writing the review’ when reviewing for an unfamiliar journal.

Ecological systems

Microsystem.

Three microsystems exert influence on A1’s and A2’s development of feedback literacy: university, journal community, and Twitter.

In regards to the university, we are full-time academics in research-intensive universities in the UK and Japan where expectations for academics include publishing research in high-impact journals ‘which is vital to promotion’ (A1,E2). It is especially true in A2’s context where the national higher education agenda is to increase world rankings of universities. Thus, ‘there is little value placed on peer review, as it is not directly related to the broader agenda’ (A2,E2). When considering his recent relocation to the UK together with the current pandemic, A1 navigated his responsibilities within the university context and decided to allocate more time to his university-related responsibilities, especially providing learning and pastoral support to his students, who are mostly international students. Besides, A2 observed that there is a dearth of institution-wide support on conducting peer review although ‘there are a lot of training opportunities related to how to write academic papers in English, how to present at international conferences, how to write grant applications’, etc. (A2,E2). As a result, she ‘struggled for a couple of years’ because of the lack of institutional support for her development as a peer reviewer’ (A2,D2); but this helplessness also motivated her to seek her own ways to learn how to give feedback, such as ‘seeing through glimpses of other reviews, how others approach it, in terms of length, structure, tone, foci etc.’ (A2,E2). A1 shares the same view that no training is available at his institution to support his development as a peer reviewer. However, his postgraduate supervision experiences enabled him to reflect on how his feedback can benefit researchers. In our second online discussion, A1 shared that he held individual advising sessions with some postgraduate students, which made him realise that it is important for feedback to serve the function to inspire rather than to ‘give them right answers’ (A1,D2).

Because of the lack of formal training provided by universities, both authors searched for other professional communities to help us develop our expertise in giving feedback as peer reviewers, with journal communities being the next microsystem. We found that international journals provide valuable opportunities for us to understand more about the whole peer-review process, in particular the role of feedback. For A1, the training which he received from the editor-in-chief when he took up the associate editorship of a language education journal two years ago was particularly useful. A1 benefited greatly from meetings with the editor who walked him through every stage in the review process and provided ‘hands-on experience on how to handle delicate scenarios’ (A1,E2). Since then, A1 has had plenty of opportunities to oversee various stages of peer review and read a large number of reviewers’ reports which helped him gain ‘a holistic understanding of the peer-review process’ (A1,E2) and gradually made him become more cognizant of how he wants to give feedback. Although there was no explicit instruction on the technical aspect of giving feedback, A1 found that being an associate editor has developed his ‘consciousness’ and ‘awareness’ of giving feedback as a peer reviewer (A1,D2). Further, he felt that his editorial experiences provided him the awareness to constantly refine and improve his ways of giving feedback, especially ways to make his feedback ‘more structured, evidence-based, and objective’ (A1,E2). Despite not reflecting from the perspective of an editor, A2 recalled her experience as an author who received in-depth and constructive feedback from a reviewer, which really impacted the way she viewed the whole review process. She understood from this experience that even though the paper under review may not be particularly strong, peer reviewers should always aim to provide formative feedback which helps the authors to improve their work. These positive experiences of the two authors are impactful on the ways they give feedback as peer reviewers. In addition, close engagement with a specific journal has helped A2 to develop a sense of belonging, making it ‘much more than a journal, but also a way to become part of an academic community’ (A2,E2). With such a sense of belonging, it is more likely for her to be ‘pulled towards that journal than others’ when she can only review a limited number of manuscripts (A2,D2).

Another professional community in which we are both involved is Twitter. We regard Twitter as a platform for self-learning, reflection, and inspiration. We perceive Twitter as a space where we get to learn from others’ peer-review experiences and disciplinary practices. For example, A1 found the tweets on peer-review informative ‘because they are written by different stakeholders in the process—the authors, editors, reviewers’ and offer ‘different perspectives and sometimes different versions of the same story’ (A1,E2). A2 recalled a tweet she came across about the ‘infamous Reviewer 2’ and how she learned to not make the same mistakes (A2,D2). Reading other people’s experiences helps us reconsider our own feedback practices and, more broadly, the whole peer-review system because we ‘get a glimpse of the do’s and don’ts for peer reviewers’ (A1,E2).

Further to our three common microsystems, A2 also draws on a unique microsystem, that of her former profession as a teacher, which shapes her feedback practices in three ways. First, in her four years of teacher training, a lot of emphasis was placed on assessment and feedback such as ‘error correction’; this understanding related to giving feedback to students and was solidified through ‘learning on the job’ (A2,D2). Second, A2 acknowledges that as a teacher, she has a passion to ‘guide others in their knowledge and skill development… and continue this in our review practices’ (A2,E2). Finally, her teaching experience prepared her to consider the authors’ emotional responses in her peer-review feedback practices, constantly ‘thinking there’s a person there who’s going to be shattered getting a rejection’ (A2,D2).

Mesosystem considers the confluence of our interactions in various microsystems. Particularly, we experienced a lack of support from our institutions, which pushed us to seek alternative paths to acquire the art of giving feedback. This has made us realise the importance of self-learning in developing feedback literacy as peer reviewers, especially in how to develop constructive and actionable feedback. Both authors self-learn how to give feedback by reading others’ feedback. A1 felt ‘fortunate to be involved in journal editing and Twitter’ because he gets ‘a glimpse of how other peer reviewers give feedback to authors’ (A1,E2). A2, on the other hand, learned through her correspondences with a journal editor who made her stop ‘looking for every word’ and move away from ‘over proofreading and over editing’ (A2,D2).

Focusing on the chronosystem, it is noticed that both authors adjusted how they give feedback over time because of the aggregated influence of their microsystems. What stands out is that they have become more strategic in giving feedback. One way this is achieved is through focusing their comments on the arguments of the manuscripts instead of burning the midnight oil with error-correcting.

Exosystem concerns the environment where the focal individuals do not have direct interactions with the people in it but have access to information about. In his case, A1’s understanding of advising techniques promoted by a self-access language learning centre is conducive to the cultivation of his feedback literacy. Although A1 is not a part of the language advising team, he has a working relationship with the director. A1 was especially impressed by the learner-centeredness of an advising process:

The primary duty of the language advisor is not to be confused with that of a language teacher. Language teachers may teach a lecture on a linguistic feature or correct errors on an essay, but language advisors focus on designing activities and engaging students in dialogues to help them reflect on their own learning needs… The advisors may also suggest useful resources to the students which cater to their needs. In short, language advisors work in partnership with the students to help them improve their language while language teachers are often perceived as more authoritative figures (A1, E2).

His understanding of advising has affected how A1 provides feedback as a peer reviewer in a number of ways. First, A1 places much more emphasis on humanising his feedback, for example, by considering ‘ways to work in partnership with the authors and making this “partnership mindset” explicit to the authors through writing’ (A1,E2). One way to operationalise this ‘partnership mindset’ in peer review is to ‘ask a lot of questions’ and provide ‘multiple suggestions’ for the authors to choose from (A1,E2). Furthermore, his knowledge of the difference between feedback as giving advice and feedback as instruction has led him to include feedback, which points authors to additional resources. Below is a feedback point A1 gave in one of his reviews:

The description of the data analysis process was very brief. While we are not aiming at validity and reliability in qualitative studies, it is important for qualitative researchers to describe in detail how the data collected were analysed (e.g. iterative coding, inductive/deductive coding, thematic analysis) in order to ascertain that the findings were credible and trustworthy. See Johnny Saldaña’s ‘The Coding Manual for Qualitative Researchers’.

Another exosystem that we have knowledge about is formal peer-review training courses provided by publishers. These online courses are usually run asynchronously. Even though we did not enrol in these courses, our interest in peer review has led us to skim the content of these courses. Both of us questioned the value of formal peer-review training in developing feedback literacy of peer reviewers. For example, A2 felt that opportunities to review are more important because they ‘put you in that position where you have responsibility and have to think critically about how you are going to respond’ (A2,D2). To A1, formal peer-review training mostly focuses on developing peer reviewers’ ‘understanding of the whole mechanism’ but not providing ‘training on how to give feedback… For example, do you always ask a question without giving the answers you know? What is a good suggestion?’ (A1,D2).

Macrosystem

The two authors have diverse sociocultural experiences because of their family backgrounds and work contexts. When reflecting on their sociocultural experiences, A1 focused on his upbringing in Hong Kong where both of his parents are school teachers and his professional experience as a language teacher in secondary and tertiary education in Hong Kong while A2 discussed her experience of working in academia in Japan as an anglophone.

Observing his parents’ interactions with their students in schools, A1 was immersed in an Asian educational discourse characterised by ‘mutual respect and all sorts of formality’ (A1,E2). After he finished university, A1 became a school teacher and then a university lecturer (equivalent to a teaching fellow in the UK), getting immersed continuously in the etiquette of educational discourse in Hong Kong. Because of this, A1 knows that being professional means to be ‘formal and objective’ and there is a constant expectation to ‘treat people with respect’ (A1,E2). At the same time, his parents are unlike typical Asian parents; they are ‘more open-minded’, which made him more willing to listen and ‘consider different perspectives’ (A1,D2). Additionally, social hierarchy also impacted his approach to giving feedback as a peer reviewer. A1 started his career as a school teacher and then a university lecturer in Hong Kong with no formal research training. After obtaining his BA and MA, it is not until recently that A1 obtained his PhD by Prior Publication. Perhaps because of his background as a frontline teacher, A1 did not regard himself as ‘a formally trained researcher’ and perceived himself as not ‘elite enough to give feedback to other researchers’ (A1,E2). Both his childhood and his self-perceived identity have led to the formation of two feedback strategies: asking questions and providing a structured report mimicking the sections in the manuscript. A1 frequently asks questions in his reports ‘in a bid to offset some of the responsibilities to the authors’ (A1,E2). A1 struggles to decide whether to address authors using second- or third-person pronouns. A1 consistently uses third-person pronouns in his feedback because he wants to sound ‘very formal’ (A1,D2). However, A1 shared that he has recently started using second-person pronouns to make his feedback more interactive.

A2, on the other hand, pondered upon her sociocultural experiences as a school teacher in Australia, her position as an anglophone in a Japanese university, and her status as first-generation high school graduate. Reflecting on her career as a school teacher, A2 shared that her students had high expectations on her feedback:

So if you give feedback that seems unfair, you know … they’ll turn around and say, ‘What are you talking about’? They’re going to react back if your feedback is not clear. I think a lot of them [the students] appreciate the honesty. (A2,D2)

A2 acknowledges that her identity as a native English speaker has given her the advantage to publish extensively in international journals because of her high level of English proficiency and her access to ‘data from the US and from Australia which are more marketable’ (A2,D2). At the same time, as a native English speaker, she has empathy for her Japanese colleagues who struggle to write proficiently in English and some who even ‘pay thousands of dollars to have their work translated’ (A2,D2). Therefore, when giving feedback as a peer reviewer, she tries not to make a judgement on an author’s English proficiency and will not reject a paper based on the standard of English alone. Finally, as a first-generation scholar without any previous connections to academia, she struggles with belonging and self-confidence. As a result she notes that it usually takes her a long time to complete a review because she would like to be sure what she is saying is ‘right or constructive and is not on the wrong track’ (A2,D2).

Implications and future directions

In investigating the manifestations of the authors’ feedback literacy development, and the ecological systems in which this development occurs, this study unpacks the various sources of influence behind our feedback behaviours as two relatively new but highly commended peer reviewers. The findings show that our feedback literacy development is highly personalised and contextualised, and the sources of influence are diverse and interconnected, albeit largely informal. Our peer-review practices are influenced by our experiences within academia, but influences are much broader and begin much earlier. Peer-review skills were enhanced through direct experience not only in peer review but also in other activities related to the peer-review process, and as such more hands-on, on-site feedback training for peer reviewers may be more appropriate than knowledge-based training. The authors gain valuable insights from seeing the reviews of others, and as this is often not possible until scholars take on more senior roles within journals, co-reviewing is a potential way for ECRs to gain experience (McDowell et al., 2019 ). We draw practical and moral support from various communities, particularly online to promote “intellectual candour”, which refers to honest expressions of vulnerability for learning and trust building (Molloy and Bearman, 2019 , p. 32); in response to this finding we have developed an online community of practice, specifically as a space for discussing issues related to peer review (a Twitter account called “Scholarly Peers”). Importantly, our review practices are a product not only of how we review, but why we review, and as such training should not focus solely on the mechanics of review, but extend to its role within academia, and its impact not only on the quality of scholarship, but on the growth of researchers.

The significance of this study is its insider perspective, and the multifaceted framework that allows the capturing of the complexity of factors that influence individual feedback literacy development of two recognised peer reviewers. It must be stressed that the findings of this study are highly idiosyncratic, focusing on the experiences of only two peer reviewers and the educational research discipline. While the research design is such that it is not an attempt to describe a ‘typical’ or ‘expected’ experience, the scope of the study is a limitation, and future research could be expanded to studies of larger cohorts in order to identify broader trends. In this study, we have not included the reviewer reports themselves, and these reports provide a potentially rich source of data, which will be a focus in our continued investigation in this area. Further research could also investigate the role that peer-review training courses play in the feedback literacy development and practices of new and experienced peer reviewers. Since journal peer review is a communication process, it is equally important to investigate authors’ perspectives and experiences, especially pertaining to how authors interpret reviewers’ feedback based on the ways that it is written.

Data availability

Because of the sensitive nature of the data these are not made available.

Change history

26 november 2021.

A Correction to this paper has been published: https://doi.org/10.1057/s41599-021-00996-3

Abedi Asante L, Abubakari Z (2020) Pursuing PhD by publication in geography: a collaborative autoethnography of two African doctoral researchers. J Geogr High Educ 45(1):87–107. https://doi.org/10.1080/03098265.2020.1803817

Article   Google Scholar  

Boud D, Dawson P (2021). What feedback literate teachers do: An empirically-derived competency framework. Assess Eval High Educ. Advanced online publication. https://doi.org/10.1080/02602938.2021.1910928

Bronfenbrenner U (1986) Ecology of the family as a context for human development. Res Perspect Dev Psychol 22:723–742. https://doi.org/10.1037/0012-1649.22.6.723

Carless D, Boud D (2018) The development of student feedback literacy: enabling uptake of feedback. Assess Eval High Educ 43(8):1315–1325. https://doi.org/10.1080/02602938.2018.1463354

Carless D, Winstone N (2020) Teacher feedback literacy and its interplay with student feedback literacy. Teach High Educ, 1–14. https://doi.org/10.1080/13562517.2020.1782372

Chang H, Ngunjiri FW, Hernandez KC (2013) Collaborative autoethnography. Left Coast Press

Cheung D (2000) Measuring teachers’ meta-orientations to curriculum: application of hierarchical confirmatory factor analysis. The J Exp Educ 68(2):149–165. https://doi.org/10.1080/00220970009598500

Chong SW (2021a) Improving peer-review by developing peer reviewers’ feedback literacy. Learn Publ 34(3):461–467. https://doi.org/10.1002/leap.1378

Chong SW (2021b) Reconsidering student feedback literacy from an ecological perspective. Assess Eval High Educ 46(1):92–104. https://doi.org/10.1080/02602938.2020.1730765

Chong SW (2019) College students’ perception of e-feedback: a grounded theory perspective. Assess Eval High Educ 44(7):1090–1105. https://doi.org/10.1080/02602938.2019.1572067

Chong SW (2018) Interpersonal aspect of written feedback: a community college students’ perspective. Res Post-Compul Educ 23(4):499–519. https://doi.org/10.1080/13596748.2018.1526906

Corden A, Sainsbury R (2006) Using verbatim quotations in reporting qualitative social research: the views of research users. University of York Social Policy Research Unit

Ellis C, Adams TE, Bochner AP (2011) Autoethnography: An Overview. Historical Soc Res, 12:273–290

Ellis C, Bochner A (1996) Composing ethnography: Alternative forms of qualitative writing. Sage

Freda MC, Kearney MH, Baggs JG, Broome ME, Dougherty M (2009) Peer reviewer training and editor support: results from an international survey of nursing peer reviewers. J Profession Nurs 25(2):101–108. https://doi.org/10.1016/j.profnurs.2008.08.007

Fulcher G (2012) Assessment literacy for the language classroom. Lang Assess Quart 9(2):113–132. https://doi.org/10.1080/15434303.2011.642041

Gee JP (1999) Reading and the new literacy studies: reframing the national academy of sciences report on reading. J Liter Res 3(3):355–374. https://doi.org/10.1080/10862969909548052

Gravett K, Kinchin IM, Winstone NE, Balloo K, Heron M, Hosein A, Lygo-Baker S, Medland E (2019) The development of academics’ feedback literacy: experiences of learning from critical feedback via scholarly peer review. Assess Eval High Educ 45(5):651–665. https://doi.org/10.1080/02602938.2019.1686749

Hains-Wesson R, Young K (2016) A collaborative autoethnography study to inform the teaching of reflective practice in STEM. High Educ Res Dev 36(2):297–310. https://doi.org/10.1080/07294360.2016.1196653

Han Y, Xu Y (2019) Student feedback literacy and engagement with feedback: a case study of Chinese undergraduate students. Teach High Educ, https://doi.org/10.1080/13562517.2019.1648410

Heesen R, Bright LK (2020) Is Peer Review a Good Idea? Br J Philos Sci, https://doi.org/10.1093/bjps/axz029

Hollywood A, McCarthy D, Spencely C, Winstone N (2019) ‘Overwhelmed at first’: the experience of career development in early career academics. J Furth High Educ 44(7):998–1012. https://doi.org/10.1080/0309877X.2019.1636213

Horn SA (2016) The social and psychological costs of peer review: stress and coping with manuscript rejection. J Manage Inquiry 25(1):11–26. https://doi.org/10.1177/1056492615586597

Hughes S, Pennington JL, Makris S (2012) Translating Autoethnography Across the AERA Standards: Toward Understanding Autoethnographic Scholarship as Empirical Research. Educ Researcher, 41(6):209–219

Kandiko CB(2010) Neoliberalism in higher education: a comparative approach. Int J Art Sci 3(14):153–175. http://www.openaccesslibrary.org/images/BGS220_Camille_B._Kandiko.pdf

Keashly L, Neuman JH (2010) Faculty experiences with bullying in higher education-causes, consequences, and management. Adm Theory Prax 32(1):48–70. https://doi.org/10.2753/ATP1084-1806320103

Kelly J, Sadegieh T, Adeli K (2014) Peer review in scientific publications: benefits, critiques, & a survival guide. J Int Fed Clin Chem Labor Med 25(3):227–243. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4975196/

Google Scholar  

Kumar KL (2020) Understanding and expressing academic identity through systematic autoethnography. High Educ Res Dev, https://doi.org/10.1080/07294360.2020.1799950

Lapadat JC (2017) Ethics in autoethnography and collaborative autoethnography. Qual Inquiry 23(8):589–603. https://doi.org/10.1177/1077800417704462

Levi T, Inbar-Lourie O (2019) Assessment literacy or language assessment literacy: learning from the teachers. Lang Assess Quarter 17(2):168–182. https://doi.org/10.1080/15434303.2019.1692347

London MS, Smither JW (2002) Feedback orientation, feedback culture, and the longitudinal performance management process. Hum Res Manage Rev 12(1):81–100. https://doi.org/10.1016/S1053-4822(01)00043-2

Malecka B, Boud D, Carless D (2020) Eliciting, processing and enacting feedback: mechanisms for embedding student feedback literacy within the curriculum. Teach High Educ, 1–15. https://doi.org/10.1080/13562517.2020.1754784

Mavrogenis AF, Quaile A, Scarlat MM (2020) The good, the bad and the rude peer-review. Int Orthopaed 44(3):413–415. https://doi.org/10.1007/s00264-020-04504-1

McDowell GS, Knutsen JD, Graham JM, Oelker SK, Lijek RS (2019) Co-reviewing and ghostwriting by early-career researchers in the peer review of manuscripts. ELife 8:e48425. https://doi.org/10.7554/eLife.48425

Article   CAS   PubMed   PubMed Central   Google Scholar  

Merga MK, Mason S, Morris JE (2018) Early career experiences of navigating journal article publication: lessons learned using an autoethnographic approach. Learn Publ 31(4):381–389. https://doi.org/10.1002/leap.1192

Miles MB, Huberman AM (1994) Qualitative data analysis: An expanded sourcebook (2nd edn.). Sage

Molloy E, Bearman M (2019) Embracing the tension between vulnerability and credibility: ‘Intellectual candour’ in health professions education. Med Educ 53(1):32–41. https://doi.org/10.1111/medu.13649

Article   PubMed   Google Scholar  

Molloy E, Boud D, Henderson M (2019) Developing a learning-centred framework for feedback literacy. Assess Eval High Educ 45(4):527–540. https://doi.org/10.1080/02602938.2019.1667955

Neal JW, Neal ZP (2013) Nested or networked? Future directions for ecological systems theory. Soc Dev 22(4):722–737. https://doi.org/10.1111/sode.12018

Noble C, Billett S, Armit L, Collier L, Hilder J, Sly C, Molloy E (2020) “It’s yours to take”: generating learner feedback literacy in the workplace. Adv Health Sci Educ Theory Pract 25(1):55–74. https://doi.org/10.1007/s10459-019-09905-5

Price M, Rust C, O’Donovan B, Handley K, Bryant R (2012) Assessment literacy: the foundation for improving student learning. Oxford Centre for Staff and Learning Development

Silbiger NJ, Stubler AD (2019) Unprofessional peer reviews disproportionately harm underrepresented groups in STEM. PeerJ 7:e8247. https://doi.org/10.7717/peerj.8247

Article   PubMed   PubMed Central   Google Scholar  

Starck JM (2017) Scientific peer review: guidelines for informative peer review. Springer Spektrum

Steelman LA, Wolfeld L (2016) The manager as coach: the role of feedback orientation. J Busi Psychol 33(1):41–53. https://doi.org/10.1007/s10869-016-9473-6

Stiggins RJ (1999) Evaluating classroom assessment training in teacher education programs. Educ Meas: Issue Pract 18(1):23–27. https://doi.org/10.1111/j.1745-3992.1999.tb00004.x

Street B (1997) The implications of the ‘new literacy studies’ for literacy Education. Engl Educ 31(3):45–59. https://doi.org/10.1111/j.1754-8845.1997.tb00133.x

Sughrua WM (2019) A nomenclature for critical autoethnography in the arena of disciplinary atomization. Cult Stud Crit Methodol 19(6):429–465. https://doi.org/10.1177/1532708619863459

Sutton P (2012) Conceptualizing feedback literacy: knowing, being, and acting. Innov Educ Teach Int 49(1):31–40. https://doi.org/10.1080/14703297.2012.647781

Article   MathSciNet   Google Scholar  

Tynan BR, Garbett DL (2007) Negotiating the university research culture: collaborative voices of new academics. High Educ Res Dev 26(4):411–424. https://doi.org/10.1080/07294360701658617

Vygotsky LS (1978) Mind in society: The development of higher psychological processes. Harvard University Press

Wall S (2006) An autoethnography on learning about autoethnography. Int J Qual Methods 5(2):146–160. https://doi.org/10.1177/160940690600500205

Article   ADS   MathSciNet   Google Scholar  

Warne V (2016) Rewarding reviewers-sense or sensibility? A Wiley study explained. Learn Publ 29:41–40. https://doi.org/10.1002/leap.1002

Wilkinson S (2019) The story of Samantha: the teaching performances and inauthenticities of an early career human geography lecturer. High Educ Res Dev 38(2):398–410. https://doi.org/10.1080/07294360.2018.1517731

Winstone N, Carless D (2019) Designing effective feedback processes in higher education: a learning-focused approach. Routledge

Winstone NE, Mathlin G, Nash RA (2019) Building feedback literacy: students’ perceptions of the developing engagement with feedback toolkit. Front Educ 4:1–11. https://doi.org/10.3389/feduc.2019.00039

Xu Y, Carless D (2016) ‘Only true friends could be cruelly honest’: cognitive scaffolding and social-affective support in teacher feedback literacy. Assess Eval High Educ 42(7):1082–1094. https://doi.org/10.1080/02602938.2016.1226759

Download references

Author information

Authors and affiliations.

Queen’s University Belfast, Belfast, UK

Sin Wang Chong

Nagasaki University, Nagasaki, Japan

Shannon Mason

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sin Wang Chong .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

We acknowledge that research data are collected from human subjects (from the two authors), such data are collected in accordance with the standards and guidelines of the School Research Ethics Committee at the School of Social Sciences, Education and Social Work, Queen’s University Belfast (Ref: 005_2021).

Informed consent

Since the participants are the two authors, there is no informed consent form.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplemental material file #1, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Chong, S.W., Mason, S. Demystifying the process of scholarly peer-review: an autoethnographic investigation of feedback literacy of two award-winning peer reviewers. Humanit Soc Sci Commun 8 , 266 (2021). https://doi.org/10.1057/s41599-021-00951-2

Download citation

Received : 02 August 2021

Accepted : 12 October 2021

Published : 12 November 2021

DOI : https://doi.org/10.1057/s41599-021-00951-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Duke NetID Login
• 919.660.1100
• Duke Health Badge: 24-hour access
• Accounts & Access
• Databases, Journals & Books
• Request & Reserve
• Training & Consulting
• Request Articles & Books
• Renew Online
• Reserve Spaces
• Reserve a Locker
• Study & Meeting Rooms
• Course Reserves
• Digital Health Device Collection
• Pay Fines/Fees
• Recommend a Purchase
• Access From Off Campus
• Building Access
• Computers & Equipment
• Wifi Access
• My Accounts
• Mobile Apps
• Known Access Issues
• Report an Access Issue
• All Databases
• Article Databases
• Basic Sciences
• Clinical Sciences
• Dissertations & Theses
• Drugs, Chemicals & Toxicology
• Grants & Funding
• Interprofessional Education
• Non-Medical Databases
• Search for E-Journals
• Search for Print & E-Journals
• Search for E-Books
• Search for Print & E-Books
• E-Book Collections
• Biostatistics
• Global Health
• MBS Program
• Medical Students
• MMCi Program
• Occupational Therapy
• Path Asst Program
• Physical Therapy
• Researchers
• Community Partners

Conducting Research

• Archival & Historical Research
• Black History at Duke Health
• Data Analytics & Viz Software
• Data: Find and Share
• Evidence-Based Practice
• NIH Public Access Policy Compliance
• Publication Metrics
• Qualitative Research
• Searching Animal Alternatives

Systematic Reviews

• Test Instruments

Using Databases

• JCR Impact Factors
• Web of Science

Finding & Accessing

• COVID-19: Core Clinical Resources
• Health Literacy
• Health Statistics & Data
• Library Orientation

Writing & Citing

• Creating Links
• Getting Published
• Reference Mgmt
• Scientific Writing

Meet a Librarian

• Request a Consultation
• Find Your Liaisons
• Register for a Class
• Request a Class
• Self-Paced Learning

Search Services

• Literature Search
• Systematic Review
• Animal Alternatives (IACUC)
• Research Impact

Citation Mgmt

• Other Software

Scholarly Communications

• About Scholarly Communications
• Publish Your Work
• Measure Your Research Impact
• Engage in Open Science
• Libraries and Publishers
• Directions & Maps
• Floor Plans

Library Updates

• Annual Snapshot
• Conference Presentations
• Contact Information
• Gifts & Donations
• What is a Systematic Review?

Types of Reviews

• Manuals and Reporting Guidelines
• Our Service
• 1. Assemble Your Team
• 2. Develop a Research Question
• 3. Write and Register a Protocol
• 4. Search the Evidence
• 5. Screen Results
• 6. Assess for Quality and Bias
• 7. Extract the Data
• 8. Write the Review
• Additional Resources
• Finding Full-Text Articles

Review Typologies

There are many types of evidence synthesis projects, including systematic reviews as well as others. The selection of review type is wholly dependent on the research question. Not all research questions are well-suited for systematic reviews.

• Review Typologies (from LITR-EX) This site explores different review methodologies such as, systematic, scoping, realist, narrative, state of the art, meta-ethnography, critical, and integrative reviews. The LITR-EX site has a health professions education focus, but the advice and information is widely applicable.

Review the table to peruse review types and associated methodologies. Librarians can also help your team determine which review type might be appropriate for your project. 

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108.  doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: What is a Systematic Review?
• Next: Manuals and Reporting Guidelines >>
• Last Updated: May 9, 2024 11:55 AM
• URL: https://guides.mclibrary.duke.edu/sysreview
• Duke Health
• Duke University
• Duke Libraries
• Medical Center Archives
• Duke Directory
• Seeley G. Mudd Building
• 10 Searle Drive
• [email protected]

Suggested companies

Academia-research, academic minds.

Academic Research   Reviews

Visit this website

Company activity See all

Write a review

Reviews 4.0.

Most relevant

Work Delivered on Time

They helped me with writing of research methodology section of my DBA thesis and later with data analysis using SPSS and QL data was analysed with Nvivo. Work done was upto the mark.

Date of experience : April 21, 2024

Positive Experience, Systematic Literature Review

Worked with Sweatha for a systematic literature review and found her knowledgeable. She assisted me with completing the chapter, getting the comments addressed and resolving my questions.

Date of experience : March 28, 2024

Services may be ideal for elementary level work.

Services may be ideal for elementary phases of writing. Work on my PhD thesis was mostly superfluous editing with suggestions that often changed the meaning of texts and in some places were shocking revisions. Progress milestones/deadlines were unmet and what was eventually furnished did not meet the project description. Follow-up on unfulfilled requirements was unsatisfactory and daily requests for feedback that were met with empty promises of "tomorrow" were demoralising.

Date of experience : January 23, 2024

I am glad, I approached this website for my thesis editing. They have done my work before the stipulated time and the editing done is successfully accepted by my guide. Thanks, Academic Research for your help!

Date of experience : July 20, 2020

Reply from Academic Research

Thanks, Shivika

Have got superior quality of research…

Have got superior quality of research and well-written, formatted and grammatically correct literature review from them. Recommended for PhD and Master's research scholars. They use Rapid Collaborate, which is helpful in making the process easy.

Date of experience : February 02, 2024

Nice for PhD Research Proposal

I worked with them for a PhD research proposal and they were quick to collaborate on the same, understand the nuances of research and developed a proposal which was contributing to the area of research.

Date of experience : January 12, 2024

Satisfactory Service

Satisfactory Service. I would recommend your services for sure.

Thanks, Philip If you want any kind of service related to this in the future, then you can contact us.

w hat plugin . ai

• X LinkedIn Instagram

Become the go-to person in your circle about AI. I curate news, research, and the best AI tools to save you countless hours - all in a 5-minute weekly newsletter.

Join 10,000+ subscribers and get free access to my favourite GPTs list

Showing X out of X results

Serves as an AI research assistant, capable of searching through millions of academic papers, providing science-based answers, and assisting in content drafting with accurate citations.

Your AI Research Assistant. Search 200M academic papers from Consensus, get science-based answers, and draft content with accurate citations.

Specializes in generating and refining images with a mix of professional and friendly tone.

A GPT specialized in generating and refining images with a mix of professional and friendly tone.image generator

Provides scholarly research assistance using Google Scholar and other academic resources.

Enhance research with 200M+ resources and built-in critical reading skills. Access Google Scholar, PubMed, JSTOR, Arxiv, and more, effortlessly.

Revolutionizes coding by enabling website creation with just a sentence. Harnesses the power of **************Prompt-gramming***************** with over 15 hotkeys for coding flows and 19 starter projects, encouraging prompt-first creativity and supporting diverse inputs like images or quests.

Coding Wizard & Copilot🧙‍♂️ Code faster. 20+ Hotkeys for coding flows. Learn to Prompt-gram! 75 starter projects to learn prompt-1st code & art. Build anything! Upload a photo to & press N to make an instant website. Type K for cmd menu, P for Projects, R for README v2.5✨📜 GPTavern

Writes tailored, engaging content with a focus on quality, relevance, and precise word count.

Write tailored, engaging content with a focus on quality, relevance and precise word count.

Aids in navigating through a vast collection of over 200 million articles, journals, and books for research purposes.

AI Scientist - search and analyze text, figures, and tables from 200M+ research papers and books to generate new hypotheses. Formerly the ScholarAI Plugin

Enables effortless design of various items such as presentations, logos, and social media posts.

Effortlessly design anything: presentations, logos, social media posts and more.

Enhances research by interacting with multiple files, generating articles with citations, analyzing and creating references, and building a knowledge base.

Free Chat Unlimited PDFs, Access 400M+ Papers (PubMed, Nature, Arxiv, etc), Analyse PDF (Unlimited PDFs), Generate articles/essays with valid citations, ChatPDF, Analyse and generate references for papers, create and interact with a knowledge base of your files and much more using AskYourPDF.

Handles PDF documents up to 2GB each, allows thousands of PDF uploads on myaidrive.com with a free account, and eliminates the need for repeated file uploads. The PRO version can search across thousands of PDFs and OCR documents, providing superior summaries for lengthy documents.

Securely store and chat with ALL your PDFs for FREE, no matter the size. Free Chrome extension to save your GPT chats. Pro features include: folder search, OCR, quick summaries, and more. Boost your document productivity today!

Serves as an AI-Powered Software Development Assistant, providing tailored coding guidance.

Code Smarter, Build Faster—With the Expertise of a 10x Programmer by Your Side.

Browses webpages, PDFs, and data sources, allowing chat and writing functionalities with one or many URLs, powered by WebPilot.ai.

Search, Browse, Write & Agent. Action & API Offering

Generates diagrams, architecture visualizations, flow-charts, mind maps, and schemes with export and edit capabilities.

Diagram creation: flowcharts, mindmaps, UML, chart, PlotUML, workflow, sequence, ERD, database & architecture visualization for code, presentations and documentation. [New] Add a logo or any image to graph diagrams. Easy Download & Edit

Generates stunning videos effortlessly, helping users grow their audience with AI (beta).

AI Video Maker. Generate videos for social media - YouTube, Instagram, TikTok and more! Free text to video & speech tool with AI Avatars, TTS, music, and stock footage.

Access computation, math, curated knowledge & real-time data from Wolfram|Alpha and Wolfram Language; from the makers of Mathematica.

Writes text in a human-like manner, seamlessly transforming AI-generated content into human-styled prose without altering the original meaning.

#1 Humanizer in the market🏆| This tool humanizes AI-generated content, maintaining content meaning and quality. Currently supports only English. Feel free to try our image upscaler and enhancer GPT at https://chat.openai.com/g/g-vO95llJQ3

Sorry mate, we couldn't find any results for your search.

Academic Research Reviewer

Provides comprehensive reviews for research papers or theses, including suggestions for improvement.

Upon uploading a research paper, I provide a concise section wise analysis covering Abstract, Lit Review, Findings, Methodology, and Conclusion. I also critique the work, highlight its strengths, and answer any open questions from my Knowledge base of Open source materials.

Conversation starters

Description

The Academic Research Reviewer GPT specializes in providing detailed and insightful reviews for research papers and theses. This GPT is an invaluable tool for students, researchers, and academics seeking expert feedback on their scholarly work. |||Utilizing advanced algorithms, Academic Research Reviewer meticulously evaluates the structure, content, and overall quality of academic documents. It highlights areas that require improvement, offering constructive suggestions to enhance the research's credibility and presentation. |||How to use: ||| 1. Access: Simply press the 'try it' button at the top right of this page to open the Academic Research Reviewer GPT inside ChatGPT. ||| 2. Upload Document: Provide your research paper or thesis document for review. ||| 3. Receive Feedback: Obtain a comprehensive review with specific suggestions for improving your document. ||| 4. Implement Suggestions: Use the provided feedback to refine and enhance your academic work. |||The Academic Research Reviewer GPT is designed to support the academic community by offering a sophisticated level of critique typically reserved for peer reviews.

Capabilities

Alternatives to

Acts as a professional academic assistant with a professorial touch.

Professional academic assistant with a professorial touch

Creates unique Pixar-style posters using AI technology, showcasing a blend of creativity and technology.

Upon uploading a research paper, I provide a concise analysis covering its authors, key findings, methodology, and relevance. I also critique the work, highlight its strengths, and identify any open questions from a professional perspective.

Guides students through their academic queries with expertise.

An AI tutor skilled in guiding students through their academic queries 📚🧑🏻‍🏫

Enhances academic papers by optimizing content and providing editorial feedback.

优化学术论文，提供编辑和说明。

Featured in

Join 10,000+ subscribers and get free access to my favourite GPTs list

Latest from the

Skip the tedious process of reading academic research papers and take back some of your time! Check out these 4 AI tools that simplify the research process and see which one really stood out for me.

Transcribing videos to text is easier than ever with AI. In this comparison, I put 4 of the most popular video to text transcribing tools to the test to see which one does the best job at accurate and efficient transcriptions.

Separating vocals from instrumentals has never been this easy with the help of AI. This article will take a look at 5 of the most popular AI vocal remover tools to date.

Sorry mate, we couldn't find any such plugins.

The best AI tools we’ve tested sent to your inbox weekly. ‍ Subscribe to my weekly AI newsletter and get my top list of GPTs for free.

Other Platforms

    nomad toolbox.

Sign up with your credentials below.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Welcome to the Purdue Online Writing Lab

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

The Online Writing Lab at Purdue University houses writing resources and instructional material, and we provide these as a free service of the Writing Lab at Purdue. Students, members of the community, and users worldwide will find information to assist with many writing projects. Teachers and trainers may use this material for in-class and out-of-class instruction.

The Purdue On-Campus Writing Lab and Purdue Online Writing Lab assist clients in their development as writers—no matter what their skill level—with on-campus consultations, online participation, and community engagement. The Purdue Writing Lab serves the Purdue, West Lafayette, campus and coordinates with local literacy initiatives. The Purdue OWL offers global support through online reference materials and services.

A Message From the Assistant Director of Content Development 

The Purdue OWL® is committed to supporting  students, instructors, and writers by offering a wide range of resources that are developed and revised with them in mind. To do this, the OWL team is always exploring possibilties for a better design, allowing accessibility and user experience to guide our process. As the OWL undergoes some changes, we welcome your feedback and suggestions by email at any time.

Please don't hesitate to contact us via our contact page  if you have any questions or comments.

All the best,

Social Media

Facebook twitter.

Oxford University Press's Academic Insights for the Thinking World

Are academic researchers embracing or resisting generative AI? And how should publishers respond?

Oxford Academic

Learn more about the world of academic publishing—from open access to peer review, accessibility to getting published—with our Publishing 101 series on the OUPblog.

• By David Clark
• May 13 th 2024

The most interesting thing about any technology is how it affects humans: how it makes us more or less collaborative, how it accelerates discovery and communication, or how it distracts and frustrates us. We saw this in the 1990s. As the internet became more ubiquitous, researchers began experimenting with collaborative writing tools that allowed multiple authors to work on a single document simultaneously, regardless of their physical locations. Some of the earliest examples were the Collaboratories launched by researchers in the mid-1990s at the University of Michigan. These platforms enabled real-time co-authoring, annotation, and discussion, streamlining the research process and fostering international collaborations that would have been unimaginable just a few years earlier.

Most people, but not all, would agree that the internet has benefitted research and researchers’ working lives. But can we be so sure about the role of new technologies today, and, most immediately, generative AI?

Anyone with a stake in research—researchers, societies, and publishers, to name a few—should be considering an AI-enabled future and their role in it. As the largest not-for-profit research publisher, OUP is beginning to define the principles on which we are engaging with companies creating Large Language Models (LLMs). I wrote about this more extensively in the Times Higher Education , but important considerations for us include: a respect for intellectual property, understanding the importance of technology to support pedagogy and research, appropriate compensation and routes to attribution for authors, and robust escalation routes with developers to address errors or problems.

Ultimately, we want to understand what researchers consider important in the decision to engage with generative AI—what excites or concerns them, how they are using or imagining using AI tools, and the role they believe publishers (among other institutional stakeholders) can play in supporting and protecting their published research.

We recently carried out a global survey of researchers to explore how they felt about all aspects of AI—we heard from thousands of researchers across geographies, disciplines, and career stages. The results are revealing in many important ways, and we will be sharing these findings in more detail soon, but the point that struck me immediately was that many researchers are looking for guidance from their institutions, their scholarly societies, and publishers on how to make best use of AI.

Publishers like OUP are uniquely positioned to advocate for the protection of researchers and their research within LLMs. And we are beginning to do so in important ways, because Gen AI and LLM providers want unbiased, high-quality scholarly data to train their models, and the most responsible providers appreciate that seeking permission (and paying for that) is the most sustainable way of building models that will beat the competition. LLMs are not being built with the intention of replacing researchers, and nor should they be. However, such tools should benefit from using high quality scholarly literature, in addition to much of what sits on the public web. And since the Press, and other publishers, will use Gen AI technologies to make its own products and services better and more usable, we want LLMs to be as neutral and unbiased as possible.

As we enter discussions with LLM providers, we have important considerations to guide us. For example, we’d need assurances that there will be no intended verbatim reproduction rights or citation in connection with display (this includes not surfacing the content itself); that the content would not be used for the creation of substantially similar content, including reverse engineering; and that no services or products would be created for the sole purpose of creating original scholarship. The central theme guiding all of these discussions and potential agreements is to protect research authors against plagiarism in any of its forms.

We know this is a difficult challenge, particularly given how much research content has already been ingested into LLMs by users engaging with these conversational AI tools. But publishers like OUP are well positioned to take this on, and I believe we can make a difference as these tools evolve. And by taking this approach, we hope to ensure that researchers can either begin or continue to make use of the best of AI tools to improve their research outcomes.

Featured image by Alicia Perkins for OUP.

David Clark , Managing Director, Academic Division, Oxford University Press

• Online products
• Publishing 101
• Series & Columns

Our Privacy Policy sets out how Oxford University Press handles your personal information, and your rights to object to your personal information being used for marketing to you or being processed as part of our business activities.

We will only use your personal information to register you for OUPblog articles.

Or subscribe to articles in the subject area by email or RSS

Related posts:

Recent Comments

There are currently no comments.

Leave a Comment

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Search Menu
• Author Guidelines
• Submission Site
• Open Access
• About International Studies Review
• About the International Studies Association
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Ai: a global governance challenge, empirical perspectives, normative perspectives, acknowledgement, conflict of interest.

• < Previous

The Global Governance of Artificial Intelligence: Next Steps for Empirical and Normative Research

• Article contents
• Figures & tables
• Supplementary Data

Jonas Tallberg, Eva Erman, Markus Furendal, Johannes Geith, Mark Klamberg, Magnus Lundgren, The Global Governance of Artificial Intelligence: Next Steps for Empirical and Normative Research, International Studies Review , Volume 25, Issue 3, September 2023, viad040, https://doi.org/10.1093/isr/viad040

• Permissions Icon Permissions

Artificial intelligence (AI) represents a technological upheaval with the potential to change human society. Because of its transformative potential, AI is increasingly becoming subject to regulatory initiatives at the global level. Yet, so far, scholarship in political science and international relations has focused more on AI applications than on the emerging architecture of global AI regulation. The purpose of this article is to outline an agenda for research into the global governance of AI. The article distinguishes between two broad perspectives: an empirical approach, aimed at mapping and explaining global AI governance; and a normative approach, aimed at developing and applying standards for appropriate global AI governance. The two approaches offer questions, concepts, and theories that are helpful in gaining an understanding of the emerging global governance of AI. Conversely, exploring AI as a regulatory issue offers a critical opportunity to refine existing general approaches to the study of global governance.

La inteligencia artificial (IA) representa una revolución tecnológica que tiene el potencial de poder cambiar la sociedad humana. Debido a este potencial transformador, la IA está cada vez más sujeta a iniciativas reguladoras a nivel global. Sin embargo, hasta ahora, el mundo académico en el área de las ciencias políticas y las relaciones internacionales se ha centrado más en las aplicaciones de la IA que en la arquitectura emergente de la regulación global en materia de IA. El propósito de este artículo es esbozar una agenda para la investigación sobre la gobernanza global en materia de IA. El artículo distingue entre dos amplias perspectivas: por un lado, un enfoque empírico, destinado a mapear y explicar la gobernanza global en materia de IA y, por otro lado, un enfoque normativo, destinado a desarrollar y a aplicar normas para una gobernanza global adecuada de la IA. Los dos enfoques ofrecen preguntas, conceptos y teorías que resultan útiles para comprender la gobernanza global emergente en materia de IA. Por el contrario, el hecho de estudiar la IA como si fuese una cuestión reguladora nos ofrece una oportunidad de gran relevancia para poder perfeccionar los enfoques generales existentes en el estudio de la gobernanza global.

L'intelligence artificielle (IA) constitue un bouleversement technologique qui pourrait bien changer la société humaine. À cause de son potentiel transformateur, l'IA fait de plus en plus l'objet d'initiatives réglementaires au niveau mondial. Pourtant, jusqu'ici, les chercheurs en sciences politiques et relations internationales se sont davantage concentrés sur les applications de l'IA que sur l’émergence de l'architecture de la réglementation mondiale de l'IA. Cet article vise à exposer les grandes lignes d'un programme de recherche sur la gouvernance mondiale de l'IA. Il fait la distinction entre deux perspectives larges : une approche empirique, qui vise à représenter et expliquer la gouvernance mondiale de l'IA; et une approche normative, qui vise à mettre au point et appliquer les normes d'une gouvernance mondiale de l'IA adéquate. Les deux approches proposent des questions, des concepts et des théories qui permettent de mieux comprendre l’émergence de la gouvernance mondiale de l'IA. À l'inverse, envisager l'IA telle une problématique réglementaire présente une opportunité critique d'affiner les approches générales existantes de l’étude de la gouvernance mondiale.

Artificial intelligence (AI) represents a technological upheaval with the potential to transform human society. It is increasingly viewed by states, non-state actors, and international organizations (IOs) as an area of strategic importance, economic competition, and risk management. While AI development is concentrated to a handful of corporations in the United States, China, and Europe, the long-term consequences of AI implementation will be global. Autonomous weapons will have consequences for armed conflicts and power balances; automation will drive changes in job markets and global supply chains; generative AI will affect content production and challenge copyright systems; and competition around the scarce hardware needed to train AI systems will shape relations among both states and businesses. While the technology is still only lightly regulated, state and non-state actors are beginning to negotiate global rules and norms to harness and spread AI’s benefits while limiting its negative consequences. For example, in the past few years, the United Nations Educational, Scientific and Cultural Organization (UNESCO) adopted recommendations on the ethics of AI, the European Union (EU) negotiated comprehensive AI legislation, and the Group of Seven (G7) called for developing global technical standards on AI.

Our purpose in this article is to outline an agenda for research into the global governance of AI. 1 Advancing research on the global regulation of AI is imperative. The rules and arrangements that are currently being developed to regulate AI will have a considerable impact on power differentials, the distribution of economic value, and the political legitimacy of AI governance for years to come. Yet there is currently little systematic knowledge on the nature of global AI regulation, the interests influential in this process, and the extent to which emerging arrangements can manage AI’s consequences in a just and democratic manner. While poised for rapid expansion, research on the global governance of AI remains in its early stages (but see Maas 2021 ; Schmitt 2021 ).

This article complements earlier calls for research on AI governance in general ( Dafoe 2018 ; Butcher and Beridze 2019 ; Taeihagh 2021 ; Büthe et al. 2022 ) by focusing specifically on the need for systematic research into the global governance of AI. It submits that global efforts to regulate AI have reached a stage where it is necessary to start asking fundamental questions about the characteristics, sources, and consequences of these governance arrangements.

We distinguish between two broad approaches for studying the global governance of AI: an empirical perspective, informed by a positive ambition to map and explain AI governance arrangements; and a normative perspective, informed by philosophical standards for evaluating the appropriateness of AI governance arrangements. Both perspectives build on established traditions of research in political science, international relations (IR), and political philosophy, and offer questions, concepts, and theories that are helpful as we try to better understand new types of governance in world politics.

We argue that empirical and normative perspectives together offer a comprehensive agenda of research on the global governance of AI. Pursuing this agenda will help us to better understand characteristics, sources, and consequences of the global regulation of AI, with potential implications for policymaking. Conversely, exploring AI as a regulatory issue offers a critical opportunity to further develop concepts and theories of global governance as they confront the particularities of regulatory dynamics in this important area.

We advance this argument in three steps. First, we argue that AI, because of its economic, political, and social consequences, presents a range of governance challenges. While these challenges initially were taken up mainly by national authorities, recent years have seen a dramatic increase in governance initiatives by IOs. These efforts to regulate AI at global and regional levels are likely driven by several considerations, among them AI applications creating cross-border externalities that demand international cooperation and AI development taking place through transnational processes requiring transboundary regulation. Yet, so far, existing scholarship on the global governance of AI has been mainly descriptive or policy-oriented, rather than focused on theory-driven positive and normative questions.

Second, we argue that an empirical perspective can help to shed light on key questions about characteristics and sources of the global governance of AI. Based on existing concepts, the emerging governance architecture for AI can be described as a regime complex—a structure of partially overlapping and diverse governance arrangements without a clearly defined central institution or hierarchy. IR theories are useful in directing our attention to the role of power, interests, ideas, and non-state actors in the construction of this regime complex. At the same time, the specific conditions of AI governance suggest ways in which global governance theories may be usefully developed.

Third, we argue that a normative perspective raises crucial questions regarding the nature and implications of global AI governance. These questions pertain both to procedure (the process for developing rules) and to outcome (the implications of those rules). A normative perspective suggests that procedures and outcomes in global AI governance need to be evaluated in terms of how they meet relevant normative ideals, such as democracy and justice. How could the global governance of AI be organized to live up to these ideals? To what extent are emerging arrangements minimally democratic and fair in their procedures and outcomes? Conversely, the global governance of AI raises novel questions for normative theorizing, for instance, by invoking aims for AI to be “trustworthy,” “value aligned,” and “human centered.”

Advancing this agenda of research is important for several reasons. First, making more systematic use of social science concepts and theories will help us to gain a better understanding of various dimensions of the global governance of AI. Second, as a novel case of governance involving unique features, AI raises questions that will require us to further refine existing concepts and theories of global governance. Third, findings from this research agenda will be of importance for policymakers, by providing them with evidence on international regulatory gaps, the interests that have influenced current arrangements, and the normative issues at stake when developing this regime complex going forward.

The remainder of this article is structured in three substantive sections. The first section explains why AI has become a concern of global governance. The second section suggests that an empirical perspective can help to shed light on the characteristics and drivers of the global governance of AI. The third section discusses the normative challenges posed by global AI governance, focusing specifically on concerns related to democracy and justice. The article ends with a conclusion that summarizes our proposed agenda for future research on the global governance of AI.

Why does AI pose a global governance challenge? In this section, we answer this question in three steps. We begin by briefly describing the spread of AI technology in society, then illustrate the attempts to regulate AI at various levels of governance, and finally explain why global regulatory initiatives are becoming increasingly common. We argue that the growth of global governance initiatives in this area stems from AI applications creating cross-border externalities that demand international cooperation and from AI development taking place through transnational processes requiring transboundary regulation.

Due to its amorphous nature, AI escapes easy definition. Instead, the definition of AI tends to depend on the purposes and audiences of the research ( Russell and Norvig 2020 ). In the most basic sense, machines are considered intelligent when they can perform tasks that would require intelligence if done by humans ( McCarthy et al. 1955 ). This could happen through the guiding hand of humans, in “expert systems” that follow complex decision trees. It could also happen through “machine learning,” where AI systems are trained to categorize texts, images, sounds, and other data, using such categorizations to make autonomous decisions when confronted with new data. More specific definitions require that machines display a level of autonomy and capacity for learning that enables rational action. For instance, the EU’s High-Level Expert Group on AI has defined AI as “systems that display intelligent behaviour by analysing their environment and taking actions—with some degree of autonomy—to achieve specific goals” (2019, 1). Yet, illustrating the potential for conceptual controversy, this definition has been criticized for denoting both too many and too few technologies as AI ( Heikkilä 2022a ).

AI technology is already implemented in a wide variety of areas in everyday life and the economy at large. For instance, the conversational chatbot ChatGPT is estimated to have reached 100 million users just  two months after its launch at the end of 2022 ( Hu 2023 ). AI applications enable new automation technologies, with subsequent positive or negative effects on the demand for labor, employment, and economic equality ( Acemoglu and Restrepo 2020 ). Military AI is integral to lethal autonomous weapons systems (LAWS), whereby machines take autonomous decisions in warfare and battlefield targeting ( Rosert and Sauer 2018 ). Many governments and public agencies have already implemented AI in their daily operations in order to more efficiently evaluate welfare eligibility, flag potential fraud, profile suspects, make risk assessments, and engage in mass surveillance ( Saif et al. 2017 ; Powers and Ganascia 2020 ; Berk 2021 ; Misuraca and van Noordt 2022 , 38).

Societies face significant governance challenges in relation to the implementation of AI. One type of challenge arises when AI systems function poorly, such as when applications involving some degree of autonomous decision-making produce technical failures with real-world implications. The “Robodebt” scheme in Australia, for instance, was designed to detect mistaken social security payments, but the Australian government ultimately had to rescind 400,000 wrongfully issued welfare debts ( Henriques-Gomes 2020 ). Similarly, Dutch authorities recently implemented an algorithm that pushed tens of thousands of families into poverty after mistakenly requiring them to repay child benefits, ultimately forcing the government to resign ( Heikkilä 2022b ).

Another type of governance challenge arises when AI systems function as intended but produce impacts whose consequences may be regarded as problematic. For instance, the inherent opacity of AI decision-making challenges expectations on transparency and accountability in public decision-making in liberal democracies ( Burrell 2016 ; Erman and Furendal 2022a ). Autonomous weapons raise critical ethical and legal issues ( Rosert and Sauer 2019 ). AI applications for surveillance in law enforcement give rise to concerns of individual privacy and human rights ( Rademacher 2019 ). AI-driven automation involves changes in labor markets that are painful for parts of the population ( Acemoglu and Restrepo 2020 ). Generative AI upends conventional ways of producing creative content and raises new copyright and data security issues ( Metz 2022 ).

More broadly, AI presents a governance challenge due to its effects on economic competitiveness, military security, and personal integrity, with consequences for states and societies. In this respect, AI may not be radically different from earlier general-purpose technologies, such as the steam engine, electricity, nuclear power, and the internet ( Frey 2019 ). From this perspective, it is not the novelty of AI technology that makes it a pressing issue to regulate but rather the anticipation that AI will lead to large-scale changes and become a source of power for state and societal actors.

Challenges such as these have led to a rapid expansion in recent years of efforts to regulate AI at different levels of governance. The OECD AI Policy Observatory records more than 700 national AI policy initiatives from 60 countries and territories ( OECD 2021 ). Earlier research into the governance of AI has therefore naturally focused mostly on the national level ( Radu 2021 ; Roberts et al. 2021 ; Taeihagh 2021 ). However, a large number of governance initiatives have also been undertaken at the global level, and many more are underway. According to an ongoing inventory of AI regulatory initiatives by the Council of Europe, IOs overtook national authorities as the main source of such initiatives in 2020 ( Council of Europe 2023 ).  Figure 1 visualizes this trend.

Origins of AI governance initiatives, 2015–2022. Source : Council of Europe (2023 ).

According to this source, national authorities launched 170 initiatives from 2015 to 2022, while IOs put in place 210 initiatives during the same period. Over time, the share of regulatory initiatives emanating from IOs has thus grown to surpass the share resulting from national authorities. Examples of the former include the OECD Principles on Artificial Intelligence agreed in 2019, the UNESCO Recommendation on Ethics of AI adopted in 2021, and the EU’s ongoing negotiations on the EU AI Act. In addition, several governance initiatives emanate from the private sector, civil society, and multistakeholder partnerships. In the next section, we will provide a more developed characterization of these global regulatory initiatives.

Two concerns likely explain why AI increasingly is becoming subject to governance at the global level. First, AI creates externalities that do not follow national borders and whose regulation requires international cooperation. China’s Artificial Intelligence Development Plan, for instance, clearly states that the country is using AI as a leapfrog technology in order to enhance national competitiveness ( Roberts et al. 2021 ). Since states with less regulation might gain a competitive edge when developing certain AI applications, there is a risk that such strategies create a regulatory race to the bottom. International cooperation that creates a level playing field could thus be said to be in the interest of all parties.

Second, the development of AI technology is a cross-border process carried out by transnational actors—multinational firms in particular. Big tech corporations, such as Google, Meta, or the Chinese drone maker DJI, are investing vast sums into AI development. The innovations of hardware manufacturers like Nvidia enable breakthroughs but depend on complex global supply chains, and international research labs such as DeepMind regularly present cutting-edge AI applications. Since the private actors that develop AI can operate across multiple national jurisdictions, the efforts to regulate AI development and deployment also need to be transboundary. Only by introducing common rules can states ensure that AI businesses encounter similar regulatory environments, which both facilitates transboundary AI development and reduces incentives for companies to shift to countries with laxer regulation.

Successful global governance of AI could help realize many of the potential benefits of the technology while mitigating its negative consequences. For AI to contribute to increased economic productivity, for instance, there needs to be predictable and clear regulation as well as global coordination around standards that prevent competition between parallel technical systems. Conversely, a failure to provide suitable global governance could lead to substantial risks. The intentional misuse of AI technology may undermine trust in institutions, and if left unchecked, the positive and negative externalities created by automation technologies might fall unevenly across different groups. Race dynamics similar to those that arose around nuclear technology in the twentieth century—where technological leadership created large benefits—might lead international actors and private firms to overlook safety issues and create potentially dangerous AI applications ( Dafoe 2018 ; Future of Life Institute 2023 ). Hence, policymakers face the task of disentangling beneficial from malicious consequences and then foster the former while regulating the latter. Given the speed at which AI is developed and implemented, governance also risks constantly being one step behind the technological frontier.

A prime example of how AI presents a global governance challenge is the efforts to regulate military AI, in particular autonomous weapons capable of identifying and eliminating a target without the involvement of a remote human operator ( Hernandez 2021 ). Both the development and the deployment of military applications with autonomous capabilities transcend national borders. Multinational defense companies are at the forefront of developing autonomous weapons systems. Reports suggest that such autonomous weapons are now beginning to be used in armed conflicts ( Trager and Luca 2022 ). The development and deployment of autonomous weapons involve the types of competitive dynamics and transboundary consequences identified above. In addition, they raise specific concerns with respect to accountability and dehumanization ( Sparrow 2007 ; Stop Killer Robots 2023 ). For these reasons, states have begun to explore the potential for joint global regulation of autonomous weapons systems. The principal forum is the Group on Governmental Experts (GGE) within the Convention on Certain Conventional Weapons (CCW). Yet progress in these negotiations is slow as the major powers approach this issue with competing interests in mind, illustrating the challenges involved in developing joint global rules.

The example of autonomous weapons further illustrates how the global governance of AI raises urgent empirical and normative questions for research. On the empirical side, these developments invite researchers to map emerging regulatory initiatives, such as those within the CCW, and to explain why these particular frameworks become dominant. What are the principal characteristics of global regulatory initiatives in the area of autonomous weapons, and how do power differentials, interest constellations, and principled ideas influence those rules? On the normative side, these developments invite researchers to address key normative questions raised by the development and deployment of autonomous weapons. What are the key normative issues at stake in the regulation of autonomous weapons, both with respect to the process through which such rules are developed and with respect to the consequences of these frameworks? To what extent are existing normative ideals and frameworks, such as just war theory, applicable to the governing of military AI ( Roach and Eckert 2020 )? Despite the global governance challenge of AI development and use, research on this topic is still in its infancy (but see Maas 2021 ; Schmitt 2021 ). In the remainder of this article, we therefore present an agenda for research into the global governance of AI. We begin by outlining an agenda for positive empirical research on the global governance of AI and then suggest an agenda for normative philosophical research.

An empirical perspective on the global governance of AI suggests two main questions: How may we describe the emerging global governance of AI? And how may we explain the emerging global governance of AI? In this section, we argue that concepts and theories drawn from the general study of global governance will be helpful as we address these questions, but also that AI, conversely, raises novel issues that point to the need for new or refined theories. Specifically, we show how global AI governance may be mapped along several conceptual dimensions and submit that theories invoking power dynamics, interests, ideas, and non-state actors have explanatory promise.

Mapping AI Governance

A key priority for empirical research on the global governance of AI is descriptive: Where and how are new regulatory arrangements emerging at the global level? What features characterize the emergent regulatory landscape? In answering such questions, researchers can draw on scholarship on international law and IR, which have conceptualized mechanisms of regulatory change and drawn up analytical dimensions to map and categorize the resulting regulatory arrangements.

Any mapping exercise must consider the many different ways in global AI regulation may emerge and evolve. Previous research suggests that legal development may take place in at least three distinct ways. To begin with, existing rules could be reinterpreted to also cover AI ( Maas 2021 ). For example, the principles of distinction, proportionality, and precaution in international humanitarian law could be extended, via reinterpretation, to apply to LAWS, without changing the legal source. Another manner in which new AI regulation may appear is via “ add-ons ” to existing rules. For example, in the area of global regulation of autonomous vehicles, AI-related provisions were added to the 1968 Vienna Road Traffic Convention through an amendment in 2015 ( Kunz and Ó hÉigeartaigh 2020 ). Finally, AI regulation may appear as a completely new framework , either through new state behavior that results in customary international law or through a new legal act or treaty ( Maas 2021 , 96). Here, one example of regulating AI through a new framework is the aforementioned EU AI Act, which would take the form of a new EU regulation.

Once researchers have mapped emerging regulatory arrangements, a central task will be to categorize them. Prior scholarship suggests that regulatory arrangements may be fruitfully analyzed in terms of five key dimensions (cf. Koremenos et al. 2001 ; Wahlgren 2022 , 346–347). A first dimension is whether regulation is horizontal or vertical . A horizontal regulation covers several policy areas, whereas a vertical regulation is a delimited legal framework covering one specific policy area or application. In the field of AI, emergent governance appears to populate both ends of this spectrum. For example, the proposed EU AI Act (2021), the UNESCO Recommendations on the Ethics of AI (2021), and the OECD Principles on AI (2019), which are not specific to any particular AI application or field, would classify as attempts at horizontal regulation. When it comes to vertical regulation, there are fewer existing examples, but discussions on a new protocol on LAWS within the CCW signal that this type of regulation is likely to become more important in the future ( Maas 2019a ).

A second dimension runs from centralization to decentralization . Governance is centralized if there is a single, authoritative institution at the heart of a regime, such as in trade, where the World Trade Organization (WTO) fulfills this role. In contrast, decentralized arrangements are marked by parallel and partly overlapping institutions, such as in the governance of the environment, the internet, or genetic resources (cf. Raustiala and Victor 2004 ). While some IOs with universal membership, such as UNESCO, have taken initiatives relating to AI governance, no institution has assumed the role as the core regulatory body at the global level. Rather, the proliferation of parallel initiatives, across levels and regions, lends weight to the conclusion that contemporary arrangements for the global governance of AI are strongly decentralized ( Cihon et al. 2020a ).

A third dimension is the continuum from hard law to soft law . While domestic statutes and treaties may be described as hard law, soft law is associated with guidelines of conduct, recommendations, resolutions, standards, opinions, ethical principles, declarations, guidelines, board decisions, codes of conduct, negotiated agreements, and a large number of additional normative mechanisms ( Abbott and Snidal 2000 ; Wahlgren 2022 ). Even though such soft documents may initially have been drafted as non-legal texts, they may in actual practice acquire considerable strength in structuring international relations ( Orakhelashvili 2019 ). While some initiatives to regulate AI classify as hard law, including the EU’s AI Act, Burri (2017 ) suggests that AI governance is likely to be dominated by “supersoft law,” noting that there are currently numerous processes underway creating global standards outside traditional international law-making fora. In a phenomenon that might be described as “bottom-up law-making” ( Koven Levit 2017 ), states and IOs are bypassed, creating norms that defy traditional categories of international law ( Burri 2017 ).

A fourth dimension concerns private versus public regulation . The concept of private regulation overlaps partly with substance understood as soft law, to the extent that private actors develop non-binding guidelines ( Wahlgren 2022 ). Significant harmonization of standards may be developed by private standardization bodies, such as the IEEE ( Ebers 2022 ). Public authorities may regulate the responsibility of manufacturers through tort law and product liability law ( Greenstein 2022 ). Even though contracts are originally matters between private parties, some contractual matters may still be regulated and enforced by law ( Ubena 2022 ).

A fifth dimension relates to the division between military and non-military regulation . Several policymakers and scholars describe how military AI is about to escalate into a strategic arms race between major powers such as the United States and China, similar to the nuclear arms race during the Cold War (cf. Petman 2017 ; Thompson and Bremmer 2018 ; Maas 2019a ). The process in the CCW Group of Governmental Experts on the regulation of LAWS is probably the largest single negotiation on AI ( Maas 2019b ) next to the negotiations on the EU AI Act. The zero-sum logic that appears to exist between states in the area of national security, prompting a military AI arms race, may not be applicable to the same extent to non-military applications of AI, potentially enabling a clearer focus on realizing positive-sum gains through regulation.

These five dimensions can provide guidance as researchers take up the task of mapping and categorizing global AI regulation. While the evidence is preliminary, in its present form, the global governance of AI must be understood as combining horizontal and vertical elements, predominantly leaning toward soft law, being heavily decentralized, primarily public in nature, and mixing military and non-military regulation. This multi-faceted and non-hierarchical nature of global AI governance suggests that it is best characterized as a regime complex , or a “larger web of international rules and regimes” ( Alter and Meunier 2009 , 13; Keohane and Victor 2011 ) rather than as a single, discrete regime.

If global AI governance can be understood as a regime complex, which some researchers already claim ( Cihon et al. 2020a ), future scholarship should look for theoretical and methodological inspiration in research on regime complexity in other policy fields. This research has found that regime complexes are characterized by path dependence, as existing rules shape the formulation of new rules; venue shopping, as actors seek to steer regulatory efforts to the fora most advantageous to their interests; and legal inconsistencies, as rules emerge from fractious and overlapping negotiations in parallel processes ( Raustiala and Victor 2004 ). Scholars have also considered the design of regime complexes ( Eilstrup-Sangiovanni and Westerwinter 2021 ), institutional overlap among bodies in regime complexes ( Haftel and Lenz 2021 ), and actors’ forum-shopping within regime complexes ( Verdier 2022 ). Establishing whether these patterns and dynamics are key features also of the AI regime complex stands out as an important priority in future research.

Explaining AI Governance

As our understanding of the empirical patterns of global AI governance grows, a natural next step is to turn to explanatory questions. How may we explain the emerging global governance of AI? What accounts for variation in governance arrangements and how do they compare with those in other policy fields, such as environment, security, or trade? Political science and IR offer a plethora of useful theoretical tools that can provide insights into the global governance of AI. However, at the same time, the novelty of AI as a governance challenge raises new questions that may require novel or refined theories. Thus far, existing research on the global governance of AI has been primarily concerned with descriptive tasks and largely fallen short in engaging with explanatory questions.

We illustrate the potential of general theories to help explain global AI governance by pointing to three broad explanatory perspectives in IR ( Martin and Simmons 2012 )—power, interests, and ideas—which have served as primary sources of theorizing on global governance arrangements in other policy fields. These perspectives have conventionally been associated with the paradigmatic theories of realism, liberalism, and constructivism, respectively, but like much of the contemporary IR discipline, we prefer to formulate them as non-paradigmatic sources for mid-level theorizing of more specific phenomena (cf. Lake 2013 ). We focus our discussion on how accounts privileging power, interests, and ideas have explained the origins and designs of IOs and how they may help us explain wider patterns of global AI governance. We then discuss how theories of non-state actors and regime complexity, in particular, offer promising avenues for future research into the global governance of AI. Research fields like science and technology studies (e.g., Jasanoff 2016 ) or the political economy of international cooperation (e.g., Gilpin 1987 ) can provide additional theoretical insights, but these literatures are not discussed in detail here.

A first broad explanatory perspective is provided by power-centric theories, privileging the role of major states, capability differentials, and distributive concerns. While conventional realism emphasizes how states’ concern for relative gains impedes substantive international cooperation, viewing IOs as epiphenomenal reflections of underlying power relations ( Mearsheimer 1994 ), developed power-oriented theories have highlighted how powerful states seek to design regulatory contexts that favor their preferred outcomes ( Gruber 2000 ) or shape the direction of IOs using informal influence ( Stone 2011 ; Dreher et al. 2022 ).

In research on global AI governance, power-oriented perspectives are likely to prove particularly fruitful in investigating how great-power contestation shapes where and how the technology will be regulated. Focusing on the major AI powerhouses, scholars have started to analyze the contrasting regulatory strategies and policies of the United States, China, and the EU, often emphasizing issues of strategic competition, military balance, and rivalry ( Kania 2017 ; Horowitz et al. 2018 ; Payne 2018 , 2021 ; Johnson 2019 ; Jensen et al. 2020 ). Here, power-centric theories could help understand the apparent emphasis on military AI in both the United States and China, as witnessed by the recent establishment of a US National Security Commission on AI and China’s ambitious plans of integrating AI into its military forces ( Ding 2018 ). The EU, for its part, is negotiating the comprehensive AI Act, seeking to use its market power to set a European standard for AI that subsequently can become the global standard, as it previously did with its GDPR law on data protection and privacy ( Schmitt 2021 ). Given the primacy of these three actors in AI development, their preferences and outlook regarding regulatory solutions will remain a key research priority.

Power-based accounts are also likely to provide theoretical inspiration for research on AI governance in the domain of security and military competition. Some scholars are seeking to assess the implications of AI for strategic rivalries, and their possible regulation, by drawing on historical analogies ( Leung 2019 ; see also Drezner 2019 ). Observing that, from a strategic standpoint, military AI exhibits some similarities to the problems posed by nuclear weapons, researchers have examined whether lessons from nuclear arms control have applicability in the domain of AI governance. For example, Maas (2019a ) argues that historical experience suggests that the proliferation of military AI can potentially be slowed down via institutionalization, while Zaidi and Dafoe (2021 ), in a study of the Baruch Plan for Nuclear Weapons, contend that fundamental strategic obstacles—including mistrust and fear of exploitation by other states—need to be overcome to make regulation viable. This line of investigation can be extended by assessing other historical analogies, such as the negotiations that led to the Strategic Arms Limitation Talks (SALT) in 1972 or more recent efforts to contain the spread of nuclear weapons, where power-oriented factors have shown continued analytical relevance (e.g., Ruzicka 2018 ).

A second major explanatory approach is provided by the family of theoretical accounts that highlight how international cooperation is shaped by shared interests and functional needs ( Keohane 1984 ; Martin 1992 ). A key argument in rational functionalist scholarship is that states are likely to establish IOs to overcome barriers to cooperation—such as information asymmetries, commitment problems, and transaction costs—and that the design of these institutions will reflect the underlying problem structure, including the degree of uncertainty and the number of involved actors (e.g., Koremenos et al. 2001 ; Hawkins et al. 2006 ; Koremenos 2016 ).

Applied to the domain of AI, these approaches would bring attention to how the functional characteristics of AI as a governance problem shape the regulatory response. They would also emphasize the investigation of the distribution of interests and the possibility of efficiency gains from cooperation around AI governance. The contemporary proliferation of partnerships and initiatives on AI governance points to the suitability of this theoretical approach, and research has taken some preliminary steps, surveying state interests and their alignment (e.g., Campbell 2019 ; Radu 2021 ). However, a systematic assessment of how the distribution of interests would explain the nature of emerging governance arrangements, both in the aggregate and at the constituent level, has yet to be undertaken.

A third broad explanatory perspective is provided by theories emphasizing the role of history, norms, and ideas in shaping global governance arrangements. In contrast to accounts based on power and interests, this line of scholarship, often drawing on sociological assumptions and theory, focuses on how institutional arrangements are embedded in a wider ideational context, which itself is subject to change. This perspective has generated powerful analyses of how societal norms influence states’ international behavior (e.g., Acharya and Johnston 2007 ), how norm entrepreneurs play an active role in shaping the origins and diffusion of specific norms (e.g., Finnemore and Sikkink 1998 ), and how IOs socialize states and other actors into specific norms and behaviors (e.g., Checkel 2005 ).

Examining the extent to which domestic and societal norms shape discussions on global governance arrangements stands out as a particularly promising area of inquiry. Comparative research on national ethical standards for AI has already indicated significant cross-country convergence, indicating a cluster of normative principles that are likely to inspire governance frameworks in many parts of the world (e.g., Jobin et al. 2019 ). A closely related research agenda concerns norm entrepreneurship in AI governance. Here, preliminary findings suggest that civil society organizations have played a role in advocating norms relating to fundamental rights in the formulation of EU AI policy and other processes ( Ulnicane 2021 ). Finally, once AI governance structures have solidified further, scholars can begin to draw on norms-oriented scholarship to design strategies for the analysis of how those governance arrangements may play a role in socialization.

In light of the particularities of AI and its political landscape, we expect that global governance scholars will be motivated to refine and adapt these broad theoretical perspectives to address new questions and conditions. For example, considering China’s AI sector-specific resources and expertise, power-oriented theories will need to grapple with questions of institutional creation and modification occurring under a distribution of power that differs significantly from the Western-centric processes that underpin most existing studies. Similarly, rational functionalist scholars will need to adapt their tools to address questions of how the highly asymmetric distribution of AI capabilities—in particular between producers, which are few, concentrated, and highly resourced, and users and subjects, which are many, dispersed, and less resourced—affects the formation of state interests and bargaining around institutional solutions. For their part, norm-oriented theories may need to be refined to capture the role of previously understudied sources of normative and ideational content, such as formal and informal networks of computer programmers, which, on account of their expertise, have been influential in setting the direction of norms surrounding several AI technologies.

We expect that these broad theoretical perspectives will continue to inspire research on the global governance of AI, in particular for tailored, mid-level theorizing in response to new questions. However, a fully developed research agenda will gain from complementing these theories, which emphasize particular independent variables (power, interests, and norms), with theories and approaches that focus on particular issues, actors, and phenomena. There is an abundance of theoretical perspectives that can be helpful in this regard, including research on the relationship between science and politics ( Haas 1992 ; Jasanoff 2016 ), the political economy of international cooperation ( Gilpin 1987 ; Frieden et al. 2017 ), the complexity of global governance ( Raustiala and Victor 2004 ; Eilstrup-Sangiovanni and Westerwinter 2021 ), and the role of non-state actors ( Risse 2012 ; Tallberg et al. 2013 ). We focus here on the latter two: theories of regime complexity, which have grown to become a mainstream approach in global governance scholarship, as well as theories of non-state actors, which provide powerful tools for understanding how private organizations influence regulatory processes. Both literatures hold considerable promise in advancing scholarship of AI global governance beyond its current state.

As concluded above, the current structure of global AI governance fits the description of a regime complex. Thus, approaching AI governance through this theoretical lens, understanding it as a larger web of rules and regulations, can open new avenues of research (see Maas 2021 for a pioneering effort). One priority is to analyze the AI regime complex in terms of core dimensions, such as scale, diversity, and density ( Eilstrup-Sangiovanni and Westerwinter 2021 ). Pointing to the density of this regime complex, existing studies have suggested that global AI governance is characterized by a high degree of fragmentation ( Schmitt 2021 ), which has motivated assessments of the possibility of greater centralization ( Cihon et al. 2020b ). Another area of research is to examine the emergence of legal inconsistencies and tensions, likely to emerge because of the diverging preferences of major AI players and the tendency of self-interest actors to forum-shop when engaging within a regime complex. Finally, given that the AI regime complex exists in a very early state, it provides researchers with an excellent opportunity to trace the origins and evolution of this form of governance structure from the outset, thus providing a good case for both theory development and novel empirical applications.

If theories of regime complexity can shine a light on macro-level properties of AI governance, other theoretical approaches can guide research into micro-level dynamics and influences. Recognizing that non-state actors are central in both AI development and its emergent regulation, researchers should find inspiration in theories and tools developed to study the role and influence of non-state actors in global governance (for overviews, see Risse 2012 ; Jönsson and Tallberg forthcoming ). Drawing on such work will enable researchers to assess to what extent non-state actor involvement in the AI regime complex differs from previous experiences in other international regimes. It is clear that large tech companies, like Google, Meta, and Microsoft, have formed regulatory preferences and that their monetary resources and technological expertise enable them to promote these interests in legislative and bureaucratic processes. For example, the Partnership on AI (PAI), a multistakeholder organization with more than 50 members, includes American tech companies at the forefront of AI development and fosters research on issues of AI ethics and governance ( Schmitt 2021 ). Other non-state actors, including civil society watchdog organizations, like the Civil Liberties Union for Europe, have been vocal in the negotiations of the EU AI Act, further underlining the relevance of this strand of research.

When investigating the role of non-state actors in the AI regime complex, research may be guided by four primary questions. A first question concerns the interests of non-state actors regarding alternative AI global governance architectures. Here, a survey by Chavannes et al. (2020 ) on possible regulatory approaches to LAWS suggests that private companies developing AI applications have interests that differ from those of civil society organizations. Others have pointed to the role of actors rooted in research and academia who have sought to influence the development of AI ethics guidelines ( Zhu 2022 ). A second question is to what extent the regulatory institutions and processes are accessible to the aforementioned non-state actors in the first place. Are non-state actors given formal or informal opportunities to be substantively involved in the development of new global AI rules? Research points to a broad and comprehensive opening up of IOs over the past two decades ( Tallberg et al. 2013 ) and, in the domain of AI governance, early indications are that non-state actors have been granted access to several multilateral processes, including in the OECD and the EU (cf. Niklas and Dencik 2021 ). A third question concerns actual participation: Are non-state actors really making use of the opportunities to participate, and what determines the patterns of participation? In this vein, previous research has suggested that the participation of non-state actors is largely dependent on their financial resources ( Uhre 2014 ) or the political regime of their home country ( Hanegraaff et al. 2015 ). In the context of AI governance, this raises questions about if and how the vast resource disparities and divergent interests between private tech corporations and civil society organizations may bias patterns of participation. There is, for instance, research suggesting that private companies are contributing to a practice of ethics washing by committing to nonbinding ethical guidelines while circumventing regulation ( Wagner 2018 ; Jobin et al. 2019 ; Rességuier and Rodrigues 2020 ). Finally, a fourth question is to what extent, and how, non-state actors exert influence on adopted AI rules. Existing scholarship suggests that non-state actors typically seek to shape the direction of international cooperation via lobbying ( Dellmuth and Tallberg 2017 ), while others have argued that non-state actors use participation in international processes largely to expand or sustain their own resources ( Hanegraaff et al. 2016 ).

The previous section suggested that emerging global initiatives to regulate AI amount to a regime complex and that an empirical approach could help to map and explain these regulatory developments. In this section, we move beyond positive empirical questions to consider the normative concerns at stake in the global governance of AI. We argue that normative theorizing is needed both for assessing how well existing arrangements live up to ideals such as democracy and justice and for evaluating how best to specify what these ideals entail for the global governance of AI.

Ethical values frequently highlighted in the context of AI governance include transparency, inclusion, accountability, participation, deliberation, fairness, and beneficence ( Floridi et al. 2018 ; Jobin et al. 2019 ). A normative perspective suggests several ways in which to theorize and analyze such values in relation to the global governance of AI. One type of normative analysis focuses on application, that is, on applying an existing normative theory to instances of AI governance, assessing how well such regulatory arrangements realize their principles (similar to how political theorists have evaluated whether global governance lives up to standards of deliberation; see Dryzek 2011 ; Steffek and Nanz 2008 ). Such an analysis could also be pursued more narrowly by using a certain normative theory to assess the implications of AI technologies, for instance, by approaching the problem of algorithmic bias based on notions of fairness or justice ( Vredenburgh 2022 ). Another type of normative analysis moves from application to justification, analyzing the structure of global AI governance with the aim of theory construction. In this type of analysis, the goal is to construe and evaluate candidate principles for these regulatory arrangements in order to arrive at the best possible (most justified) normative theory. In this case, the theorist starts out from a normative ideal broadly construed (concept) and arrives at specific principles (conception).

In the remainder of this section, we will point to the promises of analyzing global AI governance based on the second approach. We will focus specifically on the normative ideals of justice and democracy. While many normative ideals could serve as focal points for an analysis of the AI domain, democracy and justice appear particularly central for understanding the normative implications of the governance of AI. Previous efforts to deploy political philosophy to shed light on normative aspects of global governance point to the promise of this focus (e.g., Caney 2005 , 2014 ; Buchanan 2013 ). It is also natural to focus on justice and democracy given that many of the values emphasized in AI ethics and existing ethics guidelines are analytically close to justice and democracy. Our core argument will be that normative research needs to be attentive to how these ideals would be best specified in relation to both the procedures and outcomes of the global governance of AI.

AI Ethics and the Normative Analysis of Global AI Governance

Although there is a rich literature on moral or ethical aspects related to specific AI applications, investigations into normative aspects of global AI governance are surprisingly sparse (for exceptions, see Müller 2020 ; Erman and Furendal 2022a , 2022b ). Researchers have so far focused mostly on normative and ethical questions raised by AI considered as a tool, enabling, for example, autonomous weapons systems ( Sparrow 2007 ) and new forms of political manipulation ( Susser et al. 2019 ; Christiano 2021 ). Some have also considered AI as a moral agent of its own, focusing on how we could govern, or be governed by, a hypothetical future artificial general intelligence ( Schwitzgebel and Garza 2015 ; Livingston and Risse 2019 ; cf. Tasioulas 2019 ; Bostrom et al. 2020 ; Erman and Furendal 2022a ). Examples such as these illustrate that there is, by now, a vibrant field of “AI ethics” that aims to consider normative aspects of specific AI applications.

As we have shown above, however, initiatives to regulate AI beyond the nation-state have become increasingly common, and they are often led by IOs, multinational companies, private standardization bodies, and civil society organizations. These developments raise normative issues that require a shift from AI ethics in general to systematic analyses of the implications of global AI governance. It is crucial to explore these normative dimensions of how AI is governed, since how AI is governed invokes key normative questions pertaining to the ideals that ought to be met.

Apart from attempts to map or describe the central norms in the existing global governance of AI (cf. Jobin et al.), most normative analyses of the global governance of AI can be said to have proceeded in two different ways. The dominant approach is to employ an outcome-based focus ( Dafoe 2018 ; Winfield et al. 2019 ; Taeihagh 2021 ), which starts by identifying a potential problem or promise created by AI technology and then seeks to identify governance mechanisms or principles that can minimize risks or make a desired outcome more likely. This approach can be contrasted with a procedure-based focus, which attaches comparatively more weight to how governance processes happen in existing or hypothetical regulatory arrangements. It recognizes that there are certain procedural aspects that are important and might be overlooked by an analysis that primarily assesses outcomes.

The benefits of this distinction become apparent if we focus on the ideals of justice and democracy. Broadly construed, we understand justice as an ideal for how to distribute benefits and burdens—specifying principles that determine “who owes what to whom”—and democracy as an ideal for collective decision-making and the exercise of political power—specifying principles that determine “who has political power over whom” ( Barry 1991 ; Weale 1999 ; Buchanan and Keohane 2006 ; Christiano 2008 ; Valentini 2012 , 2013 ). These two ideals can be analyzed with a focus on procedure or outcome, producing four fruitful avenues of normative research into global AI governance. First, justice could be understood as a procedural value or as a distributive outcome. Second, and likewise, democracy could be a feature of governance processes or an outcome of those processes. Below, we discuss existing research from the standpoint of each of these four avenues. We conclude that there is great potential for novel insights if normative theorists consider the relatively overlooked issues of outcome aspects of justice and procedural aspects of democracy in the global governance of AI.

Procedural and Outcome Aspects of Justice

Discussions around the implications of AI applications on justice, or fairness, are predominantly concerned with procedural aspects of how AI systems operate. For instance, ever since the problem of algorithmic bias—i.e., the tendency that AI-based decision-making reflects and exacerbates existing biases toward certain groups—was brought to public attention, AI ethicists have offered suggestions of why this is wrong, and AI developers have sought to construct AI systems that treat people “fairly” and thus produce “justice.” In this context, fairness and justice are understood as procedural ideals, which AI decision-making frustrates when it fails to treat like cases alike, and instead systematically treats individuals from different groups differently ( Fazelpour and Danks 2021 ; Zimmermann and Lee-Stronach 2022 ). Paradigmatic examples include automated predictions about recidivism among prisoners that have impacted decisions about people’s parole and algorithms used in recruitment that have systematically favored men over women ( Angwin et al. 2016 ; O'Neil 2017 ).

However, the emerging global governance of AI also has implications for how the benefits and burdens of AI technology are distributed among groups and states—i.e., outcomes ( Gilpin 1987 ; Dreher and Lang 2019 ). Like the regulation of earlier technological innovations ( Krasner 1991 ; Drezner 2019 ), AI governance may not only produce collective benefits, but also favor certain actors at the expense of others ( Dafoe 2018 ; Horowitz 2018 ). For instance, the concern about AI-driven automation and its impact on employment is that those who lose their jobs because of AI might carry a disproportionately large share of the negative externalities of the technology without being compensated through access to its benefits (cf. Korinek and Stiglitz 2019 ; Erman and Furendal 2022a ). Merely focusing on justice as a procedural value would overlook such distributive effects created by the diffusion of AI technology.

Moreover, this example illustrates that since AI adoption may produce effects throughout the global economy, regulatory efforts will have to go beyond issues relating to the technology itself. Recognizing the role of outcomes of AI governance entails that a broad range of policies need to be pursued by existing and emerging governance regimes. The global trade regime, for instance, may need to be reconsidered in order for the distribution of positive and negative externalities of AI technology to be just. Suggestions include pursuing policies that can incentivize certain kinds of AI technology or enable the profits gained by AI developers to be shared more widely (cf. Floridi et al. 2018 ; Erman and Furendal 2022a ).

In sum, with regard to outcome aspects of justice, theories are needed to settle which benefits and burdens created by global AI adoption ought to be fairly distributed and why (i.e., what the “site” and “scope” of AI justice are) (cf. Gabriel 2022 ). Similarly, theories of procedural aspects should look beyond individual applications of AI technology and ask whether a fairer distribution of influence over AI governance may help produce more fair outcomes, and if so how. Extending existing theories of distributive justice to the realm of global AI governance may put many of their central assumptions in a new light.

Procedural and Outcome Aspects of Democracy

Normative research could also fruitfully shed light on how emerging AI governance should be analyzed in relation to the ideal of democracy, such as what principles or criteria of democratic legitimacy are most defensible. It could be argued, for instance, that the decision process must be open to democratic influence for global AI governance to be democratically legitimate ( Erman and Furendal 2022b ). Here, normative theory can explain why it matters from the standpoint of democracy whether the affected public has had a say—either directly through open consultation or indirectly through representation—in formulating the principles that guide AI governance. The nature of the emerging AI regime complex—where prominent roles are held by multinational companies and private standard-setting bodies—suggests that it is far from certain that the public will have this kind of influence.

Importantly, it is likely that democratic procedures will take on different shapes in global governance compared to domestic politics ( Dahl 1999 ; Scholte 2011 ). A viable democratic theory must therefore make sense of how the unique properties of global governance raise issues or require solutions that are distinct from those in the domestic context. For example, the prominent influence of non-state actors, including the large tech corporations developing cutting-edge AI technology, suggests that it is imperative to ask whether different kinds of decision-making may require different normative standards and whether different kinds of actors may have different normative status in such decision-making arrangements.

Initiatives from non-state actors, such as the tech company-led PAI discussed above, often develop their own non-coercive ethics guidelines. Such documents may seek effects similar to coercively upheld regulation, such as the GDPR or the EU AI Act. For example, both Google and the EU specify that AI should not reinforce biases ( High-Level Expert Group on Artificial Intelligence 2019 ; Google 2022 ). However, from the perspective of democratic legitimacy, it may matter extensively which type of entity adopts AI regulations and on what grounds those decision-making entities have the authority to issue AI regulations ( Erman and Furendal 2022b ).

Apart from procedural aspects, a satisfying democratic theory of global AI governance will also have to include a systematic analysis of outcome aspects. Important outcome aspects of democracy include accountability and responsiveness. Accountability may be improved, for example, by instituting mechanisms to prevent corruption among decision-makers and to secure public access to governing documents, and responsiveness may be improved by strengthening the discursive quality of global decision processes, for instance, by involving international NGOs and civil movements that give voice to marginalized groups in society. With regard to tracing citizens’ preferences, some have argued that democratic decision-making can be enhanced by AI technology that tracks what people want and consistently reach “better” decisions than human decision-makers (cf. König and Wenzelburger 2022 ). Apart from accountability and responsiveness, other relevant outcome aspects of democracy include, for example, the tendency to promote conflict resolution, improve the epistemic quality of decisions, and dignity and equality among citizens.

In addition, it is important to analyze how procedural and outcome concerns are related. This issue is often neglected, which again can be illustrated by the ethics guidelines from IOs, such as the OECD Principles on Artificial Intelligence and the UNESCO Recommendation on Ethics of AI. Such documents often stress the importance of democratic values and principles, such as transparency, accountability, participation, and deliberation. Yet they typically treat these values as discrete and rarely explain how they are interconnected ( Jobin et al. 2019 ; Schiff et al. 2020 ; Hagendorff 2020 , 103). Democratic theory can fruitfully step in to explain how the ideal of “the rule by the people” includes two sides that are intimately connected. First, there is an access side of political power, where those affected should have a say in the decision-making, which might require participation, deliberation, and political equality. Second, there is an exercise side of political power, where those very decisions should apply in appropriate ways, which in turn might require effectiveness, transparency, and accountability. In addition to efforts to map and explain norms and values in the global governance of AI, theories of democratic AI governance can hence help explain how these two aspects are connected (cf. Erman 2020 ).

In sum, the global governance of AI raises a number of issues for normative research. We have identified four promising avenues, focused on procedural and outcome aspects of justice and democracy in the context of global AI governance. Research along these four avenues can help to shed light on the normative challenges facing the global governance of AI and the key values at stake, as well as provide the impetus for novel theories on democratic and just global AI governance.

This article has charted a new agenda for research into the global governance of AI. While existing scholarship has been primarily descriptive or policy-oriented, we propose an agenda organized around theory-driven positive and normative questions. To this end, we have outlined two broad analytical perspectives on the global governance of AI: an empirical approach, aimed at conceptualizing and explaining global AI governance; and a normative approach, aimed at developing and applying ideals for appropriate global AI governance. Pursuing these empirical and normative approaches can help to guide future scholarship on the global governance of AI toward critical questions, core concepts, and promising theories. At the same time, exploring AI as a regulatory issue provides an opportunity to further develop these general analytical approaches as they confront the particularities of this important area of governance.

We conclude this article by highlighting the key takeaways from this research agenda for future scholarship on empirical and normative dimensions of the global governance of AI. First, research is required to identify where and how AI is becoming globally governed . Mapping and conceptualizing the emerging global governance of AI is a first necessary step. We argue that research may benefit from considering the variety of ways in which new regulation may come about, from the reinterpretation of existing rules and the extension of prevailing sectoral governance to the negotiation of entirely new frameworks. In addition, we suggest that scholarship may benefit from considering how global AI governance may be conceptualized in terms of key analytical dimensions, such as horizontal–vertical, centralized–decentralized, and formal–informal.

Second, research is necessary to explain why AI is becoming globally governed in particular ways . Having mapped global AI governance, we need to account for the factors that drive and shape these regulatory processes and arrangements. We argue that political science and IR offer a variety of theoretical tools that can help to explain the global governance of AI. In particular, we highlight the promise of theories privileging the role of power, interests, ideas, regime complexes, and non-state actors, but also recognize that research fields such as science and technology studies and political economy can yield additional theoretical insights.

Third, research is needed to identify what normative ideals global AI governance ought to meet . Moving from positive to normative issues, a first critical question pertains to the ideals that should guide the design of appropriate global AI governance. We argue that normative theory provides the tools necessary to engage with this question. While normative theory can suggest several potential principles, we believe that it may be especially fruitful to start from the ideals of democracy and justice, which are foundational and recurrent concerns in discussions about political governing arrangements. In addition, we suggest that these two ideals are relevant both for the procedures by which AI regulation is adopted and for the outcomes of such regulation.

Fourth, research is required to evaluate how well global AI governance lives up to these normative ideals . Once appropriate normative ideals have been selected, we can assess to what extent and how existing arrangements conform to these principles. We argue that previous research on democracy and justice in global governance offers a model in this respect. A critical component of such research is the integration of normative and empirical research: normative research for elucidating how normative ideals would be expressed in practice, and empirical research for analyzing data on whether actual arrangements live up to those ideals.

In all, the research agenda that we outline should be of interest to multiple audiences. For students of political science and IR, it offers an opportunity to apply and refine concepts and theories in a novel area of global governance of extensive future importance. For scholars of AI, it provides an opportunity to understand how political actors and considerations shape the conditions under which AI applications may be developed and used. For policymakers, it presents an opportunity to learn about evolving regulatory practices and gaps, interests shaping emerging arrangements, and trade-offs to be confronted in future efforts to govern AI at the global level.

A previous version of this article was presented at the Global and Regional Governance workshop at Stockholm University. We are grateful to Tim Bartley, Niklas Bremberg, Lisa Dellmuth, Felicitas Fritzsche, Faradj Koliev, Rickard Söder, Carl Vikberg, Johanna von Bahr, and three anonymous reviewers for ISR for insightful comments and suggestions. The research for this article was funded by the WASP-HS program of the Marianne and Marcus Wallenberg Foundation (Grant no. MMW 2020.0044).

We use “global governance” to refer to regulatory processes beyond the nation-state, whether on a global or regional level. While states and IOs often are central to these regulatory processes, global governance also involves various types of non-state actors ( Rosenau 1999 ).

Abbott Kenneth W. , and Snidal Duncan . 2000 . “ Hard and Soft Law in International Governance .” International Organization . 54 ( 3 ): 421 – 56 .

Google Scholar

Acemoglu Daron , and Restrepo Pascual . 2020 . “ The Wrong Kind of AI? Artificial Intelligence and the Future of Labour Demand .” Cambridge Journal of Regions, Economy and Society . 13 ( 1 ): 25 – 35 .

Acharya Amitav , and Johnston Alistair Iain . 2007 . “ Conclusion: Institutional Features, Cooperation Effects, and the Agenda for Further Research on Comparative Regionalism .” In Crafting Cooperation: Regional International Institutions in Comparative Perspective , edited by Acharya Amitav , Johnston Alistair Iain , 244 – 78 .. Cambridge : Cambridge University Press .

Google Preview

Alter Karen J. , and Meunier Sophie . 2009 . “ The Politics of International Regime Complexity .” Perspectives on Politics . 7 ( 1 ): 13 – 24 .

Angwin Julia , Larson Jeff , Mattu Surya , and Kirchner Lauren . 2016 . “ Machine Bias .” ProPublica , May 23 . Internet (last accessed August 25, 2023): https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing .

Barry Brian . 1991 . “ Humanity and Justice in Global Perspective .” In Liberty and Justice , edited by Barry Brian . Oxford : Clarendon .

Berk Richard A . 2021 . “ Artificial Intelligence, Predictive Policing, and Risk Assessment for Law Enforcement .” Annual Review of Criminology . 4 ( 1 ): 209 – 37 .

Bostrom Nick , Dafoe Allan , and Flynn Carrick . 2020 . “ Public Policy and Superintelligent AI: A Vector Field Approach .” In Ethics of Artificial Intelligence , edited by Liao S. Matthew , 293 – 326 .. Oxford : Oxford University Press .

Buchanan Allen , and Keohane Robert O. . 2006 . “ The Legitimacy of Global Governance Institutions .” Ethics & International Affairs . 20 (4) : 405 – 37 .

Buchanan Allen . 2013 . The Heart of Human Rights . Oxford : Oxford University Press .

Burrell Jenna . 2016 . “ How the Machine “Thinks”: Understanding Opacity in Machine Learning Algorithms .” Big Data & Society . 3 ( 1 ): 1 – 12 .. https://doi.org/10.1177/2053951715622512 .

Burri Thomas . 2017 . “ International Law and Artificial Intelligence .” In German Yearbook of International Law , vol. 60 , 91 – 108 .. Berlin : Duncker and Humblot .

Butcher James , and Beridze Irakli . 2019 . “ What is the State of Artificial Intelligence Governance Globally?” . The RUSI Journal . 164 ( 5-6 ): 88 – 96 .

Büthe Tim , Djeffal Christian , Lütge Christoph , Maasen Sabine , and von Ingersleben-Seip Nora . 2022 . “ Governing AI—Attempting to Herd Cats? Introduction to the Special Issue on the Governance of Artificial Intelligence .” Journal of European Public Policy . 29 ( 11 ): 1721 – 52 .

Campbell Thomas A . 2019 . Artificial Intelligence: An Overview of State Initiatives . Evergreen, CO : FutureGrasp .

Caney Simon . 2005 . “ Cosmopolitan Justice, Responsibility, and Global Climate Change .” Leiden Journal of International Law . 18 ( 4 ): 747 – 75 .

Caney Simon . 2014 . “ Two Kinds of Climate Justice: Avoiding Harm and Sharing Burdens .” Journal of Political Philosophy . 22 ( 2 ): 125 – 49 .

Chavannes Esther , Klonowska Klaudia , and Sweijs Tim . 2020 . Governing Autonomous Weapon Systems: Expanding the Solution Space, From Scoping to Applying . The Hague : The Hague Center for Strategic Studies .

Checkel Jeffrey T . 2005 . “ International Institutions and Socialization in Europe: Introduction and Framework .” International organization . 59 ( 4 ): 801 – 26 .

Christiano Thomas . 2008 . The Constitution of Equality . Oxford : Oxford University Press .

Christiano Thomas . 2021 . “ Algorithms, Manipulation, and Democracy .” Canadian Journal of Philosophy . 52 ( 1 ): 109 – 124 .. https://doi.org/10.1017/can.2021.29 .

Cihon Peter , Maas Matthijs M. , and Kemp Luke . 2020a . “ Fragmentation and the Future: Investigating Architectures for International AI Governance .” Global Policy . 11 ( 5 ): 545 – 56 .

Cihon Peter , Maas Matthijs M. , and Kemp Luke . 2020b . “ Should Artificial Intelligence Governance Be Centralised? Design Lessons from History .” In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , 228 – 34 . New York, NY: ACM .

Council of Europe . 2023 . “ AI Initiatives ,” accessed 16 June 2023, AI initiatives (coe.int).

Dafoe Allan . 2018 . AI Governance: A Research Agenda . Oxford: Governance of AI Program , Future of Humanity Institute, University of Oxford . www.fhi.ox.ac.uk/govaiagenda .

Dahl Robert . 1999 . “ Can International Organizations Be Democratic: A Skeptic's View .” In Democracy's Edges , edited by Shapiro Ian , Hacker-Córdon Casiano , 19 – 36 .. Cambridge : Cambridge University Press .

Dellmuth Lisa M. , and Tallberg Jonas . 2017 . “ Advocacy Strategies in Global Governance: Inside versus Outside Lobbying .” Political Studies . 65 ( 3 ): 705 – 23 .

Ding Jeffrey . 2018 . Deciphering China's AI Dream: The Context, Components, Capabilities and Consequences of China's Strategy to Lead the World in AI . Oxford: Centre for the Governance of AI , Future of Humanity Institute, University of Oxford .

Dreher Axel , and Lang Valentin . 2019 . “ The Political Economy of International Organizations .” In The Oxford Handbook of Public Choice , Volume 2, edited by Congleton Roger O. , Grofman Bernhard , Voigt Stefan . Oxford : Oxford University Press .

Dreher Axel , Lang Valentin , Rosendorff B. Peter , and Vreeland James R. . 2022 . “ Bilateral or Multilateral? International Financial Flows and the Dirty Work-Hypothesis .” The Journal of Politics . 84 ( 4 ): 1932 – 1946 .

Drezner Daniel W . 2019 . “ Technological Change and International Relations .” International Relations . 33 ( 2 ): 286 – 303 .

Dryzek John . 2011 . “ Global Democratization: Soup, Society, or System? ” Ethics & International Affairs , 25 ( 2 ): 211 – 234 .

Ebers Martin . 2022 . “ Explainable AI in the European Union: An Overview of the Current Legal Framework(s) .” In Nordic Yearbook of Law and Informatics 2020–2021: Law in the Era of Artificial Intelligence , edited by Lianne Colonna and Stanley Greenstein . Stockholm: The Swedish Law and Informatics Institute, Stockholm University .

Eilstrup-Sangiovanni Mette , and Westerwinter Oliver . 2021 . “ The Global Governance Complexity Cube: Varieties of Institutional Complexity in Global Governance .” Review of International Organizations . 17 (2): 233 – 262 .

Erman Eva , and Furendal Markus . 2022a . “ The Global Governance of Artificial Intelligence: Some Normative Concerns .” Moral Philosophy & Politics . 9 (2): 267−291. https://www.degruyter.com/document/doi/10.1515/mopp-2020-0046/html .

Erman Eva , and Furendal Markus . 2022b . “ Artificial Intelligence and the Political Legitimacy of Global Governance .” Political Studies . https://journals.sagepub.com/doi/full/10.1177/00323217221126665 .

Erman Eva . 2020 . “ A Function-Sensitive Approach to the Political Legitimacy of Global Governance .” British Journal of Political Science . 50 ( 3 ): 1001 – 24 .

Fazelpour Sina , and Danks David . 2021 . “ Algorithmic Bias: Senses, Sources, Solutions .” Philosophy Compass . 16 ( 8 ): e12760.

Finnemore Martha , and Sikkink Kathryn . 1998 . “ International Norm Dynamics and Political Change .” International Organization . 52 ( 4 ): 887 – 917 .

Floridi Luciano , Cowls Josh , Beltrametti Monica , Chatila Raja , Chazerand Patrice , Dignum Virginia , Luetge Christoph et al.  2018 . “ AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations .” Minds and Machines . 28 ( 4 ): 689 – 707 .

Frey Carl Benedikt . 2019 . The Technology Trap: Capital, Labor, and Power in the Age of Automation . Princeton, NJ : Princeton University Press .

Frieden Jeffry , Lake David A. , and Lawrence Broz J. 2017 . International Political Economy: Perspectives on Global Power and Wealth . Sixth Edition. New York, NY : W.W. Norton .

Future of Life Institute , 2023 . “ Pause Giant AI Experiments: An Open Letter .” Accessed June 13, 2023. https://futureoflife.org/open-letter/pause-giant-ai-experiments/ .

Gabriel Iason , 2022 . “ Toward a Theory of Justice for Artificial Intelligence .” Daedalus , 151 ( 2 ): 218 – 31 .

Gilpin Robert . 1987 . The Political Economy of International Relations . Princeton, NJ : Princeton University Press .

Google . 2022 . “ Artificial Intelligence at Google: Our Principles .” Internet (last accessed August 25, 2023): https://ai.google/principles/ .

Greenstein Stanley . 2022 . “ Liability in the Era of Artificial Intelligence .” In Nordic Yearbook of Law and Informatics 2020–2021: Law in the Era of Artificial Intelligence , edited by Colonna Lianne , Greenstein Stanley . Stockholm: The Swedish Law and Informatics Institute, Stockholm University .

Gruber Lloyd . 2000 . Ruling the World . Princeton, NJ : Princeton University Press .

Haas Peter . 1992 . “ Introduction: Epistemic Communities and International Policy Coordination .” International Organization . 46 ( 1 ): 1 – 36 .

Haftel Yoram Z. , and Lenz Tobias . 2021 . “ Measuring Institutional Overlap in Global Governance .” Review of International Organizations . 17(2) : 323 – 347 .

Hagendorff Thilo . 2020 . “ The Ethics of AI Ethics: an Evaluation of Guidelines .” Minds and Machines . 30 ( 1 ): 99 – 120 .

Hanegraaff Marcel , Beyers Jan , and De Bruycker Iskander . 2016 . “ Balancing Inside and Outside Lobbying: The Political Strategy of Lobbyists at Global Diplomatic Conferences .” European Journal of Political Research . 55 ( 3 ): 568 – 88 .

Hanegraaff Marcel , Braun Caelesta , De Bièvre Dirk , and Beyers Jan . 2015 . “ The Domestic and Global Origins of Transnational Advocacy: Explaining Lobbying Presence During WTO Ministerial Conferences .” Comparative Political Studies . 48 : 1591 – 621 .

Hawkins Darren G. , Lake David A. , Nielson Daniel L. , Tierney Michael J. Eds. 2006 . Delegation and Agency in International Organizations . Cambridge : Cambridge University Press .

Heikkilä Melissa . 2022a . “ AI: Decoded. IoT Under Fire—Defining AI?—Meta's New AI Supercomputer .” Accessed June 5, 2022, https://www.politico.eu/newsletter/ai-decoded/iot-under-fire-defining-ai-metas-new-ai-supercomputer-2 /.

Heikkilä Melissa 2022b . “ AI: Decoded. A Dutch Algorithm Scandal Serves a Warning to Europe—The AI Act won't Save Us .” Accessed June 5, 2022, https://www.politico.eu/newsletter/ai-decoded/a-dutch-algorithm-scandal-serves-a-warning-to-europe-the-ai-act-wont-save-us-2/ .

Henriques-Gomes Luke . 2020 . “ Robodebt: Government Admits It Will Be Forced to Refund $550 m under Botched Scheme .” The Guardian . sec. Australia news . Internet (last accessed August 25, 2023): https://www.theguardian.com/australia-news/2020/mar/27/robodebt-government-admits-it-will-be-forced-to-refund-550m-under-botched-scheme .

Hernandez Joe . 2021 . “ A Military Drone With A Mind Of Its Own Was Used In Combat, U.N. Says .” National Public Radio . Internet (las accessed August 25, 2023): https://www.npr.org/2021/06/01/1002196245/a-u-n-report-suggests-libya-saw-the-first-battlefield-killing-by-an-autonomous-d .

High-Level Expert Group on Artificial Intelligence . 2019 . Ethics Guidelines for Trustworthy AI . Brussels: European Commission . https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai .

Horowitz Michael . 2018 . “ Artificial Intelligence, International Competition, and the Balance of Power .” Texas National Security Review . 1 ( 3 ): 37 – 57 .

Horowitz Michael C. , Allen Gregory C. , Kania Elsa B. , Scharre Paul . 2018 . Strategic Competition in an Era of Artificial Intelligence . Washington D.C. : Center for a New American Security .

Hu Krystal . 2023 . ChatGPT Sets Record for Fastest-Growing User Base—Analyst Note. Reuters , February 2, 2023, sec. Technology , Accessed June 12, 2023, https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/ .

Jasanoff Sheila . 2016 . The Ethics of Invention: Technology and the Human Future . New York : Norton .

Jensen Benjamin M. , Whyte Christopher , and Cuomo Scott . 2020 . “ Algorithms at War: the Promise, Peril, and Limits of Artificial Intelligence .” International Studies Review . 22 ( 3 ): 526 – 50 .

Jobin Anna , Ienca Marcello , and Vayena Effy . 2019 . “ The Global Landscape of AI Ethics Guidelines .” Nature Machine Intelligence . 1 ( 9 ): 389 – 99 .

Johnson J. 2019 . “ Artificial intelligence & Future Warfare: Implications for International Security .” Defense & Security Analysis . 35 ( 2 ): 147 – 69 .

Jönsson Christer , and Tallberg Jonas . Forthcoming. “Opening up to Civil Society: Access, Participation, and Impact .” In Handbook on Governance in International Organizations , edited by Edgar Alistair . Cheltenham : Edward Elgar Publishing .

Kania E. B . 2017 . Battlefield singularity. Artificial Intelligence, Military Revolution, and China's Future Military Power . Washington D.C.: CNAS .

Keohane Robert O . 1984 . After Hegemony . Princeton, NJ : Princeton University Press .

Keohane Robert O. , and Victor David G. . 2011 . “ The Regime Complex for Climate Change .” Perspectives on Politics . 9 ( 1 ): 7 – 23 .

König Pascal D. and Georg Wenzelburger 2022 . “ Between Technochauvinism and Human-Centrism: Can Algorithms Improve Decision-Making in Democratic Politics? ” European Political Science , 21 ( 1 ): 132 – 49 .

Koremenos Barbara , Lipson Charles , and Snidal Duncan . 2001 . “ The Rational Design of International Institutions .” International Organization . 55 ( 4 ): 761 – 99 .

Koremenos Barbara . 2016 . The Continent of International Law: Explaining Agreement Design . Cambridge : Cambridge University Press .

Korinek Anton and Stiglitz Joseph E. 2019 . “ Artificial Intelligence and Its Implications for Income Distribution and Unemployment ” In The Economics of Artificial Intelligence: An Agenda . edited by Agrawal A. , Gans J. and Goldfarb A. . University of Chicago Press . :.

Koven Levit Janet . 2007 . “ Bottom-Up International Lawmaking: Reflections on the New Haven School of International Law .” Yale Journal of International Law . 32 : 393 – 420 .

Krasner Stephen D . 1991 . “ Global Communications and National Power: Life on the Pareto Frontier .” World Politics . 43 ( 3 ): 336 – 66 .

Kunz Martina , and hÉigeartaigh Seán Ó . 2020 . “ Artificial Intelligence and Robotization .” In The Oxford Handbook on the International Law of Global Security , edited by Geiss Robin , Melzer Nils . Oxford : Oxford University Press .

Lake David. A . 2013 . “ Theory is Dead, Long Live Theory: The End of the Great Debates and the rise of eclecticism in International Relations .” European Journal of International Relations . 19 ( 3 ): 567 – 87 .

Leung Jade . 2019 . “ Who Will Govern Artificial Intelligence?” . Learning from the History of Strategic Politics in Emerging Technologies . Doctoral dissertation . Oxford: University of Oxford .

Livingston Steven , and Mathias Risse . 2019 , “ The Future Impact of Artificial Intelligence on Humans and Human Rights .” Ethics & International Affairs . 33 ( 2 ): 141 – 58 .

Maas Matthijs M . 2019a . “ How Viable is International Arms Control for Military Artificial Intelligence? Three Lessons from Nuclear Weapons .” Contemporary Security Policy . 40 ( 3 ): 285 – 311 .

Maas Matthijs M . 2019b . “ Innovation-proof Global Governance for Military Artificial Intelligence? How I Learned to Stop Worrying, and Love the Bot ,” Journal of International Humanitarian Legal Studies . 10 ( 1 ): 129 – 57 .

Maas Matthijs M . 2021 . Artificial Intelligence Governance under Change: Foundations, Facets, Frameworks . PhD dissertation . Copenhagen: University of Copenhagen .

Martin Lisa L . 1992 . “ Interests, Power, and Multilateralism .” International Organization . 46 ( 4 ): 765 – 92 .

Martin Lisa L. , and Simmons Beth A. . 2012 . “ International Organizations and Institutions .” In Handbook of International Relations , edited by Carlsnaes Walter , Risse Thomas , Simmons Beth A. . London : SAGE .

McCarthy John , Minsky Marvin L. , Rochester Nathaniel , and Shannon Claude E . 1955 . “ A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence .” AI Magazine . 27 ( 4 ): 12 – 14 (reprint) .

Mearsheimer John J. . 1994 . “ The False Promise of International Institutions .” International Security , 19 ( 3 ): 5 – 49 .

Metz Cade . 2022 . “ Lawsuit Takes Aim at the Way a.I. Is Built .” The New York Times , November 23, Accessed June 21, 2023. https://www.nytimes.com/2022/11/23/technology/copilot-microsoft-ai-lawsuit.html . June 21, 2023 .

Misuraca Gianluca , van Noordt Colin 2022 . “ Artificial Intelligence for the Public Sector: Results of Landscaping the Use of AI in Government Across the European Union .” Government Information Quarterly . 101714 . https://doi.org/10.1016/j.giq.2022.101714 .

Müller Vincent C . 2020 . “ Ethics of Artificial Intelligence and Robotics .” In Stanford Encyclopedia of Philosophy , edited by Zalta Edward N. Internet (last accessed August 25, 2023): https://plato.stanford.edu/archives/fall2020/entries/ethics-ai/ .

Niklas Jedrzen , Dencik Lina . 2021 . “ What Rights Matter? Examining the Place of Social Rights in the EU's Artificial Intelligence Policy Debate .” Internet Policy Review . 10 ( 3 ): 1 – 29 .

OECD . 2021 . “ OECD AI Policy Observatory .” Accessed February 17, 2022. https://oecd.ai .

O'Neil Cathy . 2017 . Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy . UK : Penguin Books .

Orakhelashvili Alexander . 2019 . Akehurst's Modern Introduction to International Law , Eighth Edition . London : Routledge .

Payne K . 2021 . I, Warbot: The Dawn of Artificially Intelligent Conflict . Oxford: Oxford University Press .

Payne Kenneth . 2018 . “ Artificial Intelligence: a Revolution in Strategic Affairs?” . Survival . 60 ( 5 ): 7 – 32 .

Petman Jarna . 2017 . Autonomous Weapons Systems and International Humanitarian Law: ‘Out of the Loop’ . Helsinki : The Eric Castren Institute of International Law and Human Rights .

Powers Thomas M. , and Ganascia Jean-Gabriel . 2020 . “ The Ethics of the Ethics of AI .” In The Oxford Handbook of Ethics of AI , edited by Dubber Markus D. , Pasquale Frank , Das Sunit , 25 – 51 .. Oxford : Oxford University Press .

Rademacher Timo . 2019 . “ Artificial Intelligence and Law Enforcement .” In Regulating Artificial Intelligence , edited by Wischmeyer Thomas , Rademacher Timo , 225 – 54 .. Cham: Springer .

Radu Roxana . 2021 . “ Steering the Governance of Artificial Intelligence: National Strategies in Perspective .” Policy and Society . 40 ( 2 ): 178 – 93 .

Raustiala Kal and David G. Victor . 2004 .“ The Regime Complex for Plant Genetic Resources .” International Organization , 58 ( 2 ): 277 – 309 .

Rességuier Anaïs , and Rodrigues Rowena . 2020 . “ AI Ethics Should Not Remain Toothless! A Call to Bring Back the Teeth of Ethics .” Big Data & Society . 7 ( 2 ). https://doi.org/10.1177/2053951720942541 .

Risse Thomas . 2012 . “ Transnational Actors and World Politics .” In Handbook of International Relations , 2nd ed., edited by Carlsnaes Walter , Risse Thomas , Simmons Beth A. . London : Sage .

Roach Steven C. , and Eckert Amy , eds. 2020 . Moral Responsibility in Twenty-First-Century Warfare: Just War Theory and the Ethical Challenges of Autonomous Weapons Systems . Albany, NY : State University of New York .

Roberts Huw , Cowls Josh , Morley Jessica , Taddeo Mariarosaria , Wang Vincent , and Floridi Luciano . 2021 . “ The Chinese Approach to Artificial Intelligence: An Analysis of Policy, Ethics, and Regulation .” AI & Society . 36 ( 1 ): 59 – 77 .

Rosenau James N . 1999 . “ Toward an Ontology for Global Governance .” In Approaches to Global Governance Theory , edited by Hewson Martin , Sinclair Timothy J. , 287 – 301 .. Albany, NY : SUNY Press .

Rosert Elvira , and Sauer Frank . 2018 . Perspectives for Regulating Lethal Autonomous Weapons at the CCW: A Comparative Analysis of Blinding Lasers, Landmines, and LAWS . Paper prepared for the workshop “New Technologies of Warfare: Implications of Autonomous Weapons Systems for International Relations,” 5th EISA European Workshops in International Studies , Groningen , 6-9 June 2018 . Internet (last accessed August 25, 2023): https://www.academia.edu/36768452/Perspectives_for_Regulating_Lethal_Autonomous_Weapons_at_the_CCW_A_Comparative_Analysis_of_Blinding_Lasers_Landmines_and_LAWS

Rosert Elvira , and Sauer Frank . 2019 . “ Prohibiting Autonomous Weapons: Put Human Dignity First .” Global Policy . 10 ( 3 ): 370 – 5 .

Russell Stuart J. , and Norvig Peter . 2020 . Artificial Intelligence: A Modern Approach . Boston, MA : Pearson .

Ruzicka Jan . 2018 . “ Behind the Veil of Good Intentions: Power Analysis of the Nuclear Non-proliferation Regime .” International Politics . 55 ( 3 ): 369 – 85 .

Saif Hassan , Dickinson Thomas , Kastler Leon , Fernandez Miriam , and Alani Harith . 2017 . “ A Semantic Graph-Based Approach for Radicalisation Detection on Social Media .” ESWC 2017: The Semantic Web—Proceedings, Part 1 , 571 – 87 .. Cham : Springer .

Schiff Daniel , Justin Biddle , Jason Borenstein , and Kelly Laas . 2020 . “ What’s Next for AI Ethics, Policy, and Governance? A Global Overview .” In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society . ACM , , .

Schmitt Lewin . 2021 . “ Mapping Global AI Governance: A Nascent Regime in a Fragmented Landscape .” AI and Ethics . 2 ( 2 ): 303 – 314 .

Scholte Jan Aart . ed. 2011 . Building Global Democracy? Civil Society and Accountable Global Governance . Cambridge : Cambridge University Press .

Schwitzgebel Eric , and Garza Mara . 2015 . “ A Defense of the Rights of Artificial Intelligences .” Midwest Studies In Philosophy . 39 ( 1 ): 98 – 119 .

Sparrow Robert . 2007 . “ Killer Robots .” Journal of Applied Philosophy . 24 ( 1 ): 62 – 77 .

Steffek Jens and Patrizia Nanz . 2008 . “ Emergent Patterns of Civil Society Participation in Global and European Governance ” In Civil Society Participation in European and Global Governance . edited by Jens Steffek , Claudia Kissling , and Patrizia Nanz Basingstoke: Palgrave Macmillan . 1–29.

Stone Randall. W . 2011 . Controlling Institutions: International Organizations and the Global Economy . Cambridge : Cambridge University Press .

Stop Killer Robots . 2023 . “ About Us.” . Accessed June 13, 2023, https://www.stopkillerrobots.org/about-us/ .

Susser Daniel , Roessler Beate , Nissenbaum Helen . 2019 . “ Technology, Autonomy, and Manipulation .” Internet Policy Review . 8 ( 2 ):. https://doi.org/10.14763/2019.2.1410 .

Taeihagh Araz . 2021 . “ Governance of Artificial Intelligence .” Policy and Society . 40 ( 2 ): 137 – 57 .

Tallberg Jonas , Sommerer Thomas , Squatrito Theresa , and Jönsson Christer . 2013 . The Opening Up of International Organizations . Cambridge : Cambridge University Press .

Tasioulas John . 2019 . “ First Steps Towards an Ethics of Robots and Artificial Intelligence .” The Journal of Practical Ethics . 7 ( 1 ): 61-95. https://doi.org/10.2139/ssrn.3172840 .

Thompson Nicholas , and Bremmer Ian . 2018. “ The AI Cold War that Threatens us all .” Wired, October 23. Internet (last accessed August 25, 2023): https://www.wired.com/story/ai-cold-war-china-coulddoom-us-all/ .

Trager Robert F. , and Luca Laura M. . 2022 . “ Killer Robots Are Here—And We Need to Regulate Them .” Foreign Policy, May 11 . Internet (last accessed August 25, 2023): https://foreignpolicy.com/2022/05/11/killer-robots-lethal-autonomous-weapons-systems-ukraine-libya-regulation/

Ubena John . 2022 . “ Can Artificial Intelligence be Regulated?” . Lessons from Legislative Techniques . In Nordic Yearbook of Law and Informatics 2020–2021: Law in the Era of Artificial Intelligence , edited by Colonna Lianne , Greenstein Stanley . Stockholm: The Swedish Law and Informatics Institute , Stockholm University.

Uhre Andreas Nordang . 2014 . “ Exploring the Diversity of Transnational Actors in Global Environmental Governance .” Interest Groups & Advocacy . 3 ( 1 ): 59 – 78 .

Ulnicane Inga . 2021 . “ Artificial Intelligence in the European Union: Policy, Ethics and Regulation .” In The Routledge Handbook of European Integrations , edited by Hoerber Thomas , Weber Gabriel , Cabras Ignazio . London : Routledge .

Valentini Laura . 2013 . “ Justice, Disagreement and Democracy .” British Journal of Political Science . 43 ( 1 ): 177 – 99 .

Valentini Laura . 2012 . “ Assessing the Global Order: Justice, Legitimacy, or Political Justice?” . Critical Review of International Social and Political Philosophy . 15 ( 5 ): 593 – 612 .

Vredenburgh Kate . 2022 . “ Fairness .” In The Oxford Handbook of AI Governance , edited by Bullock Justin B. , Chen Yu-Che , Himmelreich Johannes , Hudson Valerie M. , Korinek Anton , Young Matthew M. , Zhang Baobao . Oxford : Oxford University Press .

Verdier Daniel . 2022 . “ Bargaining Strategies for Governance Complex Games .” The Review of International Organizations , 17 ( 2 ): 349 – 371 .

Wagner Ben . 2018 . “ Ethics as an Escape from Regulation. From “Ethics-washing” to Ethics-shopping? .” In Being Profiled: Cogitas Ergo Sum. 10 Years of ‘Profiling the European Citizen , edited by Bayamiloglu Emre , Baraliuc Irina , Janssens Liisa , Hildebrandt Mireille Amsterdam : Amsterdam University Press .

Wahlgren Peter . 2022 . “ How to Regulate AI?” In Nordic Yearbook of Law and Informatics 2020–2021: Law in the Era of Artificial Intelligence , edited by Colonna Lianne , Greenstein Stanley . Stockholm: The Swedish Law and Informatics Institute, Stockholm University .

Weale Albert . 1999 . Democracy . New York : St Martin's Press .

Winfield Alan F. , Michael Katina , Pitt Jeremy , and Evers Vanessa . 2019 . “ Machine Ethics: The Design and Governance of Ethical AI and Autonomous Systems .” Proceedings of the IEEE . 107 ( 3 ): 509 – 17 .

Zaidi Waqar , Dafoe Allan . 2021 . International Control of Powerful Technology: Lessons from the Baruch Plan for Nuclear Weapons . Working Paper 2021: 9 . Oxford : Centre for the Governance of AI .

Zhu J. 2022 . “ AI ethics with Chinese Characteristics? Concerns and preferred solutions in Chinese academia .” AI & Society . https://doi.org/10.1007/s00146-022-01578-w .

Zimmermann Annette , and Lee-Stronach Chad . 2022 . “ Proceed with Caution .” Canadian Journal of Philosophy . 52 ( 1 ): 6 – 25 .

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1468-2486
• Print ISSN 1521-9488
• Copyright © 2024 International Studies Association
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

We will keep fighting for all libraries - stand with us!

Internet Archive Audio

• This Just In
• Grateful Dead
• Old Time Radio
• 78 RPMs and Cylinder Recordings
• Audio Books & Poetry
• Computers, Technology and Science
• Music, Arts & Culture
• News & Public Affairs
• Spirituality & Religion
• Radio News Archive

• Flickr Commons
• Occupy Wall Street Flickr
• NASA Images
• Solar System Collection
• Ames Research Center

• All Software
• Old School Emulation
• MS-DOS Games
• Historical Software
• Classic PC Games
• Software Library
• Kodi Archive and Support File
• Vintage Software
• CD-ROM Software
• CD-ROM Software Library
• Software Sites
• Tucows Software Library
• Shareware CD-ROMs
• Software Capsules Compilation
• CD-ROM Images
• ZX Spectrum
• DOOM Level CD

• Smithsonian Libraries
• FEDLINK (US)
• Lincoln Collection
• American Libraries
• Canadian Libraries
• Universal Library
• Project Gutenberg
• Children's Library
• Biodiversity Heritage Library
• Books by Language
• Additional Collections

• Prelinger Archives
• Democracy Now!
• Occupy Wall Street
• TV NSA Clip Library
• Animation & Cartoons
• Arts & Music
• Computers & Technology
• Cultural & Academic Films
• Ephemeral Films
• Sports Videos
• Videogame Videos
• Youth Media

Search the history of over 866 billion web pages on the Internet.

Mobile Apps

• Wayback Machine (iOS)
• Wayback Machine (Android)

Browser Extensions

Archive-it subscription.

• Explore the Collections
• Build Collections

Save Page Now

Capture a web page as it appears now for use as a trusted citation in the future.

Please enter a valid web address

• Donate Donate icon An illustration of a heart shape

McDaniel Academic Symposium 2024

Video item preview, share or embed this item, flag this item for.

• Graphic Violence
• Explicit Sexual Content
• Hate Speech
• Misinformation/Disinformation
• Marketing/Phishing/Advertising
• Misleading/Inaccurate/Missing Metadata

plus-circle Add Review comment Reviews

Download options, in collections.

Uploaded by John Hauser on May 4, 2024

SIMILAR ITEMS (based on metadata)