literature review on qualitative research

Archer Library

Qualitative research: literature review .

• Archer Library This link opens in a new window
• Schedule a Reference Appointment This link opens in a new window
• Qualitative Research Handout This link opens in a new window
• Locating Books
• ebook Collections This link opens in a new window
• A to Z Database List This link opens in a new window
• Research & Stats
• Literature Review Resources
• Citation & Reference

Exploring the literature review 

Literature review model: 6 steps.

Adapted from The Literature Review , Machi & McEvoy (2009, p. 13).

Your Literature Review

Step 2: search, boolean search strategies, search limiters, ★ ebsco & google drive.

1. Select a Topic

"All research begins with curiosity" (Machi & McEvoy, 2009, p. 14)

Selection of a topic, and fully defined research interest and question, is supervised (and approved) by your professor. Tips for crafting your topic include:

• Be specific. Take time to define your interest.
• Topic Focus. Fully describe and sufficiently narrow the focus for research.
• Academic Discipline. Learn more about your area of research & refine the scope.
• Avoid Bias. Be aware of bias that you (as a researcher) may have.
• Document your research. Use Google Docs to track your research process.
• Research apps. Consider using Evernote or Zotero to track your research.

Consider Purpose

What will your topic and research address?

In The Literature Review: A Step-by-Step Guide for Students , Ridley presents that literature reviews serve several purposes (2008, p. 16-17).  Included are the following points:

• Historical background for the research;
• Overview of current field provided by "contemporary debates, issues, and questions;"
• Theories and concepts related to your research;
• Introduce "relevant terminology" - or academic language - being used it the field;
• Connect to existing research - does your work "extend or challenge [this] or address a gap;" 
• Provide "supporting evidence for a practical problem or issue" that your research addresses.

★ Schedule a research appointment

At this point in your literature review, take time to meet with a librarian. Why? Understanding the subject terminology used in databases can be challenging. Archer Librarians can help you structure a search, preparing you for step two. How? Contact a librarian directly or use the online form to schedule an appointment. Details are provided in the adjacent Schedule an Appointment box.

2. Search the Literature

Collect & Select Data: Preview, select, and organize

AU Library is your go-to resource for this step in your literature review process. The literature search will include books and ebooks, scholarly and practitioner journals, theses and dissertations, and indexes. You may also choose to include web sites, blogs, open access resources, and newspapers. This library guide provides access to resources needed to complete a literature review.

Books & eBooks: Archer Library & OhioLINK

Databases: scholarly & practitioner journals.

Review the Library Databases tab on this library guide, it provides links to recommended databases for Education & Psychology, Business, and General & Social Sciences.

Expand your journal search; a complete listing of available AU Library and OhioLINK databases is available on the Databases  A to Z list . Search the database by subject, type, name, or do use the search box for a general title search. The A to Z list also includes open access resources and select internet sites.

Databases: Theses & Dissertations

Review the Library Databases tab on this guide, it includes Theses & Dissertation resources. AU library also has AU student authored theses and dissertations available in print, search the library catalog for these titles.

Did you know? If you are looking for particular chapters within a dissertation that is not fully available online, it is possible to submit an ILL article request . Do this instead of requesting the entire dissertation.

Newspapers:  Databases & Internet

Consider current literature in your academic field. AU Library's database collection includes The Chronicle of Higher Education and The Wall Street Journal .  The Internet Resources tab in this guide provides links to newspapers and online journals such as Inside Higher Ed , COABE Journal , and Education Week .

Search Strategies & Boolean Operators

There are three basic boolean operators:  AND, OR, and NOT.

Used with your search terms, boolean operators will either expand or limit results. What purpose do they serve? They help to define the relationship between your search terms. For example, using the operator AND will combine the terms expanding the search. When searching some databases, and Google, the operator AND may be implied.

Overview of boolean terms

About the example: Boolean searches were conducted on November 4, 2019; result numbers may vary at a later date. No additional database limiters were set to further narrow search returns.

Database Search Limiters

Database strategies for targeted search results.

Most databases include limiters, or additional parameters, you may use to strategically focus search results.  EBSCO databases, such as Education Research Complete & Academic Search Complete provide options to:

• Limit results to full text;
• Limit results to scholarly journals, and reference available;
• Select results source type to journals, magazines, conference papers, reviews, and newspapers
• Publication date

Keep in mind that these tools are defined as limiters for a reason; adding them to a search will limit the number of results returned.  This can be a double-edged sword.  How? 

• If limiting results to full-text only, you may miss an important piece of research that could change the direction of your research. Interlibrary loan is available to students, free of charge. Request articles that are not available in full-text; they will be sent to you via email.
• If narrowing publication date, you may eliminate significant historical - or recent - research conducted on your topic.
• Limiting resource type to a specific type of material may cause bias in the research results.

Use limiters with care. When starting a search, consider opting out of limiters until the initial literature screening is complete. The second or third time through your research may be the ideal time to focus on specific time periods or material (scholarly vs newspaper).

★ Truncating Search Terms

Expanding your search term at the root.

Truncating is often referred to as 'wildcard' searching. Databases may have their own specific wildcard elements however, the most commonly used are the asterisk (*) or question mark (?).  When used within your search. they will expand returned results.

Asterisk (*) Wildcard

Using the asterisk wildcard will return varied spellings of the truncated word. In the following example, the search term education was truncated after the letter "t."

Explore these database help pages for additional information on crafting search terms.

• EBSCO Connect: Searching with Wildcards and Truncation Symbols
• EBSCO Connect: Searching with Boolean Operators
• EBSCO Connect: EBSCOhost Search Tips
• EBSCO Connect: Basic Searching with EBSCO
• ProQuest Help: Search Tips
• ERIC: How does ERIC search work?

★ EBSCO Databases & Google Drive

Tips for saving research directly to Google drive.

Researching in an EBSCO database?

It is possible to save articles (PDF and HTML) and abstracts in EBSCOhost databases directly to Google drive. Select the Google Drive icon, authenticate using a Google account, and an EBSCO folder will be created in your account. This is a great option for managing your research. If documenting your research in a Google Doc, consider linking the information to actual articles saved in drive.

EBSCO Databases & Google Drive

EBSCOHost Databases & Google Drive: Managing your Research

This video features an overview of how to use Google Drive with EBSCO databases to help manage your research. It presents information for connecting an active Google account to EBSCO and steps needed to provide permission for EBSCO to manage a folder in Drive.

About the Video:  Closed captioning is available, select CC from the video menu.  If you need to review a specific area on the video, view on YouTube and expand the video description for access to topic time stamps.  A video transcript is provided below.

• EBSCOhost Databases & Google Scholar

Defining Literature Review

What is a literature review.

A definition from the Online Dictionary for Library and Information Sciences .

A literature review is "a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works" (Reitz, 2014). 

A systemic review is "a literature review focused on a specific research question, which uses explicit methods to minimize bias in the identification, appraisal, selection, and synthesis of all the high-quality evidence pertinent to the question" (Reitz, 2014).

Recommended Reading

About this page

EBSCO Connect [Discovery and Search]. (2022). Searching with boolean operators. Retrieved May, 3, 2022 from https://connect.ebsco.com/s/?language=en_US

EBSCO Connect [Discover and Search]. (2022). Searching with wildcards and truncation symbols. Retrieved May 3, 2022; https://connect.ebsco.com/s/?language=en_US

Machi, L.A. & McEvoy, B.T. (2009). The literature review . Thousand Oaks, CA: Corwin Press: 

Reitz, J.M. (2014). Online dictionary for library and information science. ABC-CLIO, Libraries Unlimited . Retrieved from https://www.abc-clio.com/ODLIS/odlis_A.aspx

Ridley, D. (2008). The literature review: A step-by-step guide for students . Thousand Oaks, CA: Sage Publications, Inc.

Archer Librarians

Schedule an appointment.

Contact a librarian directly (email), or submit a request form. If you have worked with someone before, you can request them on the form.

• ★ Archer Library Help • Online Reqest Form
• Carrie Halquist • Reference & Instruction
• Jessica Byers • Reference & Curation
• Don Reams • Corrections Education & Reference
• Diane Schrecker • Education & Head of the IRC
• Tanaya Silcox • Technical Services & Business
• Sarah Thomas • Acquisitions & ATS Librarian
• << Previous: Research & Stats
• Next: Literature Review Resources >>
• Last Updated: Nov 17, 2023 7:46 AM
• URL: https://libguides.ashland.edu/qualitative

Archer Library • Ashland University © Copyright 2023. An Equal Opportunity/Equal Access Institution.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Chapter 9. Reviewing the Literature

What is a “literature review”.

No researcher ever comes up with a research question that is wholly novel. Someone, somewhere, has asked the same thing. Academic research is part of a larger community of researchers, and it is your responsibility, as a member of this community, to acknowledge others who have asked similar questions and to put your particular research into this greater context. It is not simply a convention or custom to begin your study with a review of previous literature (the “ lit review ”) but an important responsibility you owe the scholarly community.

Too often, new researchers pursue a topic to study and then write something like, “No one has ever studied this before” or “This area is underresearched.” It may be that no one has studied this particular group or setting, but it is highly unlikely no one has studied the foundational phenomenon of interest. And that comment about an area being underresearched? Be careful. The statement may simply signal to others that you haven’t done your homework. Rubin ( 2021 ) refers to this as “free soloing,” and it is not appreciated in academic work:

The truth of the matter is, academics don’t really like when people free solo. It’s really bad form to omit talking about the other people who are doing or have done research in your area. Partly, I mean we need to cite their work, but I also mean we need to respond to it—agree or disagree, clarify for extend. It’s also really bad form to talk about your research in a way that does not make it understandable to other academics.…You have to explain to your readers what your story is really about in terms they care about . This means using certain terminology, referencing debates in the literature, and citing relevant works—that is, in connecting your work to something else. ( 51–52 )

A literature review is a comprehensive summary of previous research on a topic. It includes both articles and books—and in some cases reports—relevant to a particular area of research. Ideally, one’s research question follows from the reading of what has already been produced. For example, you are interested in studying sports injuries related to female gymnasts. You read everything you can find on sports injuries related to female gymnasts, and you begin to get a sense of what questions remain open. You find that there is a lot of research on how coaches manage sports injuries and much about cultures of silence around treating injuries, but you don’t know what the gymnasts themselves are thinking about these issues. You look specifically for studies about this and find several, which then pushes you to narrow the question further. Your literature review then provides the road map of how you came to your very specific question, and it puts your study in the context of studies of sports injuries. What you eventually find can “speak to” all the related questions as well as your particular one.

In practice, the process is often a bit messier. Many researchers, and not simply those starting out, begin with a particular question and have a clear idea of who they want to study and where they want to conduct their study but don’t really know much about other studies at all. Although backward, we need to recognize this is pretty common. Telling students to “find literature” after the fact can seem like a purposeless task or just another hurdle for completing a thesis or dissertation. It is not! Even if you were not motivated by the literature in the first place, acknowledging similar studies and connecting your own research to those studies are important parts of building knowledge. Acknowledgment of past research is a responsibility you owe the discipline to which you belong.

Literature reviews can also signal theoretical approaches and particular concepts that you will incorporate into your own study. For example, let us say you are doing a study of how people find their first jobs after college, and you want to use the concept of social capital . There are competing definitions of social capital out there (e.g., Bourdieu vs. Burt vs. Putnam). Bourdieu’s notion is of one form of capital, or durable asset, of a “network of more or less institutionalized relationships of mutual acquaintance or recognition” ( 1984:248 ). Burt emphasizes the “brokerage opportunities” in a social network as social capital ( 1997:355 ). Putnam’s social capital is all about “facilitating coordination and cooperation for mutual benefit” ( 2001:67 ). Your literature review can adjudicate among these three approaches, or it can simply refer to the one that is animating your own research. If you include Bourdieu in your literature review, readers will know “what kind” of social capital you are talking about as well as what kind of social scientist you yourself are. They will likely understand that you are interested more in how some people are advantaged by their social capital relative to others rather than being interested in the mechanics of how social networks operate.

The literature review thus does two important things for you: firstly, it allows you to acknowledge previous research in your area of interest, thereby situating you within a discipline or body of scholars, and, secondly, it demonstrates that you know what you are talking about. If you present the findings of your research study without including a literature review, it can be like singing into the wind. It sounds nice, but no one really hears it, or if they do catch snippets, they don’t know where it is coming from.

Examples of Literature Reviews

To help you get a grasp of what a good literature review looks like and how it can advance your study, let’s take a look at a few examples.

Reader-Friendly Example: The Power of Peers

The first is by Janice McCabe ( 2016 ) and is from an article on peer networks in the journal Contexts . Contexts presents articles in a relatively reader-friendly format, with the goal of reaching a large audience for interesting sociological research. Read this example carefully and note how easily McCabe is able to convey the relevance of her own work by situating it in the context of previous studies:

Scholars who study education have long acknowledged the importance of peers for students’ well-being and academic achievement. For example, in 1961, James Coleman argued that peer culture within high schools shapes students’ social and academic aspirations and successes. More recently, Judith Rich Harris has drawn on research in a range of areas—from sociological studies of preschool children to primatologists’ studies of chimpanzees and criminologists’ studies of neighborhoods—to argue that peers matter much more than parents in how children “turn out.” Researchers have explored students’ social lives in rich detail, as in Murray Milner’s book about high school students, Freaks, Geeks, and Cool Kids , and Elizabeth Armstrong and Laura Hamilton’s look at college students, Paying for the Party . These works consistently show that peers play a very important role in most students’ lives. They tend, however, to prioritize social over academic influence and to use a fuzzy conception of peers rather than focusing directly on friends—the relationships that should matter most for student success. Social scientists have also studied the power of peers through network analysis, which is based on uncovering the web of connections between people. Network analysis involves visually mapping networks and mathematically comparing their structures (such as the density of ties) and the positions of individuals within them (such as how central a given person is within the network). As Nicholas Christakis and James Fowler point out in their book Connected , network structure influences a range of outcomes, including health, happiness, wealth, weight, and emotions. Given that sociologists have long considered network explanations for social phenomena, it’s surprising that we know little about how college students’ friends impact their experiences. In line with this network tradition, I focus on the structure of friendship networks, constructing network maps so that the differences we see across participants are due to the underlying structure, including each participant’s centrality in their friendship group and the density of ties among their friends. ( 23 )

What did you notice? In her very second sentence, McCabe uses “for example” to introduce a study by Coleman, thereby indicating that she is not going to tell you every single study in this area but is going to tell you that (1) there is a lot of research in this area, (2) it has been going on since at least 1961, and (3) it is still relevant (i.e., recent studies are still being done now). She ends her first paragraph by summarizing the body of literature in this area (after giving you a few examples) and then telling you what may have been (so far) left out of this research. In the second paragraph, she shifts to a separate interesting focus that is related to the first but is also quite distinct. Lit reviews very often include two (or three) distinct strands of literature, the combination of which nicely backgrounds this particular study . In the case of our female gymnast study (above), those two strands might be (1) cultures of silence around sports injuries and (2) the importance of coaches. McCabe concludes her short and sweet literature review with one sentence explaining how she is drawing from both strands of the literature she has succinctly presented for her particular study. This example should show you that literature reviews can be readable, helpful, and powerful additions to your final presentation.

Authoritative Academic Journal Example: Working Class Students’ College Expectations

The second example is more typical of academic journal writing. It is an article published in the British Journal of Sociology of Education by Wolfgang Lehmann ( 2009 ):

Although this increase in post-secondary enrolment and the push for university is evident across gender, race, ethnicity, and social class categories, access to university in Canada continues to be significantly constrained for those from lower socio-economic backgrounds (Finnie, Lascelles, and Sweetman 2005). Rising tuition fees coupled with an overestimation of the cost and an underestimation of the benefits of higher education has put university out of reach for many young people from low-income families (Usher 2005). Financial constraints aside, empirical studies in Canada have shown that the most important predictor of university access is parental educational attainment. Having at least one parent with a university degree significantly increases the likelihood of a young person to attend academic-track courses in high school, have high educational and career aspirations, and ultimately attend university (Andres et al. 1999, 2000; Lehmann 2007a). Drawing on Bourdieu’s various writing on habitus and class-based dispositions (see, for example, Bourdieu 1977, 1990), Hodkinson and Sparkes (1997) explain career decisions as neither determined nor completely rational. Instead, they are based on personal experiences (e.g., through employment or other exposure to occupations) and advice from others. Furthermore, they argue that we have to understand these decisions as pragmatic, rather than rational. They are pragmatic in that they are based on incomplete and filtered information, because of the social context in which the information is obtained and processed. New experiences and information can, however, also be allowed into one’s world, where they gradually or radically transform habitus, which in turn creates the possibility for the formation of new and different dispositions. Encountering a supportive teacher in elementary or secondary school, having ambitious friends, or chance encounters can spark such transformations. Transformations can be confirming or contradictory, they can be evolutionary or dislocating. Working-class students who enter university most certainly encounter such potentially transformative situations. Granfield (1991) has shown how initially dislocating feelings of inadequacy and inferiority of working-class students at an elite US law school were eventually replaced by an evolutionary transformation, in which the students came to dress, speak and act more like their middle-class and upper-class peers. In contrast, Lehmann (2007b) showed how persistent habitus dislocation led working-class university students to drop out of university. Foskett and Hemsley-Brown (1999) argue that young people’s perceptions of careers are a complex mix of their own experiences, images conveyed through adults, and derived images conveyed by the media. Media images of careers, perhaps, are even more important for working-class youth with high ambitions as they offer (generally distorted) windows into a world of professional employment to which they have few other sources of access. It has also been argued that working-class youth who do continue to university still face unique, class-specific challenges, evident in higher levels of uncertainty (Baxter and Britton 2001; Lehmann 2004, 2007a; Quinn 2004), their higher education choices (Ball et al. 2002; Brooks 2003; Reay et al. 2001) and fears of inadequacy because of their cultural outsider status (Aries and Seider 2005; Granfield 1991). Although the number of working-class university students in Canada has slowly increased, that of middle-class students at university has risen far more steeply (Knighton and Mizra 2002). These different enrolment trajectories have actually widened the participation gap, which in tum explains our continued concerns with the potential outsider status Indeed, in a study comparing first-generation working-class and traditional students who left university without graduating, Lehmann (2007b) found that first-generation working-class students were more likely to leave university very early in some cases within the first two months of enrollment. They were also more likely to leave university despite solid academic performance. Not “fitting in,” not “feeling university,” and not being able to “relate to these people” were key reasons for eventually withdrawing from university. From the preceding review of the literature, a number of key research questions arise: How do working-class university students frame their decision to attend university? How do they defy the considerable odds documented in the literature to attend university? What are the sources of information and various images that create dispositions to study at university? What role does their social-class background- or habitus play in their transition dispositions and how does this translate into expectations for university? ( 139 )

What did you notice here? How is this different from (and similar to) the first example? Note that rather than provide you with one or two illustrative examples of similar types of research, Lehmann provides abundant source citations throughout. He includes theory and concepts too. Like McCabe, Lehmann is weaving through multiple literature strands: the class gap in higher education participation in Canada, class-based dispositions, and obstacles facing working-class college students. Note how he concludes the literature review by placing his research questions in context.

Find other articles of interest and read their literature reviews carefully. I’ve included two more for you at the end of this chapter . As you learned how to diagram a sentence in elementary school (hopefully!), try diagramming the literature reviews. What are the “different strands” of research being discussed? How does the author connect these strands to their own research questions? Where is theory in the lit review, and how is it incorporated (e.g., Is it a separate strand of its own or is it inextricably linked with previous research in this area)?

One model of how to structure your literature review can be found in table 9.1. More tips, hints, and practices will be discussed later in the chapter.

Table 9.1. Model of Literature Review, Adopted from Calarco (2020:166)

Embracing Theory

A good research study will, in some form or another, use theory. Depending on your particular study (and possibly the preferences of the members of your committee), theory may be built into your literature review. Or it may form its own section in your research proposal/design (e.g., “literature review” followed by “theoretical framework”). In my own experience, I see a lot of graduate students grappling with the requirement to “include theory” in their research proposals. Things get a little squiggly here because there are different ways of incorporating theory into a study (Are you testing a theory? Are you generating a theory?), and based on these differences, your literature review proper may include works that describe, explain, and otherwise set forth theories, concepts, or frameworks you are interested in, or it may not do this at all. Sometimes a literature review sets forth what we know about a particular group or culture totally independent of what kinds of theoretical framework or particular concepts you want to explore. Indeed, the big point of your study might be to bring together a body of work with a theory that has never been applied to it previously. All this is to say that there is no one correct way to approach the use of theory and the writing about theory in your research proposal.

Students are often scared of embracing theory because they do not exactly understand what it is. Sometimes, it seems like an arbitrary requirement. You’re interested in a topic; maybe you’ve even done some research in the area and you have findings you want to report. And then a committee member reads over what you have and asks, “So what?” This question is a good clue that you are missing theory, the part that connects what you have done to what other researchers have done and are doing. You might stumble upon this rather accidentally and not know you are embracing theory, as in a case where you seek to replicate a prior study under new circumstances and end up finding that a particular correlation between behaviors only happens when mediated by something else. There’s theory in there, if you can pull it out and articulate it. Or it might be that you are motivated to do more research on racial microaggressions because you want to document their frequency in a particular setting, taking for granted the kind of critical race theoretical framework that has done the hard work of defining and conceptualizing “microaggressions” in the first place. In that case, your literature review could be a review of Critical Race Theory, specifically related to this one important concept. That’s the way to bring your study into a broader conversation while also acknowledging (and honoring) the hard work that has preceded you.

Rubin ( 2021 ) classifies ways of incorporating theory into case study research into four categories, each of which might be discussed somewhat differently in a literature review or theoretical framework section. The first, the least theoretical, is where you set out to study a “configurative idiographic case” ( 70 ) This is where you set out to describe a particular case, leaving yourself pretty much open to whatever you find. You are not expecting anything based on previous literature. This is actually pretty weak as far as research design goes, but it is probably the default for novice researchers. Your committee members should probably help you situate this in previous literature in some way or another. If they cannot, and it really does appear you are looking at something fairly new that no one else has bothered to research before, and you really are completely open to discovery, you might try using a Grounded Theory approach, which is a methodological approach that foregrounds the generation of theory. In that case, your “theory” section can be a discussion of “Grounded Theory” methodology (confusing, yes, but if you take some time to ponder, you will see how this works). You will still need a literature review, though. Ideally one that describes other studies that have ever looked at anything remotely like what you are looking at—parallel cases that have been researched.

The second approach is the “disciplined configurative case,” in which theory is applied to explain a particular case or topic. You are not trying to test the theory but rather assuming the theory is correct, as in the case of exploring microaggressions in a particular setting. In this case, you really do need to have a separate theory section in addition to the literature review, one in which you clearly define the theoretical framework, including any of its important concepts. You can use this section to discuss how other researchers have used the concepts and note any discrepancies in definitions or operationalization of those concepts. This way you will be sure to design your study so that it speaks to and with other researchers. If everyone who is writing about microaggressions has a different definition of them, it is hard for others to compare findings or make any judgments about their prevalence (or any number of other important characteristics). Your literature review section may then stand alone and describe previous research in the particular area or setting, irrespective of the kinds of theory underlying those studies.

The third approach is “heuristic,” one in which you seek to identify new variables, hypotheses, mechanisms, or paths not yet explained by a theory or theoretical framework. In a way, you are generating new theory, but it is probably more accurate to say that you are extending or deepening preexisting theory. In this case, having a single literature review that is focused on the theory and the ways the theory has been applied and understood (with all its various mechanisms and pathways) is probably your best option. The focus of the literature reviewed is less on the case and more on the theory you are seeking to extend.

The final approach is “theory testing,” which is much rarer in qualitative studies than in quantitative, where this is the default approach. Theory-testing cases are those where a particular case is used to see if an existing theory is accurate or accurate under particular circumstances. As with the heuristic approach, your literature review will probably draw heavily on previous uses of the theory, but you may end up having a special section specifically about cases very close to your own . In other words, the more your study approaches theory testing, the more likely there is to be a set of similar studies to draw on or even one important key study that you are setting your own study up in parallel to in order to find out if the theory generated there operates here.

If we wanted to get very technical, it might be useful to distinguish theoretical frameworks properly from conceptual frameworks. The latter are a bit looser and, given the nature of qualitative research, often fit exploratory studies. Theoretical frameworks rely on specific theories and are essential for theory-testing studies. Conceptual frameworks can pull in specific concepts or ideas that may or may not be linked to particular theories. Think about it this way: A theory is a story of how the world works. Concepts don’t presume to explain the whole world but instead are ways to approach phenomena to help make sense of them. Microaggressions are concepts that are linked to Critical Race Theory. One could contextualize one’s study within Critical Race Theory and then draw various concepts, such as that of microaggressions from the overall theoretical framework. Or one could bracket out the master theory or framework and employ the concept of microaggression more opportunistically as a phenomenon of interest. If you are unsure of what theory you are using, you might want to frame a more practical conceptual framework in your review of the literature.

Helpful Tips

How to maintain good notes for what your read.

Over the years, I have developed various ways of organizing notes on what I read. At first, I used a single sheet of full-size paper with a preprinted list of questions and points clearly addressed on the front side, leaving the second side for more reflective comments and free-form musings about what I read, why it mattered, and how it might be useful for my research. Later, I developed a system in which I use a single 4″ × 6″ note card for each book I read. I try only to use the front side (and write very small), leaving the back for comments that are about not just this reading but things to do or examine or consider based on the reading. These notes often mean nothing to anyone else picking up the card, but they make sense to me. I encourage you to find an organizing system that works for you. Then when you set out to compose a literature review, instead of staring at five to ten books or a dozen articles, you will have ten neatly printed pages or notecards or files that have distilled what is important to know about your reading.

It is also a good idea to store this data digitally, perhaps through a reference manager. I use RefWorks, but I also recommend EndNote or any other system that allows you to search institutional databases. Your campus library will probably provide access to one of these or another system. Most systems will allow you to export references from another manager if and when you decide to move to another system. Reference managers allow you to sort through all your literature by descriptor, author, year, and so on. Even so, I personally like to have the ability to manually sort through my index cards, recategorizing things I have read as I go. I use RefWorks to keep a record of what I have read, with proper citations, so I can create bibliographies more easily, and I do add in a few “notes” there, but the bulk of my notes are kept in longhand.

What kinds of information should you include from your reading? Here are some bulleted suggestions from Calarco ( 2020:113–114 ), with my own emendations:

• Citation . If you are using a reference manager, you can import the citation and then, when you are ready to create a bibliography, you can use a provided menu of citation styles, which saves a lot of time. If you’ve originally formatted in Chicago Style but the journal you are writing for wants APA style, you can change your entire bibliography in less than a minute. When using a notecard for a book, I include author, title, date as well as the library call number (since most of what I read I pull from the library). This is something RefWorks is not able to do, and it helps when I categorize.

I begin each notecard with an “intro” section, where I record the aims, goals, and general point of the book/article as explained in the introductory sections (which might be the preface, the acknowledgments, or the first two chapters). I then draw a bold line underneath this part of the notecard. Everything after that should be chapter specific. Included in this intro section are things such as the following, recommended by Calarco ( 2020 ):

• Key background . “Two to three short bullet points identifying the theory/prior research on which the authors are building and defining key terms.”
• Data/methods . “One or two short bullet points with information about the source of the data and the method of analysis, with a note if this is a novel or particularly effective example of that method.” I use [M] to signal methodology on my notecard, which might read, “[M] Int[erview]s (n-35), B[lack]/W[hite] voters” (I need shorthand to fit on my notecard!).
• Research question . “Stated as briefly as possible.” I always provide page numbers so I can go back and see exactly how this was stated (sometimes, in qualitative research, there are multiple research questions, and they cannot be stated simply).
• Argument/contributions . “Two to three short bullet points briefly describing the authors’ answer to the central research question and its implication for research, theory, and practice.” I use [ARG] for argument to signify the argument, and I make sure this is prominently visible on my notecard. I also provide page numbers here.

For me, all of this fits in the “intro” section, which, if this is a theoretically rich, methodologically sound book, might take up a third or even half of the front page of my notecard. Beneath the bold underline, I report specific findings or particulars of the book as they emerge chapter by chapter. Calarco’s ( 2020 ) next step is the following:

• Key findings . “Three to four short bullet points identifying key patterns in the data that support the authors’ argument.”

All that remains is writing down thoughts that occur upon finishing the article/book. I use the back of the notecard for these kinds of notes. Often, they reach out to other things I have read (e.g., “Robinson reminds me of Crusoe here in that both are looking at the effects of social isolation, but I think Robinson makes a stronger argument”). Calarco ( 2020 ) concludes similarly with the following:

• Unanswered questions . “Two to three short bullet points that identify key limitations of the research and/or questions the research did not answer that could be answered in future research.”

As I mentioned, when I first began taking notes like this, I preprinted pages with prompts for “research question,” “argument,” and so on. This was a great way to remind myself to look for these things in particular. You can do the same, adding whatever preprinted sections make sense to you, given what you are studying and the important aspects of your discipline. The other nice thing about the preprinted forms is that it keeps your writing to a minimum—you cannot write more than the allotted space, even if you might want to, preventing your notes from spiraling out of control. This can be helpful when we are new to a subject and everything seems worth recording!

After years of discipline, I have finally settled on my notecard approach. I have thousands of notecards, organized in several index card filing boxes stacked in my office. On the top right of each card is a note of the month/day I finished reading the item. I can remind myself what I read in the summer of 2010 if the need or desire ever arose to do so…those invaluable notecards are like a memento of what my brain has been up to!

Where to Start Looking for Literature

Your university library should provide access to one of several searchable databases for academic books and articles. My own preference is JSTOR, a service of ITHAKA, a not-for-profit organization that works to advance and preserve knowledge and to improve teaching and learning through the use of digital technologies. JSTOR allows you to search by several keywords and to narrow your search by type of material (articles or books). For many disciplines, the “literature” of the literature review is expected to be peer-reviewed “articles,” but some disciplines will also value books and book chapters. JSTOR is particularly useful for article searching. You can submit several keywords and see what is returned, and you can also narrow your search by a particular journal or discipline. If your discipline has one or two key journals (e.g., the American Journal of Sociology and the American Sociological Review are key for sociology), you might want to go directly to those journals’ websites and search for your topic area. There is an art to when to cast your net widely and when to refine your search, and you may have to tack back and forth to ensure that you are getting all that is relevant but not getting bogged down in all studies that might have some marginal relevance.

Some articles will carry more weight than others, and you can use applications like Google Scholar to see which articles have made and are continuing to make larger impacts on your discipline. Find these articles and read them carefully; use their literature review and the sources cited in those articles to make sure you are capturing what is relevant. This is actually a really good way of finding relevant books—only the most impactful will make it into the citations of journals. Over time, you will notice that a handful of articles (or books) are cited so often that when you see, say, Armstrong and Hamilton ( 2015 ), you know exactly what book this is without looking at the full cite. This is when you know you are in the conversation.

You might also approach a professor whose work is broadly in the area of your interest and ask them to recommend one or two “important” foundational articles or books. You can then use the references cited in those recommendations to build up your literature. Just be careful: some older professors’ knowledge of the literature (and I reluctantly add myself here) may be a bit outdated! It is best that the article or book whose references and sources you use to build your body of literature be relatively current.

Keep a List of Your Keywords

When using searchable databases, it is a good idea to keep a list of all the keywords you use as you go along so that (1) you do not needlessly duplicate your efforts and (2) you can more easily adjust your search as you get a better sense of what you are looking for. I suggest you keep a separate file or even a small notebook for this and you date your search efforts.

Here’s an example:

Table 9.2. Keep a List of Your Keywords

Think Laterally

How to find the various strands of literature to combine? Don’t get stuck on finding the exact same research topic you think you are interested in. In the female gymnast example, I recommended that my student consider looking for studies of ballerinas, who also suffer sports injuries and around whom there is a similar culture of silence. It turned out that there was in fact research about my student’s particular questions, just not about the subjects she was interested in. You might do something similar. Don’t get stuck looking for too direct literature but think about the broader phenomenon of interest or analogous cases.

Read Outside the Canon

Some scholars’ work gets cited by everyone all the time. To some extent, this is a very good thing, as it helps establish the discipline. For example, there are a lot of “Bourdieu scholars” out there (myself included) who draw ideas, concepts, and quoted passages from Bourdieu. This makes us recognizable to one another and is a way of sharing a common language (e.g., where “cultural capital” has a particular meaning to those versed in Bourdieusian theory). There are empirical studies that get cited over and over again because they are excellent studies but also because there is an “echo chamber effect” going on, where knowing to cite this study marks you as part of the club, in the know, and so on. But here’s the problem with this: there are hundreds if not thousands of excellent studies out there that fail to get appreciated because they are crowded out by the canon. Sometimes this happens because they are published in “lower-ranked” journals and are never read by a lot of scholars who don’t have time to read anything other than the “big three” in their field. Other times this happens because the author falls outside of the dominant social networks in the field and thus is unmentored and fails to get noticed by those who publish a lot in those highly ranked and visible spaces. Scholars who fall outside the dominant social networks and who publish outside of the top-ranked journals are in no way less insightful than their peers, and their studies may be just as rigorous and relevant to your work, so it is important for you to take some time to read outside the canon. Due to how a person’s race, gender, and class operate in the academy, there is also a matter of social justice and ethical responsibility involved here: “When you focus on the most-cited research, you’re more likely to miss relevant research by women and especially women of color, whose research tends to be under-cited in most fields. You’re also more likely to miss new research, research by junior scholars, and research in other disciplines that could inform your work. Essentially, it is important to read and cite responsibly, which means checking that you’re not just reading and citing the same white men and the same old studies that everyone has cited before you” ( Calarco 2020:112 ).

Consider Multiple Uses for Literature

Throughout this chapter, I’ve referred to the literature of interest in a rather abstract way, as what is relevant to your study. But there are many different ways previous research can be relevant to your study. The most basic use of the literature is the “findings”—for example, “So-and-so found that Canadian working-class students were concerned about ‘fitting in’ to the culture of college, and I am going to look at a similar question here in the US.” But the literature may be of interest not for its findings but theoretically—for example, employing concepts that you want to employ in your own study. Bourdieu’s definition of social capital may have emerged in a study of French professors, but it can still be relevant in a study of, say, how parents make choices about what preschools to send their kids to (also a good example of lateral thinking!).

If you are engaged in some novel methodological form of data collection or analysis, you might look for previous literature that has attempted that. I would not recommend this for undergraduate research projects, but for graduate students who are considering “breaking the mold,” find out if anyone has been there before you. Even if their study has absolutely nothing else in common with yours, it is important to acknowledge that previous work.

Describing Gaps in the Literature

First, be careful! Although it is common to explain how your research adds to, builds upon, and fills in gaps in the previous research (see all four literature review examples in this chapter for this), there is a fine line between describing the gaps and misrepresenting previous literature by failing to conduct a thorough review of the literature. A little humility can make a big difference in your presentation. Instead of “This is the first study that has looked at how firefighters juggle childcare during forest fire season,” say, “I use the previous literature on how working parents juggling childcare and the previous ethnographic studies of firefighters to explore how firefighters juggle childcare during forest fire season.” You can even add, “To my knowledge, no one has conducted an ethnographic study in this specific area, although what we have learned from X about childcare and from Y about firefighters would lead us to expect Z here.” Read more literature review sections to see how others have described the “gaps” they are filling.

Use Concept Mapping

Concept mapping is a helpful tool for getting your thoughts in order and is particularly helpful when thinking about the “literature” foundational to your particular study. Concept maps are also known as mind maps, which is a delightful way to think about them. Your brain is probably abuzz with competing ideas in the early stages of your research design. Write/draw them on paper, and then try to categorize and move the pieces around into “clusters” that make sense to you. Going back to the gymnasts example, my student might have begun by jotting down random words of interest: gymnasts * sports * coaches * female gymnasts * stress * injury * don’t complain * women in sports * bad coaching * anxiety/stress * careers in sports * pain. She could then have begun clustering these into relational categories (bad coaching, don’t complain culture) and simple “event” categories (injury, stress). This might have led her to think about reviewing literature in these two separate aspects and then literature that put them together. There is no correct way to draw a concept map, as they are wonderfully specific to your mind. There are many examples you can find online.

Ask Yourself, “How Is This Sociology (or Political Science or Public Policy, Etc.)?”

Rubin ( 2021:82 ) offers this suggestion instead of asking yourself the “So what?” question to get you thinking about what bridges there are between your study and the body of research in your particular discipline. This is particularly helpful for thinking about theory. Rubin further suggests that if you are really stumped, ask yourself, “What is the really big question that all [fill in your discipline here] care about?” For sociology, it might be “inequality,” which would then help you think about theories of inequality that might be helpful in framing your study on whatever it is you are studying—OnlyFans? Childcare during COVID? Aging in America? I can think of some interesting ways to frame questions about inequality for any of those topics. You can further narrow it by focusing on particular aspects of inequality (Gender oppression? Racial exclusion? Heteronormativity?). If your discipline is public policy, the big questions there might be, How does policy get enacted, and what makes a policy effective? You can then take whatever your particular policy interest is—tax reform, student debt relief, cap-and-trade regulations—and apply those big questions. Doing so would give you a handle on what is otherwise an intolerably vague subject (e.g., What about student debt relief?).

Sometimes finding you are in new territory means you’ve hit the jackpot, and sometimes it means you’ve traveled out of bounds for your discipline. The jackpot scenario is wonderful. You are doing truly innovative research that is combining multiple literatures or is addressing a new or under-examined phenomenon of interest, and your research has the potential to be groundbreaking. Congrats! But that’s really hard to do, and it might be more likely that you’ve traveled out of bounds, by which I mean, you are no longer in your discipline . It might be that no one has written about this thing—at least within your field— because no one in your field actually cares about this topic . ( Rubin 2021:83 ; emphases added)

Don’t Treat This as a Chore

Don’t treat the literature review as a chore that has to be completed, but see it for what it really is—you are building connections to other researchers out there. You want to represent your discipline or area of study fairly and adequately. Demonstrate humility and your knowledge of previous research. Be part of the conversation.

Supplement: Two More Literature Review Examples

Elites by harvey ( 2011 ).

In the last two decades, there has been a small but growing literature on elites. In part, this has been a result of the resurgence of ethnographic research such as interviews, focus groups, case studies, and participant observation but also because scholars have become increasingly interested in understanding the perspectives and behaviors of leaders in business, politics, and society as a whole. Yet until recently, our understanding of some of the methodological challenges of researching elites has lagged behind our rush to interview them.

There is no clear-cut definition of the term elite, and given its broad understanding across the social sciences, scholars have tended to adopt different approaches. Zuckerman (1972) uses the term ultraelites to describe individuals who hold a significant amount of power within a group that is already considered elite. She argues, for example, that US senators constitute part of the country’s political elite but that among them are the ultraelites: a “subset of particularly powerful or prestigious influentials” (160). She suggests that there is a hierarchy of status within elite groups. McDowell (1998) analyses a broader group of “professional elites” who are employees working at different levels for merchant and investment banks in London. She classifies this group as elite because they are “highly skilled, professionally competent, and class-specific” (2135). Parry (1998:2148) uses the term hybrid elites in the context of the international trade of genetic material because she argues that critical knowledge exists not in traditional institutions “but rather as increasingly informal, hybridised, spatially fragmented, and hence largely ‘invisible,’ networks of elite actors.” Given the undertheorization of the term elite, Smith (2006) recognizes why scholars have shaped their definitions to match their respondents . However, she is rightly critical of the underlying assumption that those who hold professional positions necessarily exert as much influence as initially perceived. Indeed, job titles can entirely misrepresent the role of workers and therefore are by no means an indicator of elite status (Harvey 2010).

Many scholars have used the term elite in a relational sense, defining them either in terms of their social position compared to the researcher or compared to the average person in society (Stephens 2007). The problem with this definition is there is no guarantee that an elite subject will necessarily translate this power and authority in an interview setting. Indeed, Smith (2006) found that on the few occasions she experienced respondents wanting to exert their authority over her, it was not from elites but from relatively less senior workers. Furthermore, although business and political elites often receive extensive media training, they are often scrutinized by television and radio journalists and therefore can also feel threatened in an interview, particularly in contexts that are less straightforward to prepare for such as academic interviews. On several occasions, for instance, I have been asked by elite respondents or their personal assistants what they need to prepare for before the interview, which suggests that they consider the interview as some form of challenge or justification for what they do.

In many cases, it is not necessarily the figureheads or leaders of organizations and institutions who have the greatest claim to elite status but those who hold important social networks, social capital, and strategic positions within social structures because they are better able to exert influence (Burt 1992; Parry 1998; Smith 2005; Woods 1998). An elite status can also change, with people both gaining and losing theirs over time. In addition, it is geographically specific, with people holding elite status in some but not all locations. In short, it is clear that the term elite can mean many things in different contexts, which explains the range of definitions. The purpose here is not to critique these other definitions but rather to highlight the variety of perspectives.

When referring to my research, I define elites as those who occupy senior-management- and board-level positions within organizations. This is a similar scope of definition to Zuckerman’s (1972) but focuses on a level immediately below her ultraelite subjects. My definition is narrower than McDowell’s (1998) because it is clear in the context of my research that these people have significant decision-making influence within and outside of the firm and therefore present a unique challenge to interview. I deliberately use the term elite more broadly when drawing on examples from the theoretical literature in order to compare my experiences with those who have researched similar groups.

”Changing Dispositions among the Upwardly Mobile” by Curl, Lareau, and Wu ( 2018 )

There is growing interest in the role of cultural practices in undergirding the social stratification system. For example, Lamont et al. (2014) critically assess the preoccupation with economic dimensions of social stratification and call for more developed cultural models of the transmission of inequality. The importance of cultural factors in the maintenance of social inequality has also received empirical attention from some younger scholars, including Calarco (2011, 2014) and Streib (2015). Yet questions remain regarding the degree to which economic position is tied to cultural sensibilities and the ways in which these cultural sensibilities are imprinted on the self or are subject to change. Although habitus is a core concept in Bourdieu’s theory of social reproduction, there is limited empirical attention to the precise areas of the habitus that can be subject to change during upward mobility as well as the ramifications of these changes for family life.

In Bourdieu’s (1984) highly influential work on the importance of class-based cultural dispositions, habitus is defined as a “durable system of dispositions” created in childhood. The habitus provides a “matrix of perceptions” that seems natural while also structuring future actions and pathways. In many of his writings, Bourdieu emphasized the durability of cultural tastes and dispositions and did not consider empirically whether these dispositions might be changed or altered throughout one’s life (Swartz 1997). His theoretical work does permit the possibility of upward mobility and transformation, however, through the ability of the habitus to “improvise” or “change” due to “new experiences” (Friedman 2016:131). Researchers have differed in opinion on the durability of the habitus and its ability to change (King 2000). Based on marital conflict in cross-class marriages, for instance, Streib (2015) argues that cultural dispositions of individuals raised in working-class families are deeply embedded and largely unchanging. In a somewhat different vein, Horvat and Davis (2011:152) argue that young adults enrolled in an alternative educational program undergo important shifts in their self-perception, such as “self-esteem” and their “ability to accomplish something of value.” Others argue there is variability in the degree to which habitus changes dependent on life experience and personality (Christodoulou and Spyridakis 2016). Recently, additional studies have investigated the habitus as it intersects with lifestyle through the lens of meaning making (Ambrasat et al. 2016). There is, therefore, ample discussion of class-based cultural practices in self-perception (Horvat and Davis 2011), lifestyle (Ambrasat et al. 2016), and other forms of taste (Andrews 2012; Bourdieu 1984), yet researchers have not sufficiently delineated which aspects of the habitus might change through upward mobility or which specific dimensions of life prompt moments of class-based conflict.

Bourdieu (1999:511; 2004) acknowledged simmering tensions between the durable aspects of habitus and those aspects that have been transformed—that is, a “fractured” or “cleft” habitus. Others have explored these tensions as a “divided” or “fragmented” habitus (Baxter and Britton 2001; Lee and Kramer 2013). Each of these conceptions of the habitus implies that changes in cultural dispositions are possible but come with costs. Exploration of the specific aspects of one’s habitus that can change and generate conflict contributes to this literature.

Scholars have also studied the costs associated with academic success for working-class undergraduates (Hurst 2010; Lee and Kramer 2013; London 1989; Reay 2017; Rondini 2016; Stuber 2011), but we know little about the lasting effects on adults. For instance, Lee and Kramer (2013) point to cross-class tensions as family and friends criticize upwardly mobile individuals for their newly acquired cultural dispositions. Documenting the tension many working-class students experience with their friends and families of origin, they find that the source of their pain or struggle is “shaped not only by their interactions with non-mobile family and friends but also within their own minds, by their own assessments of their social positions, and by how those positions are interpreted by others” (Lee and Kramer 2013:29). Hurst (2010) also explores the experiences of undergraduates who have been academically successful and the costs associated with that success. She finds that decisions about “class allegiance and identity” are required aspects of what it means to “becom[e] educated” (4) and that working-class students deal with these cultural changes differently. Jack (2014, 2016) also argues that there is diversity among lower-income students, which yields varied college experiences. Naming two groups, the “doubly disadvantaged” and the “privileged poor,” he argues that previous experience with “elite environments” (2014:456) prior to college informs students’ ability to take on dominant cultural practices, particularly around engagement, such as help seeking or meeting with professors (2016). These studies shed light on the role college might play as a “lever for mobility” (2016:15) and discuss the pain and difficulty associated with upward mobility among undergraduates, but the studies do not illuminate how these tensions unfold in adulthood. Neither have they sufficiently addressed potential enduring tensions with extended family members as well as the specific nature of the difficulties.

Some scholars point to the positive outcomes upwardly mobile youth (Lehmann 2009) and adults (Stuber 2005) experience when they maintain a different habitus than their newly acquired class position, although, as Jack (2014, 2016) shows, those experiences may vary depending on one’s experience with elite environments in their youth. Researchers have not sufficiently explored the specific aspects of the habitus that upwardly mobile adults change or the conflicts that emerge with family and childhood friends as they reach adulthood and experience colliding social worlds. We contribute to this scholarship with clear examples of self-reported changes to one’s cultural dispositions in three specific areas: “horizons,” food and health, and communication. We link these changes to enduring tension with family members, friends, and colleagues and explore varied responses to this tension based on race.

Further Readings

Bloomberg, Linda Dale, and Marie F. Volpe. 2012. Completing Your Qualitative Dissertation: A Road Map from Beginning to End . 2nd ed. Thousand Oaks, CA: SAGE. In keeping with its general approach to qualitative research, includes a “road map” for conducting a literature review.

Hart, Chris. 1998. Doing a Literature Review: Releasing the Social Science Research Imagination . London: SAGE. A how-to book dedicated entirely to conducting a literature review from a British perspective. Useful for both undergraduate and graduate students.

Machi, Lawrence A., and Brenda T. McEvoy. 2022. The Literature Review: Six Steps to Success . 4th ed. Newbury Park, CA: Corwin. A well-organized guidebook complete with reflection sections to prompt successful thinking about your literature review.

Ridley, Diana. 2008. The Literature Review: A Step-by-Step Guide for Students . London: SAGE. A highly recommended companion to conducting a literature review for doctoral-level students.

The process of systematically searching through pre-existing studies (“literature”) on the subject of research; also, the section of a presentation in which the pre-existing literature is discussed.

Follow-up questions used in a semi-structured interview  to elicit further elaboration.  Suggested prompts can be included in the interview guide  to be used/deployed depending on how the initial question was answered or if the topic of the prompt does not emerge spontaneously.

A tool for identifying relationships among ideas by visually representing them on paper.  Most concept maps depict ideas as boxes or circles (also called nodes), which are structured hierarchically and connected with lines or arrows (also called arcs). These lines are labeled with linking words and phrases to help explain the connections between concepts.  Also known as mind mapping.

The people who are the subjects of an interview-based qualitative study. In general, they are also known as the participants, and for purposes of IRBs they are often referred to as the human subjects of the research.

Introduction to Qualitative Research Methods Copyright © 2023 by Allison Hurst is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License , except where otherwise noted.

Dissertations and research projects

• Book a session
• Planning your research

Developing a theoretical framework

Reflecting on your position, extended literature reviews, presenting qualitative data.

• Quantitative research
• Writing up your research project
• e-learning and books
• SkillsCheck This link opens in a new window
• ⬅ Back to Skills Centre This link opens in a new window
• Review this resource

What is a theoretical framework?

Developing a theoretical framework for your dissertation is one of the key elements of a qualitative research project. Through writing your literature review, you are likely to have identified either a problem that need ‘fixing’ or a gap that your research may begin to fill.

The theoretical framework is your toolbox . In the toolbox are your handy tools: a set of theories, concepts, ideas and hypotheses that you will use to build a solution to the research problem or gap you have identified.

The methodology is the instruction manual: the procedure and steps you have taken, using your chosen tools, to tackle the research problem.

Why do I need a theoretical framework?

Developing a theoretical framework shows that you have thought critically about the different ways to approach your topic, and that you have made a well-reasoned and evidenced decision about which approach will work best. theoretical frameworks are also necessary for solving complex problems or issues from the literature, showing that you have the skills to think creatively and improvise to answer your research questions. they also allow researchers to establish new theories and approaches, that future research may go on to develop., how do i create a theoretical framework for my dissertation.

First, select your tools. You are likely to need a variety of tools in qualitative research – different theories, models or concepts – to help you tackle different parts of your research question.  

When deciding what tools would be best for the job of answering your research questions or problem, explore what existing research in your area has used. You may find that there is a ‘standard toolbox’ for qualitative research in your field that you can borrow from or apply to your own research.

You will need to justify why your chosen tools are best for the job of answering your research questions, at what stage they are most relevant, and how they relate to each other. Some theories or models will neatly fit together and appear in the toolboxes of other researchers. However, you may wish to incorporate a model or idea that is not typical for your research area – the ‘odd one out’ in your toolbox. If this is the case, make sure you justify and account for why it is useful to you, and look for ways that it can be used in partnership with the other tools you are using.

You should also be honest about limitations, or where you need to improvise (for example, if the ‘right’ tool or approach doesn’t exist in your area).

This video from the Skills Centre includes an overview and example of how you might create a theoretical framework for your dissertation:

How do I choose the 'right' approach?

When designing your framework and choosing what to include, it can often be difficult to know if you’ve chosen the ‘right’ approach for your research questions. One way to check this is to look for consistency between your objectives, the literature in your framework, and your overall ethos for the research. This means ensuring that the literature you have used not only contributes to answering your research objectives, but that you also use theories and models that are true to your beliefs as a researcher.

Reflecting on your values and your overall ambition for the project can be a helpful step in making these decisions, as it can help you to fully connect your methodology and methods to your research aims.

Should I reflect on my position as a researcher?

If you feel your position as a researcher has influenced your choice of methods or procedure in any way, the methodology is a good place to reflect on this.  Positionality  acknowledges that no researcher is entirely objective: we are all, to some extent, influenced by prior learning, experiences, knowledge, and personal biases. This is particularly true in qualitative research or practice-based research, where the student is acting as a researcher in their own workplace, where they are otherwise considered a practitioner/professional. It's also important to reflect on your positionality if you belong to the same community as your participants where this is the grounds for their involvement in the research (ie. you are a mature student interviewing other mature learners about their experences in higher education). 

The following questions can help you to reflect on your positionality and gauge whether this is an important section to include in your dissertation (for some people, this section isn’t necessary or relevant):

• How might my personal history influence how I approach the topic?
• How am I positioned in relation to this knowledge? Am I being influenced by prior learning or knowledge from outside of this course?
• How does my gender/social class/ ethnicity/ culture influence my positioning in relation to this topic?
• Do I share any attributes with my participants? Are we part of a s hared community? How might this have influenced our relationship and my role in interviews/observations?
• Am I invested in the outcomes on a personal level? Who is this research for and who will feel the benefits?

One option for qualitative projects is to write an extended literature review. This type of project does not require you to collect any new data. Instead, you should focus on synthesising a broad range of literature to offer a new perspective on a research problem or question.  

The main difference between an extended literature review and a dissertation where primary data is collected, is in the presentation of the methodology, results and discussion sections. This is because extended literature reviews do not actively involve participants or primary data collection, so there is no need to outline a procedure for data collection (the methodology) or to present and interpret ‘data’ (in the form of interview transcripts, numerical data, observations etc.) You will have much more freedom to decide which sections of the dissertation should be combined, and whether new chapters or sections should be added.

Here is an overview of a common structure for an extended literature review:

Introduction

• Provide background information and context to set the ‘backdrop’ for your project.
• Explain the value and relevance of your research in this context. Outline what do you hope to contribute with your dissertation.
• Clarify a specific area of focus.
• Introduce your research aims (or problem) and objectives.

Literature review

You will need to write a short, overview literature review to introduce the main theories, concepts and key research areas that you will explore in your dissertation. This set of texts – which may be theoretical, research-based, practice-based or policies – form your theoretical framework. In other words, by bringing these texts together in the literature review, you are creating a lens that you can then apply to more focused examples or scenarios in your discussion chapters.

Methodology

As you will not be collecting primary data, your methodology will be quite different from a typical dissertation. You will need to set out the process and procedure you used to find and narrow down your literature. This is also known as a search strategy.

Including your search strategy

A search strategy explains how you have narrowed down your literature to identify key studies and areas of focus. This often takes the form of a search strategy table, included as an appendix at the end of the dissertation. If included, this section takes the place of the traditional 'methodology' section.

If you choose to include a search strategy table, you should also give an overview of your reading process in the main body of the dissertation.  Think of this as a chronology of the practical steps you took and your justification for doing so at each stage, such as:

• Your key terms, alternatives and synonyms, and any terms that you chose to exclude.
• Your choice and combination of databases;
• Your inclusion/exclusion criteria, when they were applied and why. This includes filters such as language of publication, date, and country of origin;
• You should also explain which terms you combined to form search phrases and your use of Boolean searching (AND, OR, NOT);
• Your use of citation searching (selecting articles from the bibliography of a chosen journal article to further your search).
• Your use of any search models, such as PICO and SPIDER to help shape your approach.
• Search strategy template A simple template for recording your literature searching. This can be included as an appendix to show your search strategy.

The discussion section of an extended literature review is the most flexible in terms of structure. Think of this section as a series of short case studies or ‘windows’ on your research. In this section you will apply the theoretical framework you formed in the literature review – a combination of theories, models and ideas that explain your approach to the topic – to a series of different examples and scenarios. These are usually presented as separate discussion ‘chapters’ in the dissertation, in an order that you feel best fits your argument.

Think about an order for these discussion sections or chapters that helps to tell the story of your research. One common approach is to structure these sections by common themes or concepts that help to draw your sources together. You might also opt for a chronological structure if your dissertation aims to show change or development over time. Another option is to deliberately show where there is a lack of chronology or narrative across your case studies, by ordering them in a fragmentary order! You will be able to reflect upon the structure of these chapters elsewhere in the dissertation, explaining and defending your decision in the methodology and conclusion.

A summary of your key findings – what you have concluded from your research, and how far you have been able to successfully answer your research questions.

• Recommendations – for improvements to your own study, for future research in the area, and for your field more widely.
• Emphasise your contributions to knowledge and what you have achieved.

Alternative structure

Depending on your research aims, and whether you are working with a case-study type approach (where each section of the dissertation considers a different example or concept through the lens established in your literature review), you might opt for one of the following structures:

Splitting the literature review across different chapters:

This structure allows you to pull apart the traditional literature review, introducing it little by little with each of your themed chapters. This approach works well for dissertations that attempt to show change or difference over time, as the relevant literature for that section or period can be introduced gradually to the reader.

Whichever structure you opt for, remember to explain and justify your approach. A marker will be interested in why you decided on your chosen structure, what it allows you to achieve/brings to the project and what alternatives you considered and rejected in the planning process. Here are some example sentence starters:

In qualitative studies, your results are often presented alongside the discussion, as it is difficult to include this data in a meaningful way without explanation and interpretation. In the dsicussion section, aim to structure your work thematically, moving through the key concepts or ideas that have emerged from your qualitative data. Use extracts from your data collection - interviews, focus groups, observations - to illustrate where these themes are most prominent, and refer back to the sources from your literature review to help draw conclusions. 

Here's an example of how your data could be presented in paragraph format in this section:

Example from  'Reporting and discussing your findings ', Monash University .

• << Previous: Planning your research
• Next: Quantitative research >>
• Last Updated: Apr 5, 2024 11:10 AM
• URL: https://libguides.shu.ac.uk/researchprojects

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

A Guide to Writing a Qualitative Systematic Review Protocol to Enhance Evidence-Based Practice in Nursing and Health Care

Affiliations.

• 1 PhD candidate, School of Nursing and Midwifey, Monash University, and Clinical Nurse Specialist, Adult and Pediatric Intensive Care Unit, Monash Health, Melbourne, Victoria, Australia.
• 2 Lecturer, School of Nursing and Midwifery, Monash University, Melbourne, Victoria, Australia.
• 3 Senior Lecturer, School of Nursing and Midwifery, Monash University, Melbourne, Victoria, Australia.
• PMID: 26790142
• DOI: 10.1111/wvn.12134

Background: The qualitative systematic review is a rapidly developing area of nursing research. In order to present trustworthy, high-quality recommendations, such reviews should be based on a review protocol to minimize bias and enhance transparency and reproducibility. Although there are a number of resources available to guide researchers in developing a quantitative review protocol, very few resources exist for qualitative reviews.

Aims: To guide researchers through the process of developing a qualitative systematic review protocol, using an example review question.

Methodology: The key elements required in a systematic review protocol are discussed, with a focus on application to qualitative reviews: Development of a research question; formulation of key search terms and strategies; designing a multistage review process; critical appraisal of qualitative literature; development of data extraction techniques; and data synthesis. The paper highlights important considerations during the protocol development process, and uses a previously developed review question as a working example.

Implications for research: This paper will assist novice researchers in developing a qualitative systematic review protocol. By providing a worked example of a protocol, the paper encourages the development of review protocols, enhancing the trustworthiness and value of the completed qualitative systematic review findings.

Linking evidence to action: Qualitative systematic reviews should be based on well planned, peer reviewed protocols to enhance the trustworthiness of results and thus their usefulness in clinical practice. Protocols should outline, in detail, the processes which will be used to undertake the review, including key search terms, inclusion and exclusion criteria, and the methods used for critical appraisal, data extraction and data analysis to facilitate transparency of the review process. Additionally, journals should encourage and support the publication of review protocols, and should require reference to a protocol prior to publication of the review results.

Keywords: guidelines; meta synthesis; qualitative; systematic review protocol.

© 2016 Sigma Theta Tau International.

• Evidence-Based Practice / standards*
• Information Seeking Behavior
• Nursing / methods
• Qualitative Research*
• Research Design / standards*
• Systematic Reviews as Topic*
• Writing / standards*

Literature Review

Literature review is important because it:

• Provides ideas about what should be studied;
• Helps us conduct inquires that have not already been done
• Connects our research to existing studies

But…doing a literature review is not simply summarizing (or copying) what you think is related and useful to your work. BEING CRITICAL AND CAREFUL IS A MUST !

In reviewing existing literature, you may try to look for gaps in the field and rework your study in a different setting or with different people. Nonetheless, literature review is a continuous sense-making process -- you need to review the literature continuously in order to organize your thoughts and refine your analysis.

A good literature review should be able to:  

• Connect to your research questions
• Connect to your choice of methods and research design
• Support your data analysis
• Help you draw conclusions and make claims about your research.

Selecting your literature with a purpose

It is impossible to read everything, so when selecting literature  for reviewing, consider these:

• Is it relevant to your topic/field of study?
• Is it a primary source from the researcher(s) or secondary source (e.g. a summary you read in a book about someone’s research)?
• Is it updated?

Nature of literatures:

Your literature review can be of different dimensions. Each has its foci and purposes

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• Search Menu
• Advance Articles
• Editor's Choice
• Author Guidelines
• Submission Site
• Open Access
• About Journal of Public Administration Research and Theory
• About the Public Management Research Association
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, key claims in administrative burden research, characteristics of studies on administrative burden, qualitative analysis of key causal relationships, setting an agenda for future research, supplementary material, acknowledgment, data availability.

• < Previous

Administrative Burden in Citizen–State Interactions: A Systematic Literature Review

• Article contents
• Figures & tables
• Supplementary Data

Aske Halling, Martin Baekgaard, Administrative Burden in Citizen–State Interactions: A Systematic Literature Review, Journal of Public Administration Research and Theory , Volume 34, Issue 2, April 2024, Pages 180–195, https://doi.org/10.1093/jopart/muad023

• Permissions Icon Permissions

Based on a systematic review of 119 articles and working papers, we provide an overview of how administrative burdens in citizen–state interactions have been studied since the inception of the research agenda in 2012. We develop a new and comprehensive model of how key concepts in the framework are related, assess the evidence of the causal relationships proposed by the model, and discuss where more evidence is needed. Empirical research supports conventional claims that burdens are consequential, distributive, and constructed. However, the literature has moved further by (1) demonstrating that factors such as frontline service delivery and government communication influence experiences of burdens; (2) highlighting how factors beyond ideology influence constructions of burdens; (3) introducing the burden tolerance concept; (4) illustrating that experiences of burden influence policymakers’ and members of the publics’ burden tolerance. Based on the review, we propose an agenda for future administrative burden research. We call for studies linking experiences of burden to outcomes such as democratic behavior and take-up, and for studies connecting policymakers’ burden tolerance to actual state actions. Moreover, we argue that future studies should use qualitative methods to further explore the nature of burdens from the perspective of citizens, rely on experimental methods to establish causal links between state actions and experiences of burden, and compare burdens across contexts. Further, empirical studies should examine the tradeoffs between legitimacy and experiences of burden, and how actors outside the citizen–state interaction may influence experiences of administrative burden.

Administrative burden is defined as an individual’s experiences of policy implementation as onerous ( Burden et al. 2012 ). The concept thus emphasizes the experiences of individuals and how state actions, in the form of policies and how they are implemented in practice, influence said experiences ( Baekgaard and Tankink 2022 ). In principle, the definition applies to any individual subject to policy implementation ( Madsen, Mikkelsen, and Moynihan 2022 , 7–8), but the concept has particularly been used in the context of citizen–state interactions ( Jakobsen et al. 2016 ).

Building on research traditions on, among others, take-up of policies and benefits ( Bhargava and Manoli 2015 ; Currie 2006 ), policy feedback ( Moynihan and Soss 2014 ; Soss 1999 ), street-level bureaucracy ( Brodkin and Majmundar 2010 ; Lipsky 1980 ), and red tape ( Bozeman and Youtie 2020 ) that all draw attention to onerous experiences with the state, administrative burden has been showcased as an important concept to create an overarching framework to understand such experiences.

However, we lack a comprehensive overview of how the field has studied administrative burden since the introduction of the concept in the seminal articles by Burden et al. (2012) and Moynihan, Herd, and Harvey (2015) , and how various research questions relate to one another. Even though the standard definition of administrative burden points to individual experiences, scholars in practice refer to different phenomena when studying administrative burden. Some focus on actions made by the state (i.e., “objective” burdens), some focus on individuals’ subjective perceptions, and some focus on individual outcomes, such as take-up of benefits or health ( Baekgaard and Tankink 2022 ). Moreover, research foci differ. Some studies focus on understanding individual experiences and outcomes and how negative experiences and outcomes can be reduced, while others focus on why policies and practices associated with burdensome experiences are enacted by policymakers or how they are implemented at the frontline.

To take stock of the current state of administrative burden research and to better connect empirical knowledge and research questions in current research, we conduct a systematic review of 119 published articles and working papers focusing on administrative burdens in citizen–state interactions. We limit our sample to papers specifically claiming to draw on this framework, that is, studies published between the inception of the concept and framework in Burden et al. (2012) and Moynihan, Herd, and Harvey (2015) and the beginning of 2023. To ensure reproducibility and transparency, we follow the widely used PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines ( Page et al. 2021 ).

There have been a number of theoretical and conceptual articles and literature reviews about administrative burdens ( Baekgaard and Tankink 2022 ; Campbell, Pandey, and Arnesen 2022 ; Madsen, Mikkelsen, and Moynihan 2022 ; Peeters 2020 ). However, none of these articles have systematically covered all studies on the topic or taken up the task of connecting different streams of administrative burden research in a theoretical model. Our systematic review therefore makes two contributions to administrative burden research.

The first contribution is theoretical. Using a qualitative coding of the articles and papers included in our review, we build a theoretical model of how key concepts in the administrative burden causally relate to each other. This model is based partly on theoretical arguments in the literature, partly on empirical evidence, and seeks to connect studies of very different research questions within administrative burden research to create a coherent theoretical framework. The aim is not to make a parsimonious theoretical claim but rather to construct a model of the many antecedents, moderators, and potential consequences of administrative burden experiences identified in this literature.

The second contribution is an overview of how administrative burdens in citizen–state interactions have been studied to date. We describe the methodological and contextual characteristics of the studies included in the review and connect key concepts in the administrative burden framework to identify areas of inquiry where substantial progress has been made and to point to areas where future studies could best be directed.

The next section briefly discusses the concept of administrative burden and key causal claims in the administrative burden literature. In the methods section, we describe our literature search, criteria for including studies in the review, and how studies were coded and analyzed. This is followed by our qualitative analysis of the literature. We start out by presenting a model based on the review and then classify the evidence for seven causal claims in the model. The article concludes with a discussion of limitations and proposals for a future research agenda.

Administrative burden refers to the learning, compliance, and psychological costs experienced by citizens when interacting with the state ( Herd and Moynihan 2018 ). Learning costs are the costs of learning about rights, rules, and demands associated with interacting with the state ( Barnes and Riel 2022 ). For instance, an individual eligible for the TANF program in the United States has to be aware that the program exists and how to apply for the benefits. Compliance costs are the costs of complying with specific rules in interactions with the state. In the TANF example, the applicant has to fill out an application form and demonstrate eligibility. For the unemployed, compliance costs may manifest as the costs of having to show up for meetings at public offices to demonstrate an active search for work and of updating CVs on a regular basis ( Baekgaard et al. 2021 ; Madsen nda ). Finally, psychological costs have to do with the mental discomfort of interacting with the government ( Hattke, Hensel, and Kalucza 2020 ). For instance, interactions associated with uncertainty may lead to experiences of stress, loss of autonomy, or even stigma ( Cecchini nd ).

By emphasizing the subjective costs experienced by citizens and explicitly referring to individual experiences in the definition of the concept ( Burden et al. 2012 ), the administrative burden literature makes a key distinction between what the state does (sometimes called barriers, frictions, state actions, or state constructions of burdens) and what the individual experiences. However, this does not mean that the literature is only interested in the impact of experiences. Rather, the literature makes three key claims about burdens ( Herd and Moynihan 2018 ; Moynihan, Herd, and Harvey 2015 ) regarding what the state does, how citizens experience the actions of the state, individual differences in said experiences, and the consequences of burdensome experiences. Figure 1 summarizes the claims in a simplified model (see Baekgaard and Tankink 2022 ; Christensen et al. 2020 ; Herd and Moynihan 2018 for similar models).

Key Claims in the Administrative Burden Literature.

First, burdens are consequential. The impact of burdens is likely to extend beyond people’s experiences and influence outcomes such as civic and electoral participation, health, and take-up of benefits. This claim is rooted very much in literatures on policy feedback, benefit take-up, and applied economics. These research traditions empirically demonstrate that aspects of what the state does may have important impacts for the mobilization of citizens (e.g., Bruch, Ferree, and Soss 2010 ; Soss 1999 ), the extent to which target groups take up services and benefits for which they are eligible (e.g., Currie 2006 ), and the long-run health of citizens enrolled in welfare programs (e.g., Hoynes, Schanzenbach, and Almond 2016 ). However, they do less to explore people’s subjective experiences of burden. In this respect, the administrative burden framework contributes to previous streams of research by creating a language for the mechanisms linking state actions to outcomes.

Second, burdens have distributive consequences and are likely to fall harder on those with fewer resources in the form of human and administrative capital ( Christensen et al. 2020 ; Masood and Nisar 2021 ), administrative literacy ( Döring 2021 ), and bureaucratic self-efficacy ( Bisgaard 2023 ). Third, while some burdensome state actions are likely unintended ( Peeters and Widlak 2023 ), or the result of unconscious biases ( Olsen, Kyhse-Andersen, and Moynihan 2020 ), other state actions are constructed and the result of deliberate political and administrative decisions, that is, politicians or bureaucrats prefer to introduce burdens to, for instance, limit fraud in public programs ( Moynihan, Herd, and Ribgy 2016 ).

Thus, the literature relies on a broad understanding of the relevant domain of inquiry. This domain is not limited to the study of experiences and outcomes among citizens and target groups. As per the third causal claim above, it also encompasses decisions and rationales for decisions made by elected politicians, administrators, and frontline personnel. In our review, we therefore rely on a broad understanding of the domain of administrative burden in citizen–state interactions where studies are relevant whenever the subject matter has to do with a state arrangement introducing burden for citizens. The studies may focus either on decisions made by politicians or bureaucrats or on the consequences of such decisions for citizens. In our analysis of the literature, we seek to develop the model presented in figure 1 further by reviewing the empirical findings in existing studies.

We adhere to the PRISMA guidelines when conducting our systematic literature review ( Page et al. 2021 ). These guidelines were developed to ensure that literature reviews are comprehensive, transparent, and well documented to minimize reporting biases and ensure reproducibility. The PRISMA checklist is available in the appendix . Below we describe the eligibility criteria for inclusion in the review as well as our search and coding strategy.

Eligibility Criteria

Our focus is on administrative burdens in citizen–state interactions. The main inclusion criterion is that studies use the conceptual framework formulated by Burden et al. (2012) , Moynihan, Herd, and Harvey (2015) , and Herd and Moynihan (2018) , that is, refer explicitly to administrative burden and/or learning, compliance, and psychological costs. Other streams of literature in economics, sociology, political science, and public administration also deal with frictions in interactions between citizens and government. This includes, but is not limited to, literatures on red tape, sludge, ordeals, take-up of government benefits, street-level bureaucracy, and policy feedback ( Baekgaard and Tankink 2022 ; Madsen, Mikkelsen, and Moynihan 2022 ). However, as administrative burden has developed into a sizeable subfield of its own, which in several aspects differs from related research in other disciplines ( Madsen, Mikkelsen, and Moynihan 2022 ), it is important to take stock of the current state of this particular field and explore what the literature has taught us so far.

The second inclusion criterion is that studies focus on administrative burdens in citizen–state interactions. This means that we exclude studies that use the administrative burden framework but focus either on companies ( Petersen, Hansen, and Houlberg 2022 ), third-party organizations ( Carey et al. 2020 ), or on the costs experienced by public employees in their interactions with the organization where they are employed ( Bozeman and Youtie 2020 ; Linos and Riesch 2020 ; Sievert, Vogel, and Feeney 2020 ). We make this decision because our goal is to understand how, why, and when citizens experience burdens in their interactions with the state. In comparison, studies on public employees burdened by work routines focus on internal organizational affairs rather than a bureaucratic relationship between the state and individual outside the formal organizational hierarchy. Also, burdens among public employees have been studied extensively in the red tape literature ( George et al. 2020 ). Nevertheless, the review still includes studies where elected politicians and frontline personnel were asked about the imposition of burdens on individuals outside the organization. Thus, the review applies a broad understanding of citizens as individuals and organizations outside the formal organizational hierarchy of the state in a given case.

The remaining inclusion criteria are more straightforward. We are interested in all English-language peer-reviewed publications and working papers from 2012 until our data collection closed in February 2023. 1 We set the start to 2012, because this is when Burden et al. (2012) wrote their seminal article that introduced and defined the term “administrative burdens.” Table 1 gives an overview of the eligibility criteria.

Overview of Eligibility Criteria

Literature Search

To identify peer-reviewed journal articles, we searched all journals in the Social Sciences Citation Index using Web of Science. We searched titles, abstracts, and keywords for “administrative burden,” “psychological cost,” “compliance cost,” “learning cost,” and derivatives of these terms. We limited our search to English-language articles. We also searched 12 leading public administration journals (see list of journals in appendix table A1 ) using the same terms. We then screened titles and abstracts and then full papers to identify all papers that passed our eligibility criteria. Finally, we screened the literature list of all eligible journal articles for missing records. In total, we identified 100 peer-reviewed journal articles for the systematic literature review.

To obtain a comprehensive pool of working papers, we created a list of all authors who contributed at least two articles to the literature review (see appendix table A2 ). We then contacted all authors on the list and asked them to provide any unpublished, full-length papers on administrative burdens that they had (co-)authored. We also encouraged them to let us know if they knew of other working papers on the topic. Almost all authors replied within a few days and most sent one or more working papers. Moreover, we made a call for working papers through our Twitter accounts and a similar call through a listserv for scholars interested in administrative burden research managed by Professor Donald Moynihan. Based on these steps, we collected 19 eligible working papers. 2 In total, 119 papers are included in the review (see the full list of papers in appendix table A7 ). Figure 2 summarizes the selection process.

Selection Process.

Coding Strategy

We relied on two strategies for coding the articles. First, we systematically coded several facts about the articles (year of publication, whether empirical material was collected, methods used, country covered by empirical analysis, policy area, and type of subjects) using a closed coding strategy (see appendix table A3 for a full description of coding criteria). We present this information in the first part of the results section to give an overview of the field and the types of studies conducted.

Second, we used an open coding where we focused on core concepts covered in the articles and types of causal relations covered in the papers. This is a demanding task that requires that coders have in-depth knowledge of the literature. We therefore handled all coding ourselves and met several times during the coding process to ensure consistency in the categorization of relationships and concepts. We use the qualitative coding to summarize current knowledge about the different relationships shown in figure 1 and to extend the causal model based on the findings and arguments in extant research.

Citation Analysis

This first part of the analysis covers key characteristics of the articles on administrative burdens in citizen–state interactions. Related to the discussion of eligibility criteria, we initially explore whether studies frequently cited by our 119 eligible studies are missing in the review. Table 2 shows that among the top 10 most cited papers and publications in the review, three publications do not rely on the administrative burden framework and therefore do not meet the eligibility criteria. Two of these publications ( Brodkin and Majmundar 2010 ; Lipsky 1980 ) concern street-level bureaucracy, and the third ( Bhargava and Manoli 2015 ) focuses on take-up of benefits. Thus, while there certainly are some widely cited works outside the narrow domain of administrative burden research, the field is generally dominated by internal references, suggesting that administrative burden research indeed constitutes a distinct field of its own.

Top 10 Most Cited Publications by the 119 Papers Included in the Systematic Review

A related question is how well studies with different foci, research questions, and methodologies speak to one another. We conducted a bibliographical network analysis ( Perianes-Rodriguez, Waltman, and van Eck 2016 ) in which we explored citation patterns between articles. As shown in appendix table A4 , assortativity scores are generally low, suggesting that articles tend to cite each other to an almost equal extent despite different methodologies and research questions ( Newman 2003 ). Overall, the analysis suggests that the field is coherent in the sense that even the most different parts of the field tend to rely on each other’s work.

Methodological Characteristics

Of the 119 collected articles, 75% are empirical papers using qualitative or quantitative analysis of data, while 25% are theoretical papers, literature reviews, or case studies. Articles are published in 35 different journals. Most are published in public administration journals, but some are published in either health, economics, or political science journals. The most frequent appearances are in Public Administration Review with 17, Journal of Public Administration Research and Theory with 16, and Journal of Behavioral Public Administration with 11 articles (see appendix table A5 for full details). Figure 3 shows a timeline of all published papers on the topic. Only nine were published between 2012 and 2017, but the publication trend changed significantly in 2018. From 2018 to 2021, the number of yearly published papers almost doubled each year from 5 in 2018 to 36 in 2021. While 2022 saw a decline in publications to 20, the overall trend still indicates that the study of administrative burden has established itself as a sizeable subfield within public administration research.

Publication Timeline.

Note: n = 100. The figure shows the year studies were made available online and does not include working papers.

Figure 4 graphs methodological characteristics of the studies. Panel A shows that more than half the empirical studies use quantitative methods. However, a substantial number of articles employ qualitative methods or case studies, meaning the field is characterized by some methodological diversity. This is also evident from panel D, where we divide the quantitative and qualitative categories into more specific subcategories. We see that studies on administrative burdens use a great variety of methods, and that studies utilize both observational and experimental data to a high extent. Studies are also relatively diverse when it comes to the origin of data, as our review includes studies from all six inhabited continents. However, studies from Western countries dominate the literature, as 82% of all studies were conducted in either the United States, Europe, or Australia (see panel B). We also coded whether papers used data from more than one country. Only three papers used data do so, and neither of them used a comparative approach where they compared burdens across contexts. Panel D shows that almost half of the studies focus on target group members. This aligns well with the fact that one purpose of the administrative burden framework is to draw attention to individuals’ experiences of policy implementation ( Moynihan, Herd, and Harvey 2015 ). Finally, panel E shows that around 50% of all studies focused on means-tested welfare benefits. This may reflect that means-tested programs are often where citizens encounter the most requirements and therefore are likely to experience various burdens when interacting with the state.

Methodological Characteristics of Empirical Studies.

Note: Figures A-E display various charatersitics of empirical studies. Articles that fit into more than one category are coded into all relevant categories. Purely theoretical articles are not included in any of the figures.

This section presents the results of our qualitative analysis of the literature. Figure 5 provides an overview of our main findings. This model extends the theoretical model in figure 1 in four important respects. First, it proposes a more nuanced understanding of what state actions are. In line with Baekgaard and Tankink (2022 , 17), we understand state actions broadly to cover what the state does “including laws, rules, requirements, and how such are implemented by public officials and street-level bureaucrats.” This leads us to distinguish between formal (arrow 1) and informal policy designs (arrow 2). While formal policy design refers to the laws and rules enacted by politicians, that is, the rules that people will have to abide to get access to services and benefits, informal policy design concerns how these rules are implemented at the frontline and communicated more broadly. This allows us to discuss how different aspects of policies lead to experiences of administrative burdens. Second, the model extends the number of factors explaining state actions beyond political ideology by introducing the concepts of burden support and burden tolerance, that is, “the willingness of policymakers and people more generally to passively allow or actively impose state actions that result in others experiencing administrative burdens” ( Baekgaard, Moynihan, and Thomsen 2021 , 184). As shown, support and tolerance for burdens may sometimes be influenced by the content of state actions when people become aware of actual rules and implementation (arrow 6b).

Extended Model of Causal Claims.

Third, the model proposes that other factors than political ideology and beliefs may influence burden tolerance and state actions. In particular, the model highlights the importance of target group deservingness, personal experience, and bureaucratic processes (arrow 7). Fourth, the model proposes feedback effects of citizens’ experiences of burden on how burdens are constructed by the state and how tolerant policymakers and others are of burdens to begin with (arrows 5a and 5b).

Table 3 lists the number of studies that cover each relationship. Below, we discuss each of the seven arrows in figure 5 . Our aim is not to mention all studies discussing each specific arrow but rather to summarize current knowledge about each relationship. Our discussion therefore only covers selected articles that provide knowledge on the relationship under discussion. Appendix table A6 is an extended version of table 3 and shows the articles that provide knowledge on each relationship.

Number of Papers Studying Each Causal Relationship

Arrow 1: Formal Policy Design → Experiences of Burden

With few exceptions, studies find that state barriers are associated with experiences of learning and compliance costs. Learning costs, for instance, arise when being subject to requirements ( Cook 2021 ), misinformation ( Chudnovsky and Peeters 2021a ), and having to deal with vouchers ( Barnes 2021 ), while compliance costs arise because of transportation time to vaccinator camps ( Ali and Altaf 2021 ) and completing forms ( Yates et al. 2022 ). Some studies find that learning and compliance costs arise as a consequence of (eligibility) requirements in means-tested welfare programs ( Holler and Tarshish 2022 ) and insurance programs ( Yates et al. 2022 ). Other studies find that learning and compliance costs also arise in settings such as the restoration of voting rights ( Selin 2019 ), digital government services ( Madsen, Lindgren, and Melin 2022 ), and accessing vaccinations ( Ali and Altaf 2021 ).

Studies are conducted in diverse contexts such as Pakistan, Denmark, the United States, and Argentina, suggesting there is some universality to the claim that interacting with the state is associated with experiences of learning and compliance costs. However, one paper finds that having a scheduled compulsory meeting with frontline workers causes no changes in compliance costs and is associated with experiences of less learning costs ( Baekgaard and Madsen 2023 ). Another study finds that digital self-service solutions have the potential to both increase and reduce learning and compliance costs ( Madsen, Lindgren, and Melin 2022 ).

This suggests that more research is needed on how different types of state actions reduce and impose experiences of learning and compliance costs. Such studies could build on more qualitative approaches to obtain a better understanding of the mechanisms linking state actions to experiences. Also, when it comes to understanding the costs of dealing with different state actions, qualitative methods have major advantages over other methods. With a few exceptions ( Ali and Altaf 2021 ; Baekgaard and Madsen 2023 ), most papers indeed use qualitative methods to study the relationship between barriers and learning and compliance costs, while no papers use experimental methods. This is not surprising, as it is often hard to manipulate barriers or state actions. However, in addition to more qualitative research, the literature would benefit from studies that are able to causally link state actions to experiences of learning and compliance costs. As mentioned in the next section, a few studies document how state actions causally influence experiences of psychological costs, showing that it is possible to causally study the link between state actions and experiences of administrative burdens.

There are 50% more studies on the relationship between formal policy designs and psychological costs than on the comparable relationship with learning and compliance costs discussed above, illustrating that this relationship has received high scholarly attention. The general finding from the 16 studies discussing this topic is that state actions are associated with various forms of psychological costs. Examples of psychological costs arising from state actions are autonomy loss and stress ( Baekgaard et al. 2021 ), frustration ( Cook 2021 ), stigma ( Selin 2019 ; Thomsen, Baekgaard, and Jensen 2020 ), externalization of locus of control ( Madsen and Mikkelsen 2022 ), uncertainty ( Cecchini nd ) and confusion, anger, and frustration ( Hattke, Hensel, and Kalucza 2020 ).

Studies fall in two methodological categories: qualitative studies and experiments. Qualitative studies provide in-depth knowledge about how state actions may lead to psychological costs. One example is Yates et al.’s (2022) study of burdens in Australia’s National Disability Insurance Scheme. One interviewee mentions that it was “wearing” and “soul destroying” “to be constantly questioned about, are you disabled enough” (p. 5), showing how eligibility requirements can create psychological costs.

Experimental studies establish causal links between barriers and costs. Baekgaard et al. (2021) use survey- and field-experimental evidence to show that reductions in state compliance demands reduce stress and increase the sense of autonomy among target group members. Hattke, Hensel, and Kalucza (2020) and Hattke et al. (nd) rely on laboratory experiments to show how redundant documentation requirements and simple administrative processes can cause confusion, frustration, and anger.

In general, the link between state actions and psychological costs is relatively well covered in the literature. However, studies so far have generally examined only one or a few state actions. There is a lack of studies that compare effects of different actions on psychological costs. Such studies could provide valuable knowledge on which state actions translate into psychological costs.

Arrow 2: Informal Policy Design → Experiences of Burden

Informal policy design has to do with the actions by the state that do not directly refer to the formal rules and requirements as decided by policymakers but rather how these are processed and communicated to citizens. Two aspects of informal policy design are particularly prevalent in research on administrative burden: frontline service delivery and government communication.

Frontline Service Delivery

It is no surprise that the delivery of services at the frontline of public organizations matters for experiences of burden. Lipsky (1980) alluded to this, and subsequent work has explored this question without explicitly using the concept of administrative burden (e.g., Brodkin and Majmundar 2010 ; Soss, Fording, and Schram 2011 ). Studies applying the administrative burden framework show that workload matters for experiences of administrative burden. For instance, Bell and Meyer (nd) use administrative data from college financial aid programs to show that decreases in workload lead to an increase in program access for low-income students and that the increase is highest among students who have been subject to discrimination based on their race. Ali and Altaf (2021) show that citizens experience more burdens in areas with lower administrative capacity, while others find that stress and burnout ( Mikkelsen, Madsen, and Baekgaard 2023 ) and red tape ( Madsen ndb ) among frontline workers are associated with experiences of burden among their clients.

The behavior of frontline workers also matters for citizens’ experiences. Bell and Smith (2022) show that frontline workers who adopt a support role rather than a role as “compliance officer” are more likely to use their discretionary power to help students overcome administrative burdens. In a similar vein, Halling’s (nd) results suggest that frontline workers help citizens overcome burdens by circumventing rules. Finally, Barnes and Henly’s (2018) qualitative analysis shows that clients tend to blame their experiences of administrative burden on frontline employees.

Government Communication

Another part of informal policy design that has received considerable attention is how communication from the state affects individuals’ experiences of administrative burden. All these papers rely on field experiments with randomized exposure to different forms of government communication. Linos et al. (2022) show that disadvantaged groups prefer postcards over a telephone hotline to seek information about free dental care. They use focus groups to show that this is likely explained by lower psychological costs associated with postcards as participants fear uncomfortable interactions with bureaucrats. Moynihan et al. (2022) show how the framing of state categories matters for selecting into the right categories and that a more intuitive presentation of information increased the number of claimants providing adequate documentation. Simplified communication ( Linos, Reddy, and Rothstein 2022 ), destigmatizing language ( Lasky-Fink and Linos 2023 ), early communication ( Linos, Quan, and Kirkman 2020 ), postcards ( Hock et al. 2021 ), letters ( Bhanot 2021 ), and text messages ( Lopoo, Heflin, and Boskovski 2020 ) can also improve take-up.

Altogether, these field experiments show that different forms of nudges can be effective in increasing take-up of benefits among eligible individuals. Apart from the two first-mentioned studies, the studies do not measure experiences of burden directly. Instead, they measure different outcomes while theorizing that the link between communication and outcomes has to do with experiences of burden. Hence, there is a need for studies that show that reduction of administrative burdens is the process through which these nudges work.

Arrow 3: Distributive Effects

The argument that administrative burdens are distributive and can foster inequality is at the core of the administrative burden framework ( Christensen et al. 2020 ; Herd and Moynihan 2018 ). Thirty-one papers contribute knowledge on the distributional consequences of state actions. Differences in resources, attitudes, and expectations between citizens constitute one main type of distributive effects identified in the literature ( Christensen et al. 2020 ; Heinrich 2018 ; Nisar 2018 ). The other type, which has received less attention, focuses on how characteristics of the state may contribute to different experiences of burden among different parts of the population ( Griffiths 2021 ; Peeters, Renteria, and Cejudo nd ). We discuss both types next.

Citizen Factors

Studies show that possessing administrative literacy ( Döring 2021 ; Döring and Madsen 2022 ), self-efficacy ( Thomsen, Baekgaard, and Jensen 2020 ), habitus and different forms of capital ( Carey, Malbon, and Blackwell 2021 ; Masood and Nisar 2021 ) all make state barriers easier to handle, resulting in fewer experiences of burdens. All these contributions are important in documenting that possessing the necessary capital and skills is key when dealing with onerous state demands.

However, there is a considerable overlap between the different concepts. Apart from self-efficacy, all focus on a type of capital (or literacy) that makes state encounters easier to handle. Some are specific to encounters with the state (administrative literacy and capital), while others are more general forms of capital (human capital and Bourdieu’s capital concepts). Discussing differences and similarities between the concepts is beyond the scope of this article, but we note that using fewer concepts would strengthen the comparative potential across studies.

Other studies focus on how experiences of burdens are distributed across demographic and non-demographic characteristics. The general finding is that individuals from marginalized or low-resource groups tend to struggle more with state barriers. So far, studies have shown that individuals with low income or who are experiencing scarce financial resources ( Chudnovsky and Peeters 2021b ; Heinrich et al. 2022 ; Larsson 2021 ; Madsen, Baekgaard and Kvist 2022 ), ethnic minorities ( Heinrich 2018 ; Olsen, Kyhse-Andersen, and Moynihan 2020 ), women ( Kyle and Frakt 2021 ; Yates et al. 2022 ), individuals with low or no education ( Chudnovsky and Peeters 2021b ; Collie et al. 2021 ; Kyle and Frakt 2021 ), and those suffering from sickness and disabilities ( Bell et al. 2022 ; Collie et al. 2021 ; Kyle and Frakt 2021 ) experience more administrative burdens as a result of state actions.

Relatedly, a few studies discuss how citizens’ attitudes and expectations might influence how citizens engage with the state and hence lead to different impacts of state actions on experiences of burden. These attitudes and expectations may themselves stem from a variety of sources including prior interactions with the state ( Chudnovsky and Peters 2021b , 531), thus suggesting a potential feedback effect from outcomes on attitudes and expectations (see also Moynihan and Soss 2014 ). 3

Finally, a last stream of studies considers how individuals’ access to relevant third parties, actors outside the citizen–state interaction that provide help to citizens or otherwise influence interactions ( Moynihan, Herd, and Harvey 2015 ), may affect their experiences of administrative burden. A few papers explore the role of such actors. Barnes (2021, nd ) shows that retailers play a crucial role in shaping compliance costs in voucher programs such as WIC. Because citizens must redeem their vouchers in retail stores, retailers play a huge role in shaping how easy redemption is. Concrete examples are the degree to which eligible food is marked and displayed and whether store personnel are trained in handling vouchers. NGOs may also contribute to reduced learning and compliance costs by helping citizens overcome burdens ( Nisar 2018 ; Nisar and Masood nd ). Finally, (ex-)family members may influence experiences of administrative burden ( Nisar 2018 ). Cook (2021) illustrates how ex-partners may directly impose burdens on mothers in the child support benefit system in Australia. As an example, some fathers limit their child support liabilities or claim that they have already provided payments to mothers. Each time fathers make such changes or claims, mothers are required to respond, which can be associated with substantial compliance costs.

State Characteristics

Another possible source of distributive effects is the state itself. A key insight from this stream of research is that variations in administrative capacities to reach out to vulnerable populations may contribute to inequality in the experience of burdens. Some studies investigate how individuals may experience different burdens in states with different characteristics. The most prominent characteristic examined so far is the extent to which the state is automated and digitalized. Peeters, Renteria, and Cejudo (nd) illustrate how governments with higher information capacity are better able to “absorb” burdens, which means that citizens face fewer administrative burdens. Digital government may also create unintentional errors that contribute to considerable experiences of administrative burden. Griffiths (2021) shows how automation of benefit calculation can create burdensome experiences. For example, people with irregular pay dates risk missing out on benefits for which they are eligible because automation processes do not account for irregular cases. Likewise, Widlak and Peeters (2020) show that citizens face various administrative burdens in correcting errors made by the state, while Compton et al. (2022) show that blacks and Hispanics are disproportionally hit by administrative errors.

Other state characteristics that may influence experiences of administrative burdens are material and artificial artifacts present in physical and virtual government arenas ( Nisar and Masood nd ) and consistent application of rules ( Kaufmann, Ingrams, and Jacobs 2021 ). Finally, Johnson and Kroll (2021) theorize but find no supporting empirical evidence that representative government and shared identities between frontline employees and citizens may decrease experiences of burden.

Arrow 4: Experiences of Burden → Outcomes

According to Moynihan, Herd, and Harvey (2015) , administrative burdens are an important part of governance, “since they affect whether citizens succeed in accessing services (did I get what I want), whether public polices succeed (did a program reach the targeted group?), and the perceptions of government (was I treated fairly and with respect?)” (p. 43). However, despite the obvious importance of studying the link between experiences of burden and various outcomes, only Daigneault and Macé (2020) have done so among published papers. Based on interviews with target group members, they show that individuals experiencing compliance and learning costs are less likely to take up Quebec’s Supplement to the Work Premium program. Other papers study the link between state actions and outcomes but without subjective measures of people’s experiences of administrative burden. Notable examples are Heinrich (2016) and Jenkins and Nguyen (2022) , who convincingly, and with strong causal traction, show that various state actions influence take-up of welfare programs and might even impact long-term outcomes such as risky behaviors in adolescence ( Heinrich 2016 ; Heinrich and Brill 2015 ). These studies contribute important knowledge on how state actions influence take-up of welfare benefits but not on the relationship between subjective experiences of burden and outcomes.

Several working papers show that experiences of burden are associated with behaviors that can lead to reduced program take-up, such as compliance and autonomous motivation ( Madsen nda ), making errors on forms ( Hattke et al. nd ), and filing complaints ( Bell et al. 2022 ). While these papers make valuable contributions, none of them study actual outcomes but rather behaviors that are likely to influence take-up of benefits. The final working paper by Lasky-Fink and Linos (2023) offers a promising approach to dealing with some of the shortcomings of other research on this relationship. Contrary to the other working papers, the authors study actual take-up of welfare benefits and show that destigmatized language leads to substantially higher take-up rates. Moreover, contrary to studies linking state actions and take-up, the authors go one step further and use three survey experiments to make it probable that the mechanism linking state actions and take-up is psychological costs in the form of perceived stigma. In doing so, the working paper studies the whole causal chain from barriers over subjective experiences of administrative burdens to outcomes. This is a model for future studies to pursue because such studies will be able to show not only whether individuals experience burdens as a result of state actions, but also the extent to which these burdens subsequently influence service use or other relevant outcomes.

There is also a lack of studies that look beyond take-up and focus on other types of outcomes. In some instances, burdens may not discourage people from taking up public services, but they may still affect the adequacy and quality of services provided—in particular when citizens interact with the same public agency for a prolonged period of time ( Peeters and Campos 2021 ). Furthermore, inspired by the policy feedback literature, it has been suggested that experiences of burden may affect civic capacities such as political efficacy, trust in institutions, and civic engagement ( Christensen et al. 2020 ). However, no studies have so far examined these questions systematically.

Arrow 5: Feedback Effects: Experiences → Burden Tolerance and State Actions

While state actions are expected to trigger experiences of burdens in the original theoretical model, a few studies suggest a feedback effect, that is, experiences may influence burden tolerance and state actions. The argument is that knowledge about experiences may make policymakers and others understand the detrimental effects of state actions and hence induce less burden. This proposition finds mixed support in the three studies dealing with the question. In a survey-experimental study of Danish local politicians using a treatment cue about psychological costs experienced by target group members, Baekgaard, Moynihan, and Thomsen (2021) find no evidence of a feedback effect. Conversely, in a survey experiment, Halling and Petersen (nd) find that Danish frontline employees are more likely to reduce compliance demands in the implementation process and to help citizens who communicate psychological costs. Sievert and Bruder (2023) find mixed support in their study of the feedback effects of treatments increasing awareness of learning and compliance, costs among German citizens. While there is some evidence of feedback effects of compliance costs, exposing participants to information about learning costs does not affect burden tolerance. Finally, Gilad and Assouline (2022) do not study feedback effects directly, but rather a prerequisite of their existence, namely citizens voicing their experiences of burden. They find that citizens indeed voice their experiences to authorities but also that disadvantaged groups are less inclined to do so.

On balance, there is a need for much more research to establish the relevance of feedback effects. Such studies could investigate differences between groups of respondents (policymakers, frontline workers, citizens). They may also focus on the way in which information about experiences of burden is provided. Here, a distinction could be made between statistical and episodic information. Previous research has identified stronger effects of episodic data in other contexts ( Olsen 2017 ). Finally, studies could examine feedback effects from citizen outcomes.

Arrow 6: The Relationship Between Burden Tolerance and State Actions

The literature on burden tolerance presumes that tolerance among political decision-makers and the mass public influences the extent to which the state constructs burdens (e.g., Aarøe et al. 2021 ; Baekgaard, Moynihan, and Thomsen 2021 ; Keiser and Miller 2020 ; Nicholson-Crotty, Miller, and Keiser 2021 ). However, none of the studies in the review study the causal influence of burden tolerance on state actions, likely due to challenges obtaining causal estimates. Nevertheless, we indicate this relationship in figure 5 with a dashed line (arrow 6a) due to the strong theoretical expectation that burden tolerance influences the extent to which the state introduces burdens in public policies.

Alternatively, it is possible that knowledge about existing barriers influences the extent to which people are supportive of burdensome barriers (arrow 6b). Two empirical studies examine this question using survey experiments among the mass public. Keiser and Miller (2020) find that information about the presence of barriers increases support for welfare programs and their recipients, in particular among conservative voters. Nicholson-Crotty, Miller, and Keiser (2021) show that information about barriers has heterogeneous effects on program approval depending on whether the target group is perceived as deserving (information about more barriers reduces approval) or undeserving (information about barriers has no significant effect). While the two studies support the idea that information about state actions may influence burden tolerance, there is certainly room for more research about how state actions may influence burden tolerance in the mass public and among decision-makers. Such studies may for instance investigate how state actions are constructed in popular debates.

Arrow 7: Factors Shaping Burden Tolerance and State Actions

This section looks into other factors that shape burden tolerance and state actions. A total of seven studies examine factors shaping burden support, while 13 studies investigate factors shaping state actions. We deal with the questions jointly, because many of the key explanations are similar for burden tolerance and state actions. Overall, explanations can be divided into four broad categories.

First, a series of studies present evidence that burdens are constructed and that political ideological beliefs influence the extent to which barriers are introduced. For instance, the studies by Moynihan, Herd, and Harvey (2015) , Moynihan, Herd, and Ribgy (2016) , and Heinrich (2018) find that more barriers are introduced in states governed by conservatives than in states governed by liberals. Likewise, a series of cross-sectional studies find strong correlations between the ideological beliefs of politicians ( Baekgaard, Moynihan, and Thomsen 2021 ), street-level bureaucrats ( Bell et al. 2020 ), and the mass public ( Haeder, Sylvester, and Callaghan 2021 ; Halling, Herd, and Moynihan 2022 ) and their support for administrative burden policies.

Second, in accordance with the claim by Schneider and Ingram (1993) that target group construction matters to the benefits and burdens assigned to each group, target group deservingness and minority status appear to be of major importance to both burden tolerance ( Baekgaard, Moynihan, and Thomsen 2021 ; Haeder, Sylvester, and Callaghan 2021 ) and barriers ( Jilke, Van Dooren, and Rys 2018 ).

Third, a series of individual-specific explanations of burden tolerance have been investigated in the literature. Most factors have not been theorized very clearly, however, and have only been the subject in few empirical studies. Personal experience with benefits has been shown to be associated with less tolerance for burdensome state actions among Danish local politicians ( Baekgaard, Moynihan, and Thomse 2021 ) and a representative sample of US citizens ( Halling, Herd, and Moynihan 2022 ), while big five personality traits in the form of conscientiousness and openness to experiences have been shown to correlate with burden tolerance in the study of Aarøe et al. (2021) .

Fourth, studies of factors explaining variation in barriers find bureaucratic processes are likely to shape the barriers that citizens meet when interacting with the state. These studies are primarily based on discussions of specific exemplary cases. Peeters (2020) points out that barriers are likely to be unintentional in many cases. They can, for instance, be a result of very complex cases that make it impossible to ease application processes for citizens by means of automation ( Larsson 2021 ), or they can be unintended results of large-scale digitalization and automated decision-making processes where citizens who do not fit into predefined boxes face barriers in the implementation process ( Peeters and Widlak 2018 , 2023 ). Other studies show that bureaucratic low-trust culture and inertia may increase barriers that citizens face when interacting with government ( Bashir and Nisar 2020 ; Peeters et al. 2018 ).

Before we move on to the discussion of next steps to be taken, we note three limitations of our study. The first is publication bias. While we approached the field to include unpublished research, it is possible that some unpublished null findings have not been included or that published null findings did not show up in our literature search because publications with null findings on administrative burden hypotheses have been framed into other literatures. While we consider this a lesser concern given our extensive strategy for collecting studies, publication bias may have made evidence appear stronger than it is. The second limitation has to do with the qualitative coding of studies. While we adhere to stringent coding criteria and have conducted multiple rounds of cross-validating the coding, categorizing studies based on the kind of relationships they study is—at least for some studies—a matter of nuance and assessment. Third, the quality of the included studies is likely to vary, meaning that our review may not give an accurate picture of the strength of evidence for the many propositions studied in administrative burden research. While we have confidence in the general pattern of how different relationships have been covered, others may disagree with our coding of some studies and with the strength of evidence presented in these studies.

Limitations aside, our review points out where evidence is missing and suggests steps to be taken in future research. Next, we discuss which parts of our theoretical model warrant more empirical evidence before finishing with a discussion of new questions for future research to pursue.

More Evidence Needed

Our review points to several issues that should get more attention in future research. First, our understanding of people’s experiences is very much based on the deductive categorization of experiences as learning, compliance, and psychological costs developed in Moynihan, Herd, and Harvey (2015) . While this has laid the foundation for important research, future research could do more to supplement it with bottom-up qualitative research of what burdens are from the perspectives of those interacting with the state. Such research could also aid our understanding of what constitutes more important types of burdensome experiences and under what circumstances they arise. A good example of this kind of research is the work of Barnes (2021, nd ).

Second, it is a core claim of the administrative burden framework that what the state does is consequential for citizens’ experiences. Providing solid causal evidence about this relationship is therefore a key point for future research. Future studies could for instance rely on laboratory experiments inspired by the studies by Hattke, Hensel, and Kalucza 2020 and Hattke et al. (nd) . Another way forward may be to embed surveys and in-depth interviews as part of randomized field experiments to explore how changes in state action influence experiences and in turn outcomes. Here, the study by Lasky-Fink and Linos (2023) may also serve as an example to follow, as the authors combined their field experiment with survey experimental evidence to explore whether the impact of destigmatized language on take-up indeed was mediated by reduced perceived stigma as hypothesized by the authors.

Third, most studies examining this link are conducted among recipients of various social welfare benefits. However, experiences of burden are likely to arise in other types of interactions with the state as is evident from studies of, among others, digital government services ( Madsen, Lindgren, and Melin 2022 ) and voting rights ( Herd and Moynihan 2018 , 43–70; Selin 2019 ). To better understand the scope and importance of administrative burden, there is a need for studies that move beyond social welfare to investigate experiences of burdens in areas such as law enforcement, taxation, and regulation.

Fourth, research on how experiences of administrative burden affect outcomes such as welfare take-up, trust in government, health, and voting behavior is scarce. Most of the articles that study outcomes (primarily take-up) examine how they relate to state actions and not to experiences of burden. To get a more comprehensive picture of how burdensome encounters influence citizens’ lives, we encourage future studies to examine the link between experiences of burden and outcomes.

Fifth, the advancement of the burden tolerance concept allows researchers to examine the extent to which individuals support barriers. An important assumption is that the burden tolerance of policymakers and bureaucrats shapes the actual design of state actions, but it has never been empirically examined. Doing so would help ascertain whether burden tolerance is consequential for the actual design of polices.

Sixth, the administrative burden literature is diverse in terms of methods, policy areas, and subjects. Most studies are conducted in Western countries, but there are studies of burdens from other contexts such as Pakistan and Latin America. However, there is a general lack of comparative studies of burdens across countries and across policies, which would be valuable in terms of providing knowledge on the extent to which context matters for experiences of burden. Likewise, comparative studies of barriers or across policy areas could elucidate which types of state actions are most likely to produce experiences of burdens.

New Questions to Pursue

While we have presented a quite extensive model based on current administrative burden studies, there are still important questions that have received little to no attention in the literature. An important part of the framework formulated by Herd and Moynihan (2018) is that burdens are not inherently bad, and that they often serve legitimate purposes of protecting program integrity and avoiding fraud. While the issue of burden legitimacy has received some theoretical attention ( Doughty and Baehler 2020 ), empirical scholarship has yet to engage with it. One important question is how policymakers and citizens form preferences regarding program integrity vis à vis target group members’ onerous experiences. Studies on burden tolerance touch upon this question, but do not tackle it directly. Another question is how policymakers legitimize the existence of administrative burdens. Do they emphasize fraud protection, budget concerns, targeting the most deserving individuals, or something else? A third question that should get more attention is how actors outside the citizen–state interaction shape experiences of administrative burdens. A few studies show that various third parties such as NGOs and family members can influence experiences of burden, but the roles of these actors still warrant more attention. Further, civil society and the media may influence citizens’ experiences. For example, target group members are often negatively portrayed in the media ( Baekgaard, Herd, and Moynihan 2022 ; Schneider and Ingram 1993 ), which could increase their experiences of burden.

The administrative burden literature, while surprisingly clearly demarcated from other fields of research, has developed into a thematically and methodologically diverse research field within few years. Overall, our systematic review demonstrates that empirical research in the field generally supports the original three-fold claim made by Moynihan, Herd, and Harvey (2015) that burdens are consequential, constructed, and fall harder on groups with few resources. Yet, the review also demonstrates that the literature has moved past these claims in important ways. Based on our reading and coding of 119 articles and working papers, we build a comprehensive model of causal claims in the literature. The model illustrates different relationships that have been explored in the still nascent literature on administrative burdens, and it highlights several new theoretical insights gained since the founding work of Moynihan, Herd, and Harvey (2015) . First, experiences of administrative burdens are sometimes unrelated to how burdens are constructed by the state and instead rely on other factors such as frontline service delivery, government communication, unintended actions, and third parties. Second, the model highlights that factors beyond political ideology may affect the construction of state actions by introducing the concept of burden tolerance. Third, the model shows that factors such as personal experience with programs, personality traits, and the structure of bureaucratic processes affect individuals’ burden tolerance. Finally, the model illustrates a potential feedback effect of citizens’ experiences of administrative burden on policymakers’ burden tolerance.

Our systematic coverage of the administrative burden literature offers promising avenues for new research. First, we call for studies that causally link state actions and experiences of administrative burden, for studies that link experiences of burden to outcomes such as democratic behavior and take-up, and for studies that connect policymakers’ burden tolerance to actual state actions. Methodologically, we call for in-depth qualitative studies of how burdens are experienced by people taking part in citizen–state interactions and comparative studies. Last, we argue that important questions remain unexplored. One topic that future research should address is how policymakers, bureaucrats, and members of the public balance the legitimacy of public policies against target group members’ experiences of administrative burden. Is it acceptable to enhance experiences of administrative burdens to avoid fraud or to target the right populations? Another topic that warrants more attention is how actors outside the citizen–state interaction shape experiences of administrative burden. For example, we know that welfare recipients are often negatively constructed in the media and society ( Baekgaard, Herd, and Moynihan 2022 ; Schneider and Ingram 1993 ), yet we have limited knowledge about whether this leads to them experiencing administrative burdens to a larger extent when interacting with the state.

Supplementary data is available at the Journal of Public Administration Research and Theory online.

We thank Arne Hørlück Høeg for providing excellent research assistance. We are also thankful for the great comments we received from participants at the Administrative Burden pre-conference workshop at the 2022 PMRC.

This work was supported by funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement no. 802244).

No new data were generated or analyzed in support of this research.

Aarøe , Lene , Martin Baekgaard , Julian Christensen , and Donald P. Moynihan . 2021 . Personality and public administration: Policymaker tolerance of administrative burdens in welfare services . Public Administration Review 81 ( 4 ): 652 – 63 .

Google Scholar

Ali , Sameen A. Mohsin , and Samia W. Altaf . 2021 . Citizen trust, administrative capacity and administrative burden in Pakistan’s immunization program . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 17 .

Baekgaard , Martin , Pamela Herd , and Donald P. Moynihan . 2022 . Of “Welfare Queens” and “Poor Carinas” social constructions, deservingness messaging, and the mental health of welfare clients . British Journal of Political Science 53 ( 2 ): 594 – 612 .

Baekgaard , Martin , and Jonas Krogh Madsen . 2023 . Anticipated administrative burden: How proximity to upcoming compulsory meetings affect welfare recipients’ experiences of administrative burden . Public Administration 1 – 19 . doi: 10.1111/padm.12928

Baekgaard , Martin , Kim Sass Mikkelsen , Jonas Krogh Madsen , and Julian Christensen . 2021 . Reducing compliance demands in government benefit programs improves the psychological well-being of target group members . Journal of Public Administration Research and Theory 31 ( 4 ): 806 – 21 .

Baekgaard , Martin , Donald P. Moynihan , and Mette Kjaergaard Thomsen . 2021 . Why do policymakers support administrative burdens? The roles of deservingness, political ideology, and personal experience . Journal of Public Administration Research and Theory 31 ( 1 ): 184 – 200 .

Baekgaard , Martin , and Tara Tankink . 2022 . Administrative burden: Untangling a bowl of conceptual spaghetti . Perspectives on Public Management and Governance 5 ( 1 ): 16 – 21 .

Barnes , Carolyn. nd . Decoupling policy and practice: The redemption costs of WIC . Working Paper.

Google Preview

Barnes , Carolyn , and Virginia Riel . 2022 . “I don’t know nothing about that”: How “learning costs” undermine COVID-related efforts to make SNAP and WIC more accessible . Administration & Society 54 ( 10 ): 1902 – 30 .

Barnes , Carolyn Y. 2021 . “It takes a while to get used to”: The costs of redeeming public benefits . Journal of Public Administration Research and Theory 31 ( 2 ): 295 – 310 .

Barnes , Carolyn Y. , and Julia R. Henly . 2018 . ‘They are underpaid and understaffed’: How clients interpret encounters with street-level bureaucrats . Journal of Public Administration Research and Theory 28 ( 2 ): 456 – 456 .

Bashir , Mohsin , and Muhammed Azfar Nisar . 2020 . Expectation versus reality: Political expediency and implementation of information to information laws . Public Administration Quarterly 44 ( 1 ): 3 – 30 .

Bell , Elizabeth , Julian Christensen , Pamela Herd , and Donald P. Moynihan . 2022 . Health in citizen-state interactions: How physical and mental health problems shape experiences of administrative burden and reduce take-up . The American Review of Public Administration 83 ( 2 ): 385 – 400 .

Bell , Elizabeth , and Katharine Meyer . nd . Can reducing workload enhance equity at the front-lines? How street-level bureaucrats’ capacity impacts access to burdensome public programs . Working Paper.

Bell , Elizabeth , and Kylie Smith . 2022 . Working within a system of administrative burden: How street-level bureaucrats’ role perceptions shape access to the promise of higher education . Administration & Society 54 ( 2 ): 167 – 211 .

Bell , Elizabeth , Ani Ter-Mkrtchyan , Wesley Wehde , and Kylie Smith . 2020 . Just or unjust? How ideological beliefs shape street-level bureaucrats’ perceptions of administrative burden . Public Administration Review 81 ( 4 ): 610 – 24 .

Bhanot , Syon P. 2021 . Good for you or good for us? A field experiment on motivating citizen behavior change . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 14 .

Bhargava , Saurabh , and Dayanand Manoli . 2015 . Psychological frictions and the incomplete take-up of social benefits: Evidence from an IRS field experiment . American Economic Review 105 ( 11 ): 3489 – 529 .

Bisgaard , Mette . 2023 . Dealing with bureaucracy: measuring citizens’ bureaucratic self-efficacy . International Review of Public Administration 28 ( 1 ): 45 – 63 .

Bozeman , Barry , and Jan Youtie . 2020 . Robotic bureaucracy: Administrative burden and red tape in university research . Public Administration Review 80 ( 1 ): 157 – 62 .

Brodkin , Evelyn Z. , and Malay Majmundar . 2010 . Administrative exclusion: Organizations and the hidden costs of welfare claiming . Journal of Public Administration Research and Theory 20 ( 4 ): 827 – 48 .

Bruch , Sarah K. , Myra Marx Ferree , and Joe Soss . 2010 . From policy to polity: Democracy, paternalism, and the incorporation of disadvantaged citizens . American Sociological Review 75 ( 2 ): 205 – 26 .

Burden , Barry C. , David T. Canon , Kenneth R. Mayer , and Donald P. Moynihan . 2012 . The effect of administrative burden on bureaucratic perception of policies: Evidence from election administration . Public Administration Review 72 ( 5 ): 741 – 51 .

Campbell , Jesse W. , Sanjay K. Pandey , and Lars Arnesen . 2022 . The ontology, origin, and impact of divisive public sector rules: A meta‐narrative review of the red tape and administrative burden literatures . Public Administration Review 83 ( 2 ): 296 – 315 .

Carey , Gemma , Helen Dickinson , Eleanor Malbon , Megan Weier , and Gordon Duff . 2020 . Burdensome administration and its risks: Competing logics in policy implementation . Administration & Society 52 ( 9 ): 1362 – 81 .

Carey , Gemma , Eleanor Malbon , and James Blackwell . 2021 . Administering inequality? The national disability insurance scheme and administrative burdens on individuals . Australian Journal of Public Administration 80 ( 4 ): 854 – 72 .

Cecchini , Mathilde. nd . Into the unknown—The administrative burden of uncertainty in citizen-state interactions . Working Paper.

Christensen , Julian , Lene Aarøe , Martin Baekgaard , Pamela Herd , and Donald P. Moynihan . 2020 . Human capital and administrative burden: The role of cognitive resources in citizen-state interactions . Public Administration Review 80 ( 1 ): 127 – 36 .

Chudnovsky , Mariana , and Rik Peeters . 2021a . A cascade of exclusion: Administrative burdens and access to citizenship in the case of Argentina’s National Identity Document . International Review of Administrative Sciences 88 ( 4 ): 1068 – 85 .

———. 2021b . The unequal distribution of administrative burden: A framework and an illustrative case study for understanding variation in people’s experience of burdens . Social Policy and Administration 55 ( 4 ): 527 – 42 .

Collie , Alex , Luke Sheehan , Ashley McAllister , and Genevieve Grant . 2021 . The learning, compliance, and psychological costs of applying for the disability support pension . Australian Journal of Public Administration 80 ( 4 ): 873 – 90 .

Compton , Mallory E. , Matthew M. Young , Justin B. Bullock , and Robert Greer . 2022 . Administrative errors and race: Can technology mitigate inequitable administrative outcomes ? Journal of Public Administration Research and Theory 33 : 512 – 28 .

Cook , Kay. 2021 . Gender, malice, obligation and the state: Separated mothers’ experiences of administrative burdens with Australia’s child support program . Australian Journal of Public Administration 80 ( 4 ): 912 – 32 .

Currie , Janet. 2006 . The take-up of social benefits . In Public policy and the distribution of income , ed. A. J. Auerbach , D. Card , and J. M. Quigley , 80 – 148 . New York : Russell Sage Foundation .

Daigneault , Pierre Marc , and Christian Macé . 2020 . Program awareness, administrative burden, and non-take-up of Québec’s supplement to the work premium . International Journal of Public Administration 43 ( 6 ): 527 – 39 .

Döring , Matthias. 2021 . How-to bureaucracy: A concept of citizens’ administrative literacy . Administration & Society 53 ( 8 ): 1155 – 77 .

Döring , Matthias , and Jonas Krogh Madsen . 2022 . Mitigating psychological costs—The role of citizens’ administrative literacy and social capital . Public Administration Review 82 ( 4 ): 671 – 81 .

Doughty , Meghan , and Karen J. Baehler . 2020 . “Hostages to compliance”: Towards a reasonableness test for administrative burdens . Perspectives on Public Management and Governance 3 ( 4 ): 273 – 87 .

George , Bert , Sanjay K. Pandey , Bram Steijn , Adelien Decramer , and Mieke Audenaert . 2020 . Red tape, organizational performance and employee outcomes: Meta‐analysis, meta‐regression and research agenda . Public Administration Review 81 ( 4 ): 638 – 51 .

Gilad , Sharon , and Michaela Assouline . 2022 . Citizens’ choice to voice in response to administrative burdens . International Public Management Journal : 1 – 22 . doi: 10.1080/10967494.2022.2072988

Griffiths , Rita. 2021 . Universal credit and automated decision making: A case of the digital tail wagging the policy dog ? Social Policy and Society : 1 – 18 .

Haeder , Simon F. , Steven M. Sylvester , and Timothy Callaghan . 2021 . Lingering legacies: Public attitudes about Medicaid beneficiaries and work requirements . Journal of Health Politics, Policy and Law 46 ( 2 ): 305 – 55 .

Halling , Aske. nd . Taxing language: Do interpreting fees affect healthcare usage? Working Paper.

Halling , Aske , Pamela Herd , and Donald Moynihan . 2022 . How difficult should it be? Evidence of burden tolerance from a nationally representative sample . Public Management Review : 1 – 20 . doi: 10.1080/14719037.2022.2056910

Halling , Aske , and Niels Bjørn Grund Petersen . nd . Burden feedback: When citizens communicate burdens, frontline employees respond evidence from an experiment . Working Paper.

Hattke , Fabian , Judith Hattke , David Hensel , Pamela Herd , Janne Kaluzca , Donald P. Moynihan , and Rick Vogel . nd . Face-to-face with administrative burdens: Physiological measures and behavioral consequences of psychological costs . Working Paper.

Hattke , Fabian , David Hensel , and Janne Kalucza . 2020 . Emotional responses to bureaucratic red tape . Public Administration Review 80 ( 1 ): 53 – 63 .

Heinrich , Carolyn J. 2016 . The bite of administrative burden: A theoretical and empirical investigation . Journal of Public Administration Research and Theory 26 ( 3 ): 403 – 20 .

———. 2018 . Presidential address: “A thousand petty fortresses”: Administrative burden in US immigration policies and its consequences . Journal of Policy Analysis and Management 37 ( 2 ): 211 – 39 .

Heinrich , Carolyn J. , and Robert Brill . 2015 . Stopped in the name of the law: Administrative burden and its implications for cash transfer program effectiveness . World Development 72 ( August ): 277 – 95 .

Heinrich , Carolyn J. , Sayil Camacho , Kaitlin Binsted , and Shadlan Gale . 2022 . An audit test evaluation of state practices for supporting access to and promoting Covid-19 vaccinations . Social Science and Medicine 301 : 114880 .

Herd , Pamela , Thomas DeLeire , Hope Harvey , and Donald P. Moynihan . 2013 . Shifting administrative burden to the state: The case of Medicaid take-up . Public Administration Review 73 ( s1 ): s69 – 81 .

Herd , Pamela , and Donald P. Moynihan . 2018 . Administrative burden: Policymaking by other means , 1st ed. New York : Russell Sage Foundation .

Hock , Heinrich , John T. Jones , Michael Levere , and David Wittenburg . 2021 . Using behavioral outreach to counteract administrative burden and encourage take-up of simplified disability payment rules . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 15 .

Holler , Roni , and Noam Tarshish . 2022 . Administrative burden in citizen-state encounters: The role of waiting, communication breakdowns and administrative errors . Social Policy and Society : 1 – 18 .

Hoynes , Hilary , Diane W. Schanzenbach , and Douglas Almond . 2016 . Long-run impacts of childhood access to the safety net . American Economic Review 106 ( 4 ): 903 – 34 .

Jakobsen , Morten , Oliver James , Donald P. Moynihan , and Tina Nabatchi . 2016 . JPART virtual issue on citizen-state interactions in public administration research . Journal of Public Administration Research and Theory 29 ( 4 ): e8 – 15 .

Jenkins , Jade Marcus , and Tutrang Nguyen . 2022 . Keeping kids in care: Reducing administrative burden in state child care development fund policy . Journal of Public Administration Research and Theory 32 ( 1 ): 23 – 40 .

Jilke , Sebastian , Wouter Van Dooren , and Sabine Rys . 2018 . Discrimination and administrative burden in public service markets: Does a public-private difference exist ? Journal of Public Administration Research and Theory 28 ( 3 ): 423 – 39 .

Johnson , Donavon , and Alexander Kroll . 2021 . What makes us tolerant of administrative burden? Race, representation, and identity . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 9 .

Kaufmann , Wesley , Alex Ingrams , and Daan Jacobs . 2021 . Being consistent matters: Experimental evidence on the effect of rule consistency on citizen red tape . American Review of Public Administration 51 ( 1 ): 28 – 39 .

Keiser , Lael R. , and Susan M. Miller . 2020 . Does administrative burden influence public support for government programs? Evidence from a survey experiment . Public Administration Review 80 ( 1 ): 137 – 50 .

Kyle , Michael Anne , and Austin B. Frakt . 2021 . Patient administrative burden in the US health care system . Health Services Research 56 ( 5 ): 755 – 65 .

Larsson , Karl Kristian. 2021 . Digitization or equality: When government automation covers some, but not all citizens . Government Information Quarterly 38 ( 1 ): 101547 – 10 .

Lasky-Fink , Jessica , and Linos . Elizabeth . 2023 . Improving delivery of the social safety net: The role of stigma . Journal of Public Administration Research and Theory .

Linos , Elizabeth , Lisa T. Quan , and Elspeth Kirkman . 2020 . Nudging early reduces administrative burden: Three field experiments to improve code enforcement . Journal of Policy Analysis and Management 39 ( 1 ): 243 – 65 .

Linos , Elizabeth , Vikash Reddy , and Jesse Rothstein . 2022 . Demystifying college costs: How nudges can and can’t help . Behavioral Public Policy : 1 – 22 .

Linos , Elizabeth , and Nefara Riesch . 2020 . Thick red tape and the thin blue line: A field study on reducing administrative burden in police recruitment . Public Administration Review 80 ( 1 ): 92 – 103 .

Linos , Katerina , Melissa Carlson , Laura Jakli , Nadia Dalma , Isabelle Cohen , Afroditi Veloudaki , and Stavros Nikiforos Spyrellis . 2022 . How do disadvantaged groups seek information about public services? A randomized controlled trial of communication technologies . Public Administration Review 82 ( 4 ): 708 – 20 .

Lipsky , Michael. 1980 . Street-level bureaucracy: Dilemmas of the individual in public services . New York : Russell Sage Foundation .

Lopoo , Leonard M. , Colleen Heflin , and Joseph Boskovski . 2020 . Testing behavioral interventions designed to improve on-time SNAP recertification . Journal of Behavioral Public Administration 3 ( 2 ): 1 – 8 .

Madsen , Christian Østergaard , Ida Lindgren , and Ulf Melin . 2022 . The accidental caseworker—How digital self-service influences citizens’ administrative burden . Government Information Quarterly 39 ( 1 ): 101653 .

Madsen , Jonas Krogh. nda . Compliant but discouraged? How administrative burden influence unemployment benefit recipients’ job search motivation . Working Paper.

———. ndb . Frictions on both sides of the counter? A study of red tape among street-level bureaucrats and administrative burden among their clients . Working Paper.

Madsen , Jonas Krogh , Martin Baekgaard , and Jon Kvist . 2022 . Scarcity and the mindsets of social welfare recipients: Evidence from a field experiment . Journal of Public Administration Research and Theory 33 : 675 – 87 .

Madsen , Jonas Krogh , and Kim Sass Mikkelsen . 2022 . How salient administrative burden affects job seekers’ locus of control and responsibility attribution: Evidence from a survey experiment . International Public Management Journal 25 ( 2 ): 241 – 60 .

Madsen , Jonas Krogh , Kim Sass Mikkelsen , and Donald P. Moynihan . 2022 . Burdens, sludge, ordeals, red tape, oh my! A user’s guide to the study of frictions . Public Administration 100 ( 2 ): 375 – 93 .

Masood , Ayesha , and Muhammed Azfar Nisar . 2021 . Administrative capital and citizens’ responses to administrative burden . Journal of Public Administration Research and Theory 31 ( 1 ): 56 – 72 .

Mikkelsen , Kim Sass , Jonas Krogh Madsen , and Martin Baekgaard . 2023 . Is stress among street level bureaucrats associated with experiences of administrative burden among clients? A multilevel study of the Danish unemployment sector . Public Administration Review 1 – 13 . doi: 10.1111/puar.13673

Moynihan , Donald , Eric Giannella , Pamela Herd , and Julie Sutherland . 2022 . Matching to categories: Learning and compliance costs in administrative processes . Journal of Public Administration Research and Theory 32 ( 4 ): 750 – 64 .

Moynihan , Donald , and Joe Soss . 2014 . Policy feedback and the politics of administration . Public Administration Review 74 ( 3 ): 320 – 32 .

Moynihan , Donald P. , Pamela Herd , and Hope Harvey . 2015 . Administrative burden: Learning, psychological, and compliance costs in citizen-state interactions . Journal of Public Administration Research and Theory 25 ( 1 ): 43 – 69 .

Moynihan , Donald P. , Pamela Herd , and Elizabeth Ribgy . 2016 . Policymaking by other means: Do states use administrative barriers to limit access to Medicaid . Administration & Society 48 ( 4 ): 497 – 524 .

Newman , Mark E. J. 2003 . Mixing patterns in networks . Physical Review 67 ( 026126 ): 1 – 13 .

Nicholson-Crotty , Jill , Susan M. Miller , and Lael R. Keiser . 2021 . Administrative burden, social construction, and public support for government programs . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 29 .

Nisar , Muhammad A. 2018 . Children of a lesser god: Administrative burden and social equity in citizen-state interactions . Journal of Public Administration Research and Theory 28 ( 1 ): 104 – 19 .

Nisar , Muhammed A. , and Ayesha Masood . nd . Governance by artifacts: Theory and evidence on materiality of administrative burdens . Working Paper.

Olsen , Asmus Leth. 2017 . Human interest or hard numbers? Experiments on citizens’ selection, exposure, and recall of performance information . Public Administration Review 77 ( 3 ): 408 – 20 .

Olsen , Asmus Leth , Jonas Høgh Kyhse-Andersen , and Donald Moynihan . 2020 . The unequal distribution of opportunity: A national audit study of bureaucratic discrimination in primary school access . American Journal of Political Science , Early view version, 66 ( 3 ): 587 – 603 .

Page , Matthew J. , Joanne E. McKenzie , Patrick M. Bossuyt , Isabelle Boutron , Tammy C. Hoffmann , Cynthia D. Mulrow , Larissa Shamseer , et al.  2021 . The PRISMA 2020 statement: An updated guideline for reporting systematic reviews . BMJ 372 : n71 .

Peeters , Rik. 2020 . The political economy of administrative burdens: A theoretical framework for analyzing the organizational origins of administrative burden . Administration & Society 52 ( 4 ): 566 – 92 .

Peeters , Rik , and Sergio A. Campos . 2021 . Taking the bite out of administrative burdens: How beneficiaries of a Mexican social program ease administrative burdens in street-level interactions . Governance 34 ( 4 ): 1001 – 18 .

Peeters , Rik , Humberto Trujillo Jimenez , Elizabeth O’Connor , Pascual Ogarrio Rojas , Michele Gonzalez Galindo , and Daniela Morales Tenorio . 2018 . Low-trust bureaucracy: Understanding the Mexican bureaucratic experience . Public Administration and Development 38 ( 2 ): 65 – 74 .

Peeters , Rik , César Renteria , and Guillermo M. Cejudo . nd . How information capacity shapes administrative burdens: A comparison of the citizen experience of the COVID-19 vaccination programs in the United States, Mexico, and the Netherlands . Working Paper.

Peeters , Rik , and Arjan Widlak . 2018 . The digital cage: Administrative exclusion through information architecture—The case of the Dutch civil registry’s master data management system . Government Information Quarterly 35 : 175 – 83 .

——— . 2023 . Administrative exclusion in the infrastructure-level bureaucracy: The case of the Dutch daycare benefit scandal . Public Administration Review 83 ( 4 ): 863 – 877 . doi: 10.1111/puar.13615

Perianes-Rodriguez , Antonio , Ludo Waltman , and Nees Jan van Eck . 2016 . Constructing bibliometric networks: A comparison between full and fractional counting . Journal of Informetrics 10 ( 4 ): 1178 – 95 .

Petersen , Ole Helby , Jesper Rosenberg Hansen , and Kurt Houlberg . 2022 . The administrative burden of doing business with the government: Learning and compliance costs in Business-Government interactions . Public Administration Review 1 – 19 . doi: 10.1111/padm.12904

Schneider , Anne , and Helen Ingram . 1993 . Social construction of target populations: Implications for politics and policy . American Political Science Review 87 ( 2 ): 334 – 47 .

Selin , Jennifer L. 2019 . The best laid plans: How administrative burden complicates voting rights restoration . Missouri Law Review 84 ( 4 ): 1 – 38 .

Sievert , Martin , and Jonas Bruder . 2023 . Unpacking the effects of burdensome state actions on citizens’ policy perceptions . Public Administration 1 – 21 .

Sievert , Martin , Dominik Vogel , and Mary K. Feeney . 2020 . Formalization and administrative burden as obstacles to employee recruitment: Consequences for the public sector . Review of Public Personnel Administration 42 ( 1 ): 3 – 30 .

Soss , Joe. 1999 . Lessons of welfare: Policy design, political learning, and political action . American Political Science Review 93 ( 2 ): 363 – 80 .

Soss , Joe , Richard Fording , and Sanford F. Schram . 2011 . The organization of discipline: From performance management to perversity and punishment . Journal of Public Administration Research and Theory 21 : i203 – 32 .

Thomsen , Mette Kjaergaard , Martin Baekgaard , and Ulrich Thy Jensen . 2020 . The psychological costs of citizen coproduction . Journal of Public Administration Research and Theory 30 ( 4 ): 656 – 73 .

Widlak , Arjan , and Rik Peeters . 2020 . Administrative errors and the burden of correction and consequence: How information technology exacerbates the consequences of bureaucratic mistakes for citizens . International Journal of Electronic Governance 12 ( 1 ): 1 – 56 .

Yates , Sophie , Gemma Carey , Eleanor Malbon , and Jen Hargrave . 2022 . “Faceless monster, secret society”: Women’s experiences navigating the administrative burden of Australia’s National Disability Insurance Scheme . Health and Social Care in the Community 50 ( 5 ): e2308 – 17 .

As the only exception, we excluded Herd and Moynihan (2018) from the review. The main points in this book have been covered in several journal articles by the authors and including it would therefore introduce the risk of double-counting arguments.

Many of the working papers were later published. The initial number of working papers was 30.

Since this feedback effect is mainly inspired by policy feedback research, for the sake of simplicity we chose not to show this as an independent arrow in the model.

Supplementary data

Email alerts, citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1477-9803
• Print ISSN 1053-1858
• Copyright © 2024 Public Management Research Association
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Retirement planning – a systematic review of literature and future research directions

• Published: 28 October 2023

Cite this article

• Kavita Karan Ingale   ORCID: orcid.org/0000-0003-3570-4211 1 &
• Ratna Achuta Paluri 2  

510 Accesses

Explore all metrics

Rising life expectancy and an aging population across nations are leading to an increased need for long-term financial savings and a focus on the financial well-being of retired individuals amidst changing policy framework. This study is a systematic review based on a scientific way of producing high-quality evidence based on 191 articles from the Scopus and Web of Science databases. It adopts the Theory, Context, Characteristics, and Method (TCCM) framework to analyze literature. This study provides collective insights into financial decision-making for retirement savings and identifies constructs for operationalizing and measuring financial behavior for retirement planning. Further, it indicates the need for an interdisciplinary approach. Though cognitive areas were studied extensively, the non-cognitive areas received little attention. Qualitative research design is gaining prominence in research over other methods, with the sparse application of mixed methods design. The study’s TCCM framework explicates several areas for further research. Furthermore, it guides the practice and policy by integrating empirical evidence and concomitant findings. Coherent synthesis of the extant literature reconciles the highly fragmented field of retirement planning. No research reports prospective areas for further analysis based on the TCCM framework on retirement planning, which highlights the uniqueness of the study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

A Research Proposal to Examine Psychological Factors Influence on Financial Planning for Retirement in China

Domains and determinants of retirement timing: A systematic review of longitudinal studies

Micky Scharn, Ranu Sewdas, … Allard J. van der Beek

Reinventing Retirement

Deanna L. Sharpe

Data Availability

The research data will be made available on request.

Acknowledgment.

Elderly population is defined as a population aged 65 years and over.

Defined benefit plan guarantees benefits to the employee, while defined contribution plan requires employees to decide on their own investment and bear the financial risks identified with it.

“The old-age dependency ratio is defined as the number of individuals aged 65 and over per 100 people of working age defined as those at ages 20 to 64”(OECD 2023 ).

Adams GA, Rau BL (2011) Putting off tomorrow to do what you want today: planning for Retirement. Am Psychol 66(3):180–192. https://doi.org/10.1037/a0022131

Article   Google Scholar  

Aegon Cfor, Longevity, Retirement ICR (2016) The Aegon Retirement Readiness Survey 2016. In The Aegon Retirement Readiness Survey 2016 . https://www.aegon.com/contentassets/c6a4b1cdded34f1b85a4f21d4c66e5d3/2016-aegon-retirement-readiness-report-india.pdf

Agarwalla SK, Barua SK, Jacob J, Varma JR (2015) Financial Literacy among Working Young in Urban India. World Development , 67 (2013), 101–109. https://doi.org/10.1016/j.worlddev.2014.10.004

Ajzen I (1991) The theory of Planned Behavior. Organ Behav Hum Decis Process 50:179–211. https://doi.org/10.47985/dcidj.475

Anderson A, Baker F, Robinson DT (2017) Precautionary savings, retirement planning, and misperceptions of financial literacy. J Financ Econ 126(2):383–398. https://doi.org/10.1016/j.jfineco.2017.07.008

Atkinson A, Messy FA (2011) Assessing financial literacy in 12 countries: an OECD/INFE international pilot exercise. J Pension Econ Finance 10(4):657–665. https://doi.org/10.1017/S1474747211000539`

Aydin AE, Akben Selcuk E (2019) An investigation of financial literacy, money ethics, and time preferences among college students: a structural equation model. Int J Bank Mark 37(3):880–900. https://doi.org/10.1108/IJBM-05-2018-0120

Bapat D (2020) Antecedents to responsible financial management behavior among young adults: the moderating role of financial risk tolerance. Int J Bank Mark 38(5):1177–1194. https://doi.org/10.1108/IJBM-10-2019-0356

Beckett A, Hewer P, Howcroft B (2000) An exposition of consumer behaviour in the financial services industry. Int J Bank Mark 18(1):15–26. https://doi.org/10.1108/02652320010315325

Białowolski P (2019) Economic sentiment as a driver for household financial behavior. J Behav Experimental Econ 80(August 2017):59–66. https://doi.org/10.1016/j.socec.2019.03.006

Binswanger J, Carman KG (2012) How real people make long-term decisions: the case of retirement preparation. J Economic Behav Organ 81(1):39–60. https://doi.org/10.1016/j.jebo.2011.08.010

Brounen D, Koedijk KG, Pownall RAJ (2016) Household financial planning and savings behavior. J Int Money Finance 69:95–107. https://doi.org/10.1016/j.jimonfin.2016.06.011

Brown R, Jones M (2015) Mapping and exploring the topography of contemporary financial accounting research. Br Acc Rev 47(3):237–261. https://doi.org/10.1016/j.bar.2014.08.006

Brown S, Gray D (2016) Household finances and well-being in Australia: an empirical analysis of comparison effects. J Econ Psychol 53:17–36. https://doi.org/10.1016/j.joep.2015.12.006

Brown S, Taylor K (2014) Household finances and the big five personality traits. J Econ Psychol 45:197–212. https://doi.org/10.1016/j.joep.2014.10.006

Brown S, Taylor K (2016) Early influences on saving behaviour: analysis of British panel data. J Bank Finance 62:1–14. https://doi.org/10.1016/j.jbankfin.2015.09.011

Brüggen EC, Post T, Schmitz K (2019) Interactivity in online pension planners enhances engagement with retirement planning – but not for everyone. J Serv Mark 33(4):488–501. https://doi.org/10.1108/JSM-02-2018-0082

Bruggen E, Post T, Katharina S (2019) Interactivity in online pension planners enhances engagement with retirement planning but not for everyone. J Serv Mark 33(4):488–501

Calcagno R, Monticone C (2015) Financial literacy and the demand for financial advice. J Bank Finance 50:363–380. https://doi.org/10.1016/j.jbankfin.2014.03.013

Campbell JY (2006) Household finance. J Finance 61(4):1553–1604. https://doi.org/10.1111/j.1540-6261.2006.00883.x

Choudhury K (2015) Service quality and customers’ behavioural intentions: class and mass banking and implications for the consumer and society. Asia Pac J Mark Logistics 27(5):735–757

Chowdhry N, Jung J, Dholakia U (2018) Association for consumer research. Adv Consum Res 42:42–46

Google Scholar  

Clark GL, Knox-Hayes J, Strauss K (2009) Financial sophistication, salience, and the scale of deliberation in UK retirement planning. Environ Plann A 41(10):2496–2515. https://doi.org/10.1068/a41265

Clark R, Lusardi A, Mitchell OS (2017) Employee Financial Literacy and Retirement Plan Behavior: a case study. Econ Inq 55(1):248–259. https://doi.org/10.1111/ecin.12389

Collins JM, Urban C (2016) The role of information on Retirement Planning: evidence from a field study. Econ Inq 54(4):1860–1872. https://doi.org/10.1111/ecin.12349

Creswell J (2009) Research Design Qualitative Quantitative and Mixed Methods Approaches. In Sage Publishing: Vol. Third edit . https://doi.org/10.1002/tl.20234

Csorba L (2020) The determining factors of financial culture, financial literacy, and financial behavior. Public Finance Q 65:67–83. https://doi.org/10.35551/PFQ_2020_1_6

Davidoff T, Gerhard P, Post T (2017) Reverse mortgages: what homeowners (don’t) know and how it matters. J Economic Behav Organ 133:151–171. https://doi.org/10.1016/j.jebo.2016.11.007

Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly: Management Information Systems 13(3):319–339. https://doi.org/10.2307/249008

Devlin J (2001) Consumer evaluation and competitive advantage in retail financial services - a research agenda. Eur J Mark 35(5/6):639–660

Dholakia U, Tam L, Yoon S, Wong N (2016) The ant and the grasshopper: understanding personal saving orientation of consumers. J Consum Res 43(1):134–155. https://doi.org/10.1093/jcr/ucw004

Dolls M, Doerrenberg P, Peichl A, Stichnoth H (2018) Do retirement savings increase in response to information about retirement and expected pensions? J Public Econ 158(July 2017):168–179. https://doi.org/10.1016/j.jpubeco.2017.12.014

Dragos SL, Dragos CM, Muresan GM (2020) From intention to the decision in purchasing life insurance and private pensions: different effects of knowledge and behavioural factors. J Behav Experimental Econ 87(March):101555. https://doi.org/10.1016/j.socec.2020.101555

Drever AI, Odders-white E, Kalish CW, Hoagland EM, Nelms EN, Drever AI, Odders-white E, Charles W, Else-quest NM, Hoagland EM, Nelms EN (2015) Foundations of Financial Weil-Being: Insights into the Role of Executive Function, Financial Socialization, and Experience-Based Learning in Childhood and Youth Source : The Journal of Consumer Affairs, Vol. 49, No. 1, Special Issue on Starting Ea. The Journal of Consumer Affairs , 49 (1)

Duflo E, Saez E (2002) Participation and investment decisions in a retirement plan: the influence of colleagues’ choices. J Public Econ 85(1):121–148. https://doi.org/10.1016/S0047-2727(01)00098-6

Duxbury D, Summers B, Hudson R, Keasey K (2013) How people evaluate defined contribution, annuity-based pension arrangements: a behavioral exploration. J Econ Psychol 34:256–269. https://doi.org/10.1016/j.joep.2012.10.008

Earl J, Bednall T, Muratore A (2015) A matter of time: why some people plan for retirement and others do not. Work Aging and Retirement 1(2):181–189. https://doi.org/10.1093/workar/wau005

Employees Benefits Research Institute (2020) EBRI Retirement Confidence Survey Report (Issue 202)

Engel JF, Kollat DT, Blackwell RD (1968) A model of consumer motivation and behavior. In: Research in consumer behavior. Holt, Rinehart and Winston, Inc., New York, pp 3–20

Erasmus A, Boshoff E, Rousseau G (2001) Consumer decision-making models within the discipline of consumer science: a critical approach. J Family Ecol Consumer Sci /Tydskrif Vir Gesinsekologie En Verbruikerswetenskappe 29(1):82–90. https://doi.org/10.4314/jfecs.v29i1.52799

Farrell L, Fry TRL, Risse L (2016) The significance of financial self-efficacy in explaining women’s personal finance behaviour. J Econ Psychol 54:85–99

Fernandes D, Lynch JG, Netemeyer RG (2014) Financial literacy, financial education, and downstream financial behaviors. Manage Sci 60(8):1861–1883. https://doi.org/10.1287/mnsc.2013.1849

Filbec G, Ricciardi V, Evensky H, Fan S, Holzhauer H, Spieler A (2017) Behavioral finance: a panel discussion. J Behav Experimental Finance 15:52–58. https://doi.org/10.1016/j.jbef.2015.07.003

Fishbein M (1979) A theory of reasoned action: some applications and implications. Nebraska Symposium on Motivation 27:65–116

Fisher PJ, Montalto CP (2010) Effect of saving motives and horizon on saving behaviors. J Econ Psychol 31(1):92–105. https://doi.org/10.1016/j.joep.2009.11.002

Flores SAM, Vieira KM (2014) Propensity toward indebtedness: an analysis using behavioral factors. J Behav Exp Finance 3:1–10

Foxall GR, Pallister JG (1998) Measuring purchase decision involvement for financial services: comparison of the Zaichkowsky and Mittal scales. Int J Bank Mark 16(5):180–194. https://doi.org/10.1108/02652329810228181

Friedman M (1957) Introduction to “A theory of the consumption function”. In: A theory of the consumption function. Princeton University Press, pp 1–6

Frydman C, Camerer CF (2016) The psychology and neuroscience of financial decision making. Trends Cogn Sci 20(9):661–675. https://doi.org/10.1016/j.tics.2016.07.003

Gardarsdóttir RB, Dittmar H (2012) The relationship of materialism to debt and financial well-being: the case of Iceland’s perceived prosperity. J Econ Psychol 33(3):471–481. https://doi.org/10.1016/j.joep.2011.12.008

Gathergood J (2012) Self-control, financial literacy and consumer over-indebtedness. J Econ Psychol 33(3):590–602

Gerhard P, Gladstone JJ, Hoffmann AOI (2018) Psychological characteristics and household savings behavior: the importance of accounting for latent heterogeneity. J Economic Behav Organ 148:66–82. https://doi.org/10.1016/j.jebo.2018.02.013

Gibbs PT (2009) Time, temporality, and

Goedde-Menke M, Lehmensiek-Starke M, Nolte S (2014) An empirical test of competing hypotheses for the annuity puzzle. J Econ Psychol 43:75–91

Gough O, Nurullah M (2009) Understanding what drives the purchase decision in pension and investment products. J Financial Serv Mark 14(2):152–172. https://doi.org/10.1057/fsm.2009.14

Griffin B, Loe D, Hesketh B (2012) Using Proactivity, Time Discounting, and the theory of Planned Behavior to identify predictors of Retirement Planning. Educ Gerontol 38(12):877–889. https://doi.org/10.1080/03601277.2012.660857

Gritten A (2011) New insights into consumer confidence in financial services. Int J Bank Mark 29(2):90–106. https://doi.org/10.1108/02652321111107602

Grohmann A (2018) Financial literacy and financial behavior: Evidence from the emerging Asian middle class. Pacific Basin Finance Journal , 48 (November 2017), 129–143. https://doi.org/10.1016/j.pacfin.2018.01.007

Grohmann A, Kouwenberg R, Menkhoff L (2015) Childhood roots of financial literacy. J Econ Psychol 51:114–133. https://doi.org/10.1016/j.joep.2015.09.002

Hair JF, Sarstedt M, Ringle CM, Mena JA (2012) An assessment of the use of partial least squares structural equation modeling in marketing research . 414–433. https://doi.org/10.1007/s11747-011-0261-6

Hanna SD, Kim KT, Chen SCC (2016) Retirement savings. In: Handbook of consumer finance research, pp 33–43

Harrison T, Waite K, White P (2006) Analysis by paralysis: the pension purchase decision process. Int J Bank Mark 24(1):5–23. https://doi.org/10.1108/02652320610642317

Hastings J, Mitchell O (2011) How financial literact and impatience shape retirement wealth and investment behaviors. Pengaruh Harga Diskon Dan Persepsi Produk Terhadap Nilai Belanja Serta Perilaku Pembelian Konsumen, NBER Working paper, 1–28

Hauff J, Carlander A, Amelie G, Tommy G, Holmen M (2016) Breaking the ice of low financial involvement: does narrative information format from a trusted sender increase savings in mutual funds? Int J Bank Mark 34(2):151–170

Hentzen JK, Hoffmann A, Dolan R, Pala E (2021) Artificial intelligence in customer-facing financial services: a systematic literature review and agenda for future research. Int J Bank Mark. https://doi.org/10.1108/IJBM-09-2021-0417

Hershey DA, Mowen JC (2000) Psychological determinants of financial preparedness for retirement. Gerontologist 40(6):687–697. https://doi.org/10.1093/geront/40.6.687

Hershey DA, Henkens K, Van Dalen HP (2007) Mapping the minds of retirement planners: a cross-cultural perspective. J Cross-Cult Psychol 38(3):361–382. https://doi.org/10.1177/0022022107300280

Hershey DA, Jacobs-Lawson JM, McArdle JJ, Hamagami F (2007b) Psychological foundations of financial planning for retirement. J Adult Dev 14(1–2):26–36. https://doi.org/10.1007/s10804-007-9028-1

Hershey DA, Jacobs-Lawson JM, McArdle JJ, Hamagami F (2008) Psychological foundations of financial planning for retirement. J Adult Dev 14(1–2):26–36. https://doi.org/10.1007/s10804-007-9028-1

Hershfield H, Goldstein D, Sharpe W, Fox J, Yeykelis L, Carstensen L, Bailenson J (2011) Increasing saving behavior through age-progressed renderings of the future self. J Mark Res 48:23–37

Hoffmann AOI, Broekhuizen TLJ (2009) Susceptibility to and impact of interpersonal influence in an investment context. J Acad Mark Sci 37:488–503

Hoffmann AOI, Broekhuizen TLJ (2010) Understanding investors’ decisions to purchase innovative products: drivers of adoption timing and range. Int J Res Mark 27(4):342–355. https://doi.org/10.1016/j.ijresmar.2010.08.002

Hoffmann AOI, Plotkina D (2020a) Positive framing when assessing the personal resources to manage one’s finances increases consumers’ retirement self-efficacy and improves retirement goal clarity. Psychol Mark 38(12):2286–2304. https://doi.org/10.1002/mar.21563

Hoffmann AOI, Plotkina D (2020b) Why and when does financial information affect retirement planning intentions and which consumers are more likely to act on them? Journal of Business Research , 117 (September 2019), 411–431. https://doi.org/10.1016/j.jbusres.2020.06.023

Hoffmann AOI, Plotkina D (2021) Let your past define your future. How recalling successful financial experiences can increase beliefs of self-efficacy in financial planning. J Consum Aff 55(3):847–871. https://doi.org/10.1111/joca.12378

Hoffmann AOI, Risse L (2020) Do good things come in pairs? How personality traits help explain individuals’ simultaneous pursuit of a healthy lifestyle and financially responsible behavior. J Consum Aff 54(3):1082–1120. https://doi.org/10.1111/joca.12317

Hsiao YJ, Tsai WC (2018) Financial literacy and participation in the derivatives markets. J Bank Finance 88:15–29

Huhmann BA, McQuitty S (2009) A model of consumer financial numeracy. Int J Bank Mark 27(4):270–293. https://doi.org/10.1108/02652320910968359

Huston SJ (2010) Measuring financial literacy. J Consum Aff 44(2):296–316. https://doi.org/10.1111/j.1745-6606.2010.01170.x

Ijevleva K, Arefjevs I (2014) Analysis of the Aggregate Financial Behaviour of customers using the Transtheoretical Model of Change. Procedia - Social and Behavioral Sciences 156(April):435–438. https://doi.org/10.1016/j.sbspro.2014.11.217

Ingale KK, Paluri RA (2020) Financial literacy and financial behavior: a bibliometric analysis. Rev Behav Finance. https://doi.org/10.1108/RBF-06-2020-0141

Jacobs-Lawson J, Hershey D (2005) Influence of future time perspective, financial knowledge, and financial risk tolerance on retirement savings behavior. Financial Serv Rev 14:331–344. https://doi.org/10.1088/1751-8113/44/8/085201

Jappelli T, Padula M (2013) Investment in financial literacy and saving decisions. J Bank Finance 37(8):2779–2792. https://doi.org/10.1016/j.jbankfin.2013.03.019

Kadoya Y, Rahim Khan MS (2020) Financial literacy in Japan: new evidence using financial knowledge, behavior, and attitude. Sustain (Switzerland) 12(9). https://doi.org/10.3390/su12093683

Kamil NSSN, Musa R, Sahak SZ (2014) Examining the Role of Financial Intelligence Quotient (FiQ) in explaining credit card usage behavior: a conceptual Framework. Procedia - Social and Behavioral Sciences 130:568–576. https://doi.org/10.1016/j.sbspro.2014.04.066

Kerry MJ (2018) Psychological antecedents of retirement planning: a systematic review. Front Psychol 9(OCT). https://doi.org/10.3389/fpsyg.2018.01870

Kerry MJ, Embretson SE (2018) An experimental evaluation of competing age predictions of future time perspective between workplace and retirement domains. Front Psychol 8(JAN):1–9. https://doi.org/10.3389/fpsyg.2017.02316

Kiliyanni AL, Sivaraman S (2016) The perception-reality gap in financial literacy: evidence from the most literate state in India. Int Rev Econ Educ 23:47–64. https://doi.org/10.1016/j.iree.2016.07.001

Kimiyaghalam F, Mansori S, Safari M, Yap S (2017) Parents’ influence on retirement planning in Malaysia. Family Consumer Sci Res J 45(3):315–325

Klapper L, Lusardi A, Panos GA (2013) Financial literacy and its consequences: evidence from Russia during the financial crisis. J Bank Finance 37(10):3904–3923

Koehler DJ, Langstaff J, Liu WQ (2015) A simulated financial savings task for studying consumption and retirement decision-making. J Econ Psychol 46:89–97. https://doi.org/10.1016/j.joep.2014.12.004

Kramer MM (2016) Financial literacy, confidence, and financial advice seeking. Journal of Economic Behavior and Organization , 131 (June 2015), 198–217. https://doi.org/10.1016/j.jebo.2016.08.016

Kumar S, Tomar S, Verma D (2019) Women’s financial planning for retirement: systematic literature review and future research agenda. Int J Bank Mark 37(1):120–141. https://doi.org/10.1108/IJBM-08-2017-0165

Kwon KN, Lee J (2009) The effects of reference point, knowledge, and risk propensity on the evaluation of financial products. J Bus Res 62(7):719–725. https://doi.org/10.1016/j.jbusres.2008.07.002

Landerretche OM, Martínez C (2013) Voluntary savings, financial behavior, and pension finance literacy: evidence from Chile. J Pension Econ Finance 12(3):251–297. https://doi.org/10.1017/S1474747212000340

Lee T (2017) (David). Clear, conspicuous, and improving: US corporate websites for critical financial literacy in retirement. International Journal of Bank Marketing , 35 (5), 761–780. https://doi.org/10.1108/IJBM-01-2016-0010

Liang C-J, Wang Wen‐Hung, Farquhar JD (2009) (2009). The influence of customer perceptions on financial performance in financial services. International Journal of Bank Marketing , 27 (2), 129–149

Liberman N, Trope Y (2003) Construal level theory of intertemporal judgment and decision. In: Loewenstein G, Read D, Baumeister R (eds) Time and decision: economic and psychological perspectives on intertemporal choice, pp 245–276

Lim KL, Soutar GN, Lee JA (2013) Factors affecting investment intentions: a consumer behaviour perspective. J Financ Serv Mark 18:301–315

Lin C, Hsiao YJ, Yeh CY (2017) Financial literacy, financial advisors, and information sources on demand for life insurance. Pac Basin Finance J 43(March):218–237. https://doi.org/10.1016/j.pacfin.2017.04.002

Lown JM (2011) Development and validation of a Financial Self-Efficacy Scale. J Financial Couns Plann 22(2):54–63

Lusardi A, Mitchell OS (2007) Baby Boomer retirement security: the roles of planning, financial literacy, and housing wealth. J Monet Econ 54(1):205–224. https://doi.org/10.1016/j.jmoneco.2006.12.001

Maloney M, McCarthy A (2017) Understanding pension communications at the organizational level: insights from bounded rationality theory & implications for HRM. Hum Resource Manage Rev 27(2):338–352. https://doi.org/10.1016/j.hrmr.2016.08.001

Marjanovic Z, Fiksenbaum L, Greenglass E (2018) Financial threat correlates with acute economic hardship and behavioral intentions that can improve one’s personal finances and health. J Behav Experimental Econ 77(April):151–157. https://doi.org/10.1016/j.socec.2018.09.012

Marques S, Mariano J, Lima ML, Abrams D (2018) Are you talking to the future me? The moderator role of future self-relevance on the effects of aging salience in retirement savings. J Appl Soc Psychol 48(7):360–368. https://doi.org/10.1111/jasp.12516

McKechnie S (1992) Consumer buying behaviour in financial services: an overview. Int J Bank Mark 10(5):5–39. https://doi.org/10.1108/02652329210016803

Milner T, Rosenstreich D (2013a) A review of consumer decision-making models and development of a new model for financial services. J Financial Serv Mark 18(2):106–120. https://doi.org/10.1057/fsm.2013.7

Milner T, Rosenstreich D (2013b) Insights into mature consumers of financial services. J Consumer Mark 30(3):248–257. https://doi.org/10.1108/07363761311328919

Mitchell OS, Mukherjee A (2017) Assessing the demand for micro pensions among India’s poor. J Econ Ageing 9:30–40. https://doi.org/10.1016/j.jeoa.2016.05.004

Mitchell O, Utkus S (2003) Lessons from Behavioral Finance for Retirement Plan Design (PRC WP 2003-6). http://prc.wharton.upenn.edu/prc/prc.html

Modigliani F, Brumberg RH (1954) Utility analysis and the consumption function: an interpretation of cross-section data. In: Kurihara KK (ed) Post-Keynesian economics. Rutgers University Press, New Brunswick, pp 388–436

Moher D, Liberati A, Tetzlaff J, Altman DG, Altman D, Antes G, Atkins D, Barbour V, Barrowman N, Berlin JA, Clark J, Clarke M, Cook D, D’Amico R, Deeks JJ, Devereaux PJ, Dickersin K, Egger M, Ernst E, …, Tugwell P (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(7). https://doi.org/10.1371/journal.pmed.1000097

Monti M, Pelligra V, Martignon L, Berg N (2014) Retail investors and financial advisors: new evidence on trust and advice taking heuristics. J Bus Res 67(8):1749–1757. https://doi.org/10.1016/j.jbusres.2014.02.022

Mouna A, Anis J (2017) Financial literacy in Tunisia: its determinants and its implications on investment behavior. Res Int Bus Finance 39:568–577

Mullainathan S, Thaler R (2000) Massachusetts Institute of Technology Department of Economics Working Paper Series . September

Nga KH, Yeoh KK (2018) An exploratory model on retirement savings behaviour: a Malaysian study. Int J Bus Soc 19(3):637–659

OECD (2023) Old-age dependency ratio (indicator). https://doi.org/10.1787/e0255c98-en . Accessed 13 Oct 2023

Onwuegbuzie AJ, Collins KM (2007) A typology of mixed methods sampling designs in social science research. Qualitative Rep 12(2):474–498

Pallister JG, Wang HC, Foxall GR (2007) An application of the style/involvement model to financial services. Technovation 27(1–2):78–88. https://doi.org/10.1016/j.technovation.2005.10.001

Pan L, Pezzuti T, Lu W, Pechmann C (2019) Hyperopia and frugality: different motivational drivers and yet similar effects on consumer spending. J Bus Res 95(August 2018):347–356

Parise G, Peijnenburg K (2017) Understanding the Determinants of Financial Outcomes and Choices: The Role of Noncognitive Abilities. BIS Working Papers

Paul J, Rosado-Serrano A (2019) Gradual internationalization vs Born-Global/International new venture models: a review and research agenda. Int Mark Rev 36(6):830–858. https://doi.org/10.1108/IMR-10-2018-0280

Paul J, Criado AR (2020) The art of writing literature review: what do we know and what do we need to know? Int Bus Rev 29(4):101717. https://doi.org/10.1016/j.ibusrev.2020.101717

Paul J, Khatri P, Kaur Duggal H (2023) Frameworks for developing impactful systematic literature reviews and theory building: what, why and how? J Decis Syst 00(00):1–14. https://doi.org/10.1080/12460125.2023.2197700

Petkoska J, Earl JK (2009) Understanding the influence of demographic and psychological variables on Retirement Planning. Psychol Aging 24(1):245–251. https://doi.org/10.1037/a0014096

Piotrowska M (2019) The importance of personality characteristics and behavioral constraints for retirement saving. Econ Anal Policy 64:194–220

Plath DA, Stevenson TH (2005) Financial services consumption behavior across Hispanic American consumers. J Bus Res 58(8):1089–1099. https://doi.org/10.1016/j.jbusres.2004.03.003

Poterba JM (2015) Saver heterogeneity and the challenge of assessing retirement saving adequacy. Natl Tax J 68(2):377–388. https://doi.org/10.17310/ntj.2015.2.06

Potrich ACG, Vieira KM, Kirch G (2018) How well do women do when it comes to financial literacy? Proposition of an indicator and analysis of gender differences. J Behav Experimental Finance 17:28–41. https://doi.org/10.1016/j.jbef.2017.12.005

Rai D, Lin CW (2019) (Wilson). The influence of implicit self-theories on consumer financial decision making. Journal of Business Research , 95 (August 2018), 316–325. https://doi.org/10.1016/j.jbusres.2018.08.016

Ramalho TB, Forte D (2019) Financial literacy in Brazil – do knowledge and self-confidence relate with behavior? RAUSP Manage J 54(1):77–95. https://doi.org/10.1108/RAUSP-04-2018-0008

Rana J, Paul J (2017) Consumer behavior and purchase intention for organic food: a review and research agenda. J Retailing Consumer Serv 38(June):157–165. https://doi.org/10.1016/j.jretconser.2017.06.004

Ranyard R, McNair S, Nicolini G, Duxbury D (2020) An item response theory approach to constructing and evaluating brief and in-depth financial literacy scales. J Consum Aff 54(3):1121–1156. https://doi.org/10.1111/joca.12322

RBI Household Finance Committee (2017) Indian household finance. Reserve Bank of India, Mumbai

Ruefenacht M, Schlager T, Maas P, Puustinen P (2015) Drivers of long-term savings behavior from consumer’s perspective. Electron Libr 34(1):1–5

Scholz JK, Seshadri A, Khitatrakun S (2006) Are Americans saving “optimally” for retirement? J Polit Econ 114(4):607–643

Schuabb T, França LH, Amorim SM (2019) Retirement savings model tested with Brazilian private health care workers. Front Psychol 10(JULY):1–11. https://doi.org/10.3389/fpsyg.2019.01701

Schuhen M, Schurkmann S (2014) International Review of Economics Education. Int Rev Econ Educ 16:1–11

Segel-Karpas D, Werner P (2014) Perceived financial retirement preparedness and its correlates: a national study in Israel. Int J Aging Hum Dev 79(4):279–301. https://doi.org/10.1177/0091415015574177

Seth H, Talwar S, Bhatia A, Saxena A, Dhir A (2020) Consumer resistance and inertia of retail investors: Development of the resistance adoption inertia continuance (RAIC) framework. Journal of Retailing and Consumer Services , 55 (August 2019), 102071. https://doi.org/10.1016/j.jretconser.2020.102071

Sewell M (2008) Behavioural finance. Economist 389(8604):1–13. https://doi.org/10.1057/9780230280786_5

Shefrin HM, Thaler RH (1988) The behavioral life‐cycle hypothesis. Econ Inq 26(4):609–643

Shim S, Serido J, Tang C (2012) The ant and the grasshopper revisited: the present psychological benefits of saving and future oriented financial behavior. J Econ Psychol 33(1):155–165

Simon HA (1978) Information-processing theory of human problem solving. In: Handbook of learning and cognitive processes, vol 5, pp 271–295

Sivaramakrishnan S, Srivastava M, Rastogi A (2017) Attitudinal factors, financial literacy, and stock market participation. Int J Bank Mark 34(1):1–5

Snyder H (2019) Literature review as a research methodology: an overview and guidelines. J Bus Res 104(August):333–339. https://doi.org/10.1016/j.jbusres.2019.07.039

Stawski RS, Hershey DA, Jacobs-Lawson JM (2007) Goal clarity and financial planning activities as determinants of retirement savings contributions. Int J Aging Hum Dev 64(1):13–32. https://doi.org/10.2190/13GK-5H72-H324-16P2

Steinert JI, Zenker J, Filipiak U, Movsisyan A, Cluver LD, Shenderovich Y (2018) Do saving promotion interventions increase household savings, consumption, and investments in Sub-saharan Africa? A systematic review and meta-analysis. World Dev 104:238–256. https://doi.org/10.1016/j.worlddev.2017.11.018

Steinhart Y, Mazursky D (2010) Purchase availability and involvement antecedents among financial products. Int J Bank Mark 28(2):113–135. https://doi.org/10.1108/02652321011018314

Strömbäck C, Lind T, Skagerlund K, Västfjäll D, Tinghög G (2017) Does self-control predict financial behavior and financial well-being? J Behav Experimental Finance 14:30–38. https://doi.org/10.1016/j.jbef.2017.04.002

Strömbäck C, Skagerlund K, Västfjäll D, Tinghög G (2020) Subjective self-control but not objective measures of executive functions predict financial behavior and well-being. Journal of Behavioral and Experimental Finance , 27 . https://doi.org/10.1016/j.jbef.2020.100339

Tam L, Dholakia U (2014) Saving in cycles: how to get people to save more money. Psychol Sci 25(2):531–537. https://doi.org/10.1177/0956797613512129

Tang N, Baker A (2016) Self-esteem, financial knowledge and financial behavior. J Econ Psychol 54:164–176

Tate M, Evermann J, Gable G (2015) An integrated framework for theories of individual attitudes toward technology. Inform Manage 52(6):710–727. https://doi.org/10.1016/j.im.2015.06.005

Taylor MP, Jenkins SP, Sacker A (2011) Financial capability and psychological health. J Econ Psychol 32(5):710–723. https://doi.org/10.1016/j.joep.2011.05.006

Tennyson S, Yang HK (2014) The role of life experience in long-term care insurance decisions. Journal of Economic Psychology , 42 (2014), 175–188. https://doi.org/10.1016/j.joep.2014.04.002

Thaler BRH (1994) Psychology and savings policies. Am Econ Rev 84(2):175–179. http://www.jstor.org/stable/3132220

Thaler R (1980) Toward a positive theory of consumer choice. J Econ Behav Organ 1:39–60

Thaler RH (2005) Advances in behavioral finance. Adv Behav Finance 2:1–694. https://doi.org/10.2307/2329257

Thaler R, Shefrin H (1981) An economic theory of self-control. J Polit Econ 89(2):392–406

Tomar S, Kent Baker H, Kumar S, Hoffmann AOI (2021) Psychological determinants of retirement financial planning behavior. Journal of Business Research , 133 (November 2020), 432–449. https://doi.org/10.1016/j.jbusres.2021.05.007

Topa G, Moriano JA, Depolo M, Alcover CM, Morales JF (2009) Antecedents and consequences of retirement planning and decision-making: a meta-analysis and model. J Vocat Behav 75(1):38–55. https://doi.org/10.1016/j.jvb.2009.03.002

Topa G, Moriano JA, Depolo M, Alcover CM, Moreno A (2011) Retirement and wealth relationships: Meta-analysis and SEM. Res Aging 33(5):501–528. https://doi.org/10.1177/0164027511410549

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br J Manag 14(3):207–222. https://doi.org/10.1111/1467-8551.00375

Ülkümen G, Cheema A (2011) Framing goals to influence personal savings: the role of specificity and construal level. J Mark Res 48(6):958–969. https://doi.org/10.1509/jmr.09.0516

United Nations, Department of Economic and Social, Affairs PD (2020) (2019). World Population Ageing 2019. In United Nations . http://link.springer.com/chapter/10.1007/ 978-94-007-5204-7_6

Utkarsh, Pandey A, Ashta A, Spiegelman E, Sutan A (2020) Catch them young: impact of financial Socialization, financial literacy and attitude towards money on the financial well-being of young adults. Int J Consumer Stud 44(6):531–541. https://doi.org/10.1111/ijcs.12583

Valente TW, Paredes P, Poppe P (1998) Matching the message to the process: the relative ordering of knowledge, attitudes, and practices in behavior change research. Hum Commun Res 24(3):366–385

Van Rooij M, Teppa F (2014) Personal traits and individual choices: taking action in economic and non-economic decisions. J Econ Behav Organ 100:33–43

van Rooij M, Lusardi A, Alessie R (2011) Financial literacy and stock market participation. J Financ Econ 101(2):449–472. https://doi.org/10.1016/j.jfineco.2011.03.006

Van Rooij MCJ, Lusardi A, Alessie RJM (2011a) Financial literacy and retirement planning in the Netherlands. J Econ Psychol 32(4):593–608. https://doi.org/10.1016/j.joep.2011.02.004

van Schie RJG, Dellaert BGC, Donkers B (2015) Promoting later planned retirement: construal level intervention impact reverses with age. J Econ Psychol 50:124–131. https://doi.org/10.1016/j.joep.2015.06.010

Venkatesh V, Morris M, Davis G, Davis F (2003) Factors influencing the Use of M-Banking by academics: Case Study sms-based M-Banking. MIS Q 27(3):425–478

Vitt LA (2004) Consumers’ financial decisions and the psychology of values. J Financial Service Professionals 58(November):68–77. http://search.ebscohost.com/login.aspx?direct=true &db=bth&AN=14888952&site=ehost-live

Wang L, Lu W, Malhotra NK (2011) Demographics, attitude, personality, and credit card features correlate with credit card debt: a view from China. J Econ Psychol 32(1):179–193. https://doi.org/10.1016/j.joep.2010.11.006

World Economic Forum (2019) Investing in (and for) our future. Issue June. www.weforum.org

Xia T, Wang Z, Li K (2014) Financial literacy overconfidence and stock market participation. Soc Indic Res 119(3):1233–1245. https://doi.org/10.1007/s11205-013-0555-9

Xiao JJ, Chen C, Chen F (2014) Consumer financial capability and financial satisfaction. Soc Indic Res 118(1):415–432. https://doi.org/10.1007/s11205-013-0414-8

Yeung DY, Zhou X (2017) Planning for retirement: longitudinal effect on retirement resources and post-retirement well-being. Front Psychol 8:1300

Zhou R, Pham MT (2004) Promotion and prevention across mental accounts: when financial products dictate consumers’ investment goals. J Consum Res 31(1):125–135. https://doi.org/10.1086/383429

Download references

Acknowledgements

Authors would like to acknowledge the academicians and researchers who guided the search of the article and would like to thank the experts for the valuable inputs to refine the work.

There is no funding received for this research.

Author information

Authors and affiliations.

Symbiosis International (Deemed University), Pune, India

Kavita Karan Ingale

Symbiosis Institute of Operations Management, Symbiosis International (Deemed) University, Pune, India

Ratna Achuta Paluri

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors contributed to the conceptualization, research design, methodology, analysis of the data,writing of the manuscript and its revision.

Corresponding author

Correspondence to Kavita Karan Ingale .

Ethics declarations

Conflict of interest.

The authors declare that there is no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Ingale, K.K., Paluri, R.A. Retirement planning – a systematic review of literature and future research directions. Manag Rev Q (2023). https://doi.org/10.1007/s11301-023-00377-x

Download citation

Received : 14 December 2022

Accepted : 04 October 2023

Published : 28 October 2023

DOI : https://doi.org/10.1007/s11301-023-00377-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Retirement planning
• Systematic literature review
• Financial behavior
• Household finance
• Long-term savings
• Pension plan
• Financial literacy
• TCCM framework
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 03 April 2024

Capturing artificial intelligence applications’ value proposition in healthcare – a qualitative research study

• Jasmin Hennrich 1 ,
• Eva Ritz 2 ,
• Peter Hofmann 1 , 4 &
• Nils Urbach 1 , 3  

BMC Health Services Research volume  24 , Article number:  420 ( 2024 ) Cite this article

892 Accesses

Metrics details

Artificial intelligence (AI) applications pave the way for innovations in the healthcare (HC) industry. However, their adoption in HC organizations is still nascent as organizations often face a fragmented and incomplete picture of how they can capture the value of AI applications on a managerial level. To overcome adoption hurdles, HC organizations would benefit from understanding how they can capture AI applications’ potential.

We conduct a comprehensive systematic literature review and 11 semi-structured expert interviews to identify, systematize, and describe 15 business objectives that translate into six value propositions of AI applications in HC.

Our results demonstrate that AI applications can have several business objectives converging into risk-reduced patient care, advanced patient care, self-management, process acceleration, resource optimization, and knowledge discovery.

We contribute to the literature by extending research on value creation mechanisms of AI to the HC context and guiding HC organizations in evaluating their AI applications or those of the competition on a managerial level, to assess AI investment decisions, and to align their AI application portfolio towards an overarching strategy.

Peer Review reports

Applications based on artificial intelligence (AI) have the potential to transform the healthcare (HC) industry [ 1 ]. AI applications can be characterized as applications or agents with capabilities that typically demand intelligence [ 2 , 3 ]. In our context, we understand AI as a collection of technological solutions from the field of applied computer science, in which algorithms are trained on medical and HC data to perform tasks that are normally associated with human intelligence (i.e., medical decision-making) [ 4 ]. AI is not a single type of technology, instead, it encompasses a diverse array of technologies spread across various application areas in HC, such as diagnostics (e.g., [ 5 ], biomedical research (e.g., [ 6 ], clinical administration (e.g., [ 7 ], therapy (e.g., [ 8 ], and intelligent robotics (e.g., [ 9 ]. These areas are expected to benefit from AI applications’ capabilities, such as accuracy, objectivity, rapidity, data processing, and automation [ 10 , 11 ]. Accordingly, AI applications are said to have the potential to drive business value and enhance HC [ 12 ], paving the way for transformative innovations in the HC industry [ 13 ]. There are already many promising AI use cases in HC that are expected to improve patient care and create value for HC organizations. For instance, AI applications can advance the quality of patient care by supporting radiologists with more accurate and rapid diagnosis, compensating for humans’ limitations (e.g., data processing speeds) and weaknesses (e.g., inattention, distraction, and fatigue) [ 10 , 14 ]. Klicken oder tippen Sie hier, um Text einzugeben.While the use of AI applications in HC has the overarching goal of creating significant value for patients through improved care, they also come with the potential for business value creation and the opportunity for HC organizations to gain a competitive edge (e.g., [ 15 , 16 ]).

Despite the promised advantages, AI applications’ implementation is slow, and the full realization of their potential within the HC industry is yet to be achieved [ 11 , 17 ]. With just a handful of practical examples of AI applications in the HC industry [ 13 , 18 ], the adoption of AI applications is still in its infancy. The AI in Healthcare Survey Report stated that in 2021, only 9% of respondents worldwide have reached a sophisticated adoption of AI Models, while 32% of respondents are still in the early stages of adopting AI models. According to the survey, the majority of HC organizations (60%) are not actively considering AI as a solution, or they are currently evaluating AI use cases and experimenting with the implementation [ 19 ]. Nevertheless, HC startups are increasingly entering the market [ 20 ], pressuring incumbent HC organizations to evaluate and adopt AI applications. Existing studies already investigate AI technologies in various use cases in HC and provide insights on how to design AI-based services [ 21 ], explain in detail the technical functions and capabilities of AI technologies [ 10 , 11 ], or take on a practical perspective with a focus on concrete examples of AI applications [ 14 ]. However, to foster the adoption of AI applications, HC organizations should understand how they can unfold AI applications’ capabilities into business value to ensure effective investments. Previous studies on the intersection of information systems and value creation have expressed interest into how organizations can actually gain value through the use of technology and thus, enhance their adoption [ 22 , 23 ]. However, to the best of our knowledge, a comprehensive investigation of the value creation of AI applications in the context of HC from a managerial level is currently missing. Thus, our study aims to investigate AI applications’ value creation and capture mechanisms in the specific HC context by answering the following question: How can HC organizations create and capture AI applications’ value?

We conduct a systematic literature analysis and semi structured expert interviews to answer this research question. In the systematic literature analysis, we identify and analyze a heterogeneous set of 21 AI use cases across five different HC application fields and derive 15 business objectives and six value propositions for HC organizations. We then evaluate and refine the categorized business objectives and value propositions with insights from 11 expert interviews. Our study contributes to research on the value creation mechanism of AI applications in the HC context. Moreover, our results have managerial implications for HC organizations since they can draw on our results to evaluate AI applications, assess investment decisions, and align their AI application portfolio toward an overarching strategy.

In what follows, this study first grounds on relevant work to gain a deeper understanding of the underlying constructs of AI in HC. Next, we describe our qualitative research method by describing the process of data collection and analysis, followed by our derived results on capturing AI applications’ value proposition in HC. Afterward, we discuss our results, including this study’s limitations and pathways for further research. Finally, we summarize our findings and their contribution to theory and practice in the conclusion.

Relevant work

In the realm of AI, a thorough exploration of its key subdiscipline, machine learning (ML), is essential [ 24 , 25 ]. ML is a computational model that learns from data without explicitly programming the data [ 24 ] and can be further divided into supervised, unsupervised, and reinforcement learning [ 26 ]. In supervised learning, the machine undergoes training with labeled data, making it well-suited for tasks involving regression and classification problems [ 27 ]. In contrast, unsupervised learning is designed to automatically identify patterns within unlabeled datasets [ 28 ], with its primary utility lying in the extraction of features [ 11 ]. Reinforcement learning, characterized as a method of systematic experimentation or trial and error, involves a situated agent taking specific actions and observing the rewards it gains from those actions, facilitating the learning of behavior in a given environment [ 29 ]. The choice of which type of ML will be used in the different application areas depends on the specific problem, the availability of labeled data, and the nature of the desired outcome.

In recent years, the rapid advances in AI have triggered a revolution in various areas, with numerous impressive advantages. In the financial sector, AI applications can significantly improve security by detecting anomalies and preventing fraud [ 30 ]. Within education, AI has emerged as a powerful tool for tailoring learning experiences, aiming to enhance engagement, understanding, and retention [ 31 ]. In the energy market, the efficacy of AI extends to fault detection and diagnosis in building energy systems, showcasing its robust capabilities in ensuring system integrity [ 32 ]. Moreover, the HC industry is expected to be a promising application area for AI applications. The HC sector is undergoing a significant transformation due to the increasing adoption of digital technologies, with AI technologies at the forefront of this shift. The increasing relevance of AI technologies in HC is underlined by a growing and multidisciplinary stream of AI research, as highlighted by Secinaro et al. [ 33 ]. Taking a closer look at the different application areas in HC, AI applications offer promising potential, as demonstrated by the following exemplary AI use cases. In diagnosis, AI applications can identify complex patterns in medical image data more accurately, resulting in precise and objective disease recognition. This can improve patient safety by reducing the risks of misinterpretation [ 5 ]. Another use case can be found in biomedical research. For example, AI technology is commonly used for de novo drug design. AI can rapidly browse through molecule libraries to detect nearly \({10}^{60}\) drug-like molecules, accelerating the drug development process [ 6 ]. Furthermore, AI applications are used in clinical administration. They enable optimized operation room capacities by automating the process and by including information about absence or waiting times, as well as predicting interruptions [ 34 ]. Furthermore, AI applications are used in therapy by predicting personalized medication dosages. As this helps to reduce the mortality risk, it leads to enhanced patient outcomes and quality of care [ 35 ]. Intelligent prostheses by which patients can improve interactions are another use case. The AI algorithm continuously detects and classifies myoelectric signal patterns to predict movements, leading to reduced training expenditure and more self-management by the patient [ 36 ]. In summary, envisioning that AI applications successfully address persisting challenges, such as lack of transparency (e.g., [ 37 ], bias (e.g., [ 38 ], privacy concerns, and trust issues (e.g., [ 39 ], the potential of AI applications is vast. The conceivable benefits extend to individual practitioners and HC organizations, including hospitals, enabling them to harness AI applications for creating business value and ultimately enhancing competitiveness. Thereby, we follow Schryen’s (p. 141) revisited definition of business value of technologies: “the impact of investments on the multidimensional performance and capabilities of economic entities at various levels, complemented by the ultimate meaning of performance in the economic environment” [ 40 ]. His perspective includes all kinds of tangible value (such as an increase in productivity or reduced costs) to intangible value (such as service innovation or customer satisfaction), as well as internal value for the HC organizations and external value for stakeholders, shareholders, and customers. To create business value, it is essential to have a clear understanding of how the potential of AI applications can be captured. The understanding of how information systems, in general, create value is already covered in the literature. For example, Badakhshan et al. [ 31 ] focus on how process mining can pave the way to create business value. Leidner et al. [ 32 ] examine how enterprise social media adds value for new employees, and Lehrer et al. [ 33 ] answer the question of how big data analytics can enable service. There are also studies focusing on the value creation of information systems in the context of HC. For instance, the study by Haddad and Wickramasinghe [ 41 ] shows that information technology in HC can capture value by improving the quality of HC delivery, increasing safety, or offering additional services. Strong et al. [ 42 ] analyze how electronic health records afford value for HC organizations and determine goal-oriented actions to capture this potential. There is even literature on how machine learning adds value within the discipline of radiology (e.g., [ 43 ].

However, these studies either do not address the context of HC, consider technologies other than AI or information systems in general, or focus only on a small area of HC (e.g., radiology) and a subset of AI technology (e.g., machine learning). Although these studies deliver valuable insights into the value creation of information systems, a comprehensive picture of how HC organizations can capture business value with AI applications is missing.

To answer our research question, we adopted a qualitative inductive research design. This research design is consistent with studies that took a similar perspective on how technologies can create business value [ 44 ]. In conducting our structured literature review, we followed the approach of Webster and Watson [ 45 ] and included recommendations of Wolfswinkel et al. [ 46 ] when considering the inclusion and exclusion criteria. We started by collecting relevant data on different successful AI use cases across five application areas in HC. Siggelkow [ 47 ] argued that use cases are able to provide persuasive arguments for causal relationships. In an initial literature screening, we identified five promising application domains focusing on AI applications for patients and HC providers: disease diagnostics (DD) (e.g., [ 5 ], biomedical research (BR) (e.g., [ 6 ], clinical administration (CA) (e.g., [ 7 ], therapy (T) (e.g., [ 8 ], and intelligent robotics (IR) (e.g., [ 9 ]. Second, to sample AI use cases, we aimed to collect a heterogeneous set of AI use cases within these application domains and consider the heterogeneity in AI applications, underlying data, innovation types, and implementation stages when selecting 21 AI use cases for our in-depth analysis. The AI use case and an exemplary study for each use case are listed in Table  1 .

After sampling the AI use cases, we used PubMed to identify papers for each use case. PubMed is recognized as a common database for biomedical and medical research for HC topics in the information systems domain (e.g., [ 62 , 63 ]. Our search included journal articles, clinical conferences, clinical studies, and comparative studies in English as of 2010. Based on the AI use case sample, we derived a search string based on keywords [ 45 ] considering titles and abstracts by following Shepherd et al. [ 62 ] guidelines. It was aimed to narrow and specific selection to increase data collection replicability for the use cases. Boolean operators (AND, OR) are used to improve results by combining search terms [ 62 ].

((artificial intelligence AND (radiology OR (cancer AND imaging) OR (radiology AND error) OR (cancer AND genomics) OR (speech AND cognitive AND impairment) OR (voice AND parkinson) OR EEG OR (facial AND analysis) OR (drug AND design) OR (Drug AND Biomarker) OR De-identification OR Splicing OR (emergency AND triage) OR (mortality AND prediction) OR (operating AND room) OR text summarization OR (artificial AND pancreas) OR vasopressor OR Chatbot OR (myoelectric prosthesis) OR (automated surgery task) OR (surgery AND workflow)))

The initial search led to 877 results (see Fig.  1 ). After title screening, we eliminated 516 papers that are not relevant (i.e., not covering a specific AI application, only including the description of AI algorithm, or not including a managerial perspective and the value created by AI applications). We further excluded 162 papers because their abstract is not concurrent with any specific use case (e.g., because they were literature reviews on overarching topics and did not include a specific AI application). We screened the remaining 199 papers for eligibility through two content-related criteria. First, papers need to cover an AI use case’s whole value proposition creation path, including information on data, algorithms, functions, competitive advantage, and business value of a certain AI application. The papers often only examine how a certain application works but lack the value proposition perspective, which leads to the exclusion of 63 articles. Second, we removed 89 papers that do not match any of our use cases. This step led to a remaining set of 47 relevant papers. During a backward-forward search according to Webster and Watson [ 45 ] and Levy and Ellis [ 64 ], we additionally included 35 papers. We also incorporated previous and subsequent clinical studies of the same researcher, resulting in an additional six papers. The final set contains 88 relevant papers describing the identified AI use cases, whereby at least three papers describe each AI use case.

Search strategy

In the second step, we engaged in open, axial, and selective coding of the AI use cases following analysis techniques of grounded theory [ 65 ]. We focused on extracting business objectives, detailing how each AI application drives value. We documented these for each AI use case by recording codes of business objectives and value propositions and assigning relationships among the open codes. For example, from the following text passage of Berlyand et al. [ 56 ], who investigate the use case CA1: “Rapidly interpreting clinical data to classify patients and predict outcomes is paramount to emergency department operations, with direct impacts on cost, efficiency, and quality of care”, we derived the code rapid task execution.

After analyzing the AI use cases, we revised the documented tuples to foster consistency and comparability. Then, we iteratively coded the identified tuples by relying on selective coding techniques which is a process to identify and refine categories at a highly generalizable degree [ 65 ]. In all 14 coding iterations, one author continuously compares, relates, and associates categories and properties and discusses the coding results with another author. We modified some tuples during the coding process in two ways. First, we equalized small phrasing disparities for homogenous and refined wording. Second, we carefully adjusted the tuples regarding coherency. Finally, we reviewed the coding schema for internal validity through a final comparison with the data [ 66 ]. Then, we set the core variables “business objectives” and “value propositions”. We refer to business objectives as improvements through implementing the technology that drives a value proposition. We define value proposition as the inherent commitment to deliver reciprocal value to the organization, its customers, and/or partners [ 67 ].

In the third step following Schultze and Avital [ 68 ], we conducted semi structured expert interviews to evaluate and refine the value propositions and business objectives. We developed and refined an interview script following the guidelines of Meyers and Newman [ 69 ] for qualitative interviews. An additional file shows the used interview script (see Additional file 1 ). We conducted expert sampling to select suitable interviewees [ 70 ]. Due to the interdisciplinarity of the research topic, we chose experts in the two knowledge areas, AI and HC. In the process of expert selection, we ensured that interviewees possessed a minimum of two years of experience in their respective fields. We aimed for a well-balanced mix of diverse professions and positions among the interviewees. Additionally, for those with a primary background in HC, we specifically verified their proficiency and understanding of AI, ensuring a comprehensive perspective across the entire expert panel. Table 2 provides an overview of our expert sample. The interviewees were recruited in the authors’ networks and by cold calling. Identified experts were first contacted by email, including some brief information regarding the study. If there was no response within two weeks, they were contacted again by telephone to arrange an interview date. In total, we conducted 11 interviews that took place in a time range between 40 and 75 min. The expert interviews are transcribed verbatim using the software f4. As a coding aid, we use the software MAXQDA—a tool for qualitative data analysis which is frequently used in the analyses of qualitative data in the HC domain (e.g., [ 38 , 71 , 72 ]).

To systematically decompose how HC organizations can realize value propositions from AI applications, we identified 15 business objectives and six value propositions (see Fig.  2 ). These business objectives and value propositions resulted from analyzing the collected data, which we derived from the literature and refined through expert interviews. In the following, we describe the six value propositions and elaborate on how the specific AI business objectives can result in value propositions. This will be followed by a discussion of the results in the discussion of the paper.

Business objectives and value propositions risk-reduced patient care

This value proposition follows business objectives that may identify and reduce threats and adverse factors during medical procedures. HC belongs to a high-risk domain since there are uncertain external factors (E4), including physicians’ fatigue, distractions, or cognitive biases [ 73 , 74 ]. AI applications can reduce certain risks by enabling precise decision support, detecting misconduct, reducing emergent side effects, and reducing invasiveness.

Precise decision support stems from AI applications’ capability to integrate various data types into the decision-making process, gaining a sophisticated overview of a phenomenon. Precise knowledge about all uncertainty factors reduces the ambiguity of decision-making processes [ 49 ]. E5 confirms that AI applications can be seen as a “perceptual enhancement”, enabling more comprehensive and context-based decision support. Humans are naturally prone to innate and socially adapted biases that also affect HC professionals [ 14 ]. Use Case CA1 highlights how rapid decision-making by HC professionals during emergency triage may lead to overlooking subtle yet crucial signs. AI applications can offer decision support based on historical data, enhancing objectivity and accuracy [ 56 ].

Detection of misconduct is possible since AI applications can map and monitor clinical workflows and recognize irregularities early. In this context, E10 highlights that “one of the best examples is the interception of abnormalities.” For instance, AI applications can assist in allocating medications in hospitals (Use case T2). Since HC professionals can be tired or distracted in medication preparation, AI applications may avoid serious consequences for patients by monitoring allocation processes and patients’ reactions. Thus, AI applications can reduce abuse and increase safety.

Reduction of emergent side effects is enabled by AI applications that continuously monitor and process data. If different treatments and medications are combined during a patient’s clinical pathway, it may cause overdosage or evoke co-effects and comorbidities, causing danger for the patient [ 75 ]. AI applications can prevent these by detecting and predicting these effects. For instance, AI applications can calculate the medication dosage for the individual and predict contraindications (Use case T2) [ 76 ]. E3 adds that the reduction of side effects also includes “cross-impacts between medications or possible symptoms that only occur for patients of a certain age or disease.” Avoidable side effects can thus be detected at an early stage, resulting in better outcomes.

Reduction of invasiveness of medical treatments or surgeries is possible by allowing AI applications to compensate for and overcome human weaknesses and limitations. During surgery, AI applications can continuously monitor a robot’s position and accurately predict its trajectories [ 77 ]. Intelligent robots can eliminate human tremors and access hard-to-reach body parts [ 60 ]. E2 validates, “a robot does not tremble; a robot moves in a perfectly straight line.” The precise AI-controlled movement of surgical robots minimizes the risk of injuring nearby vessels and organs [ 61 ]. Use cases DD5 and DD7 elucidate how AI applications enable new methods to perform noninvasive diagnoses. Reducing invasiveness has a major impact on the patient’s recovery, safety, and outcome quality.

Advanced patient care

Advanced patient care follows business objectives that extend patient care to increase the quality of care. One of HC’s primary goals is to provide the most effective treatment outcome. AI applications can advance patient care as they enable personalized care and accurate prognosis.

Personalized care can be enabled by the ability of AI technologies to integrate and process individual structured and unstructured patient data to increase the compatibility of patient and health interventions. For instance, by analyzing genome mutations, AI applications precisely assess cancer, enabling personalized therapy and increasing the likelihood of enhancing outcome quality (Use case DD4). E11 sums up that “we can improve treatment or even make it more specific for the patient. This is, of course, the dream of healthcare”. Use case T1 exemplifies how the integration of AI applications facilitates personalized products, such as an artificial pancreas. The pancreas predicts glucose levels in real time and adapts insulin supplementation. Personalized care allows good care to be made even better by tailoring care to the individual.

Accurate prognosis is achieved by AI applications that track, combine, and analyze HC data and historical data to make accurate predictions. For instance, AI applications can precisely analyze tumor tissue to improve the stratification of cancer patients. Based on this result, the selection of adjuvant therapy can be refined, improving the effectiveness of care [ 48 ]. Use case DD6 shows how AI applications can predict seizure onset zones to enhance the prognosis of epileptic seizures. In this context, E10 adds that an accurate prognosis fosters early and preventive care.

Self-management

Self-management follows the business objectives that increase disease controllability through the support of intelligent medical products. AI applications can foster self-management by self-monitoring and providing a new way of delivering information.

Self-monitoring is enhanced by AI applications, which can automatically process frequently measured data. There are AI-based chatbots, mobile applications, wearables, and other medical products that gather periodic data and are used by people to monitor themselves in the health context (e.g., [ 78 , 79 ]. Frequent data collection of these products (e.g., using sensors) enables AI applications to analyze periodic data and become aware of abnormalities. While the amount of data rises, the applications can improve their performance continuously (E2). Through continuous tracking of heartbeats via wearables, AI applications can precisely detect irregularities, notify their users in the case of irregularities, empower quicker treatment (E2), and may reduce hospital visits (E9). Self-monitoring enhances patient safety and allows the patient to be more physician-independent and involved in their HC.

Information delivery to the patient is enabled by AI applications that give medical advice adjusted to the patient’s needs. Often, patients lack profound knowledge about their anomalies. AI applications can contextualize patients’ symptoms to provide anamnesis support and deliver interactive advice [ 59 ]. While HC professionals must focus on one diagnostic pathway, AI applications can process information to investigate different diagnostic branches simultaneously (E5). Thus, these applications can deliver high-quality information based on the patient’s feedback, for instance, when using an intelligent conversational agent (use case T3). E4 highlights that this can improve doctoral consultations because “the patient is already informed and already has information when he comes to talk to doctors”.

Process acceleration

Process acceleration comprises business objectives that enable speed and low latencies. Speed describes how fast one can perform a task, while latency specifies how much time elapses from an event until a task is executed. AI applications can accelerate processes by rapid task execution and reducing latency.

Rapid task execution can be achieved by the ability of AI applications to process large amounts of data and identify patterns in a short time. In this context, E4 mentions that AI applications can drill diagnosis down to seconds. For instance, whereas doctors need several minutes for profound image-based detection, AI applications have a much faster report turnaround time (use case DD1). Besides, rapid data processing also opens up new opportunities in drug development. AI applications can rapidly browse through molecule libraries to detect nearly 10^60 molecules, which are synthetically available (use case BR1). This immense speed during a discovery process has an essential influence on the business potential and can enormously decrease research costs (E10).

Latency reduction can be enabled by AI technologies monitoring and dynamically processing information and environmental factors. By continuously evaluating vital signs and electrocardiogram records, AI applications can predict the in-house mortality of patients in real time [ 57 ]. The AI application can detect an increased mortality risk faster than HC professionals, enabling a more rapid emergency intervention. In this case, AI applications decrease the time delay between the cause and the reaction, which positively impacts patient care. E7 emphasizes the importance of short latencies: “One of the most important things is that the timeframe between the point when all the data is available, and a decision has been made, […] must be kept short.”

Resource optimization

Resource optimization follows the business objectives that manage limited resources and capacities. The HC industry faces a lack of sufficient resources, especially through a shortage of specialists (E8), which in turn negatively influences waiting times. AI applications can support efficient resource allocation by optimizing device utilization, organizational capacities and unleashing personnel capabilities.

Optimized device utilization can be enhanced by AI applications that track, analyze, and precisely predict load of times of medical equipment in real-time. For instance, AI applications can maximize X-Ray or magnetic resonance tomography device utilization (use case CA3). Besides, AI applications can enable a dynamic replanning of device utilization by including absence or waiting times and predicting interruptions. Intelligent resource optimization may include various key variables (e.g., the maximized lifespan of a radiation scanner) [ 48 ]. Optimized device utilization reduces the time periods when the device is not utilized, and thus, losses are made.

Optimized organizational capacities are possible due to AI applications breaking up static key performance indicators and finding more dynamic measuring approaches for the required workflow changes (E5, E10). The utilization of capacities in hospitals relies on various known and unknown parameters, which are often interdependent [ 80 ]. AI applications can detect and optimize these dependencies to manage capacity. An example is the optimization of clinical occupancy in the hospital (use case CA3), which has a strong impact on cost. E5 adds that the integration of AI applications may increase the reliability of planning HC resources since they can predict capacity trends from historical occupancy rates. Optimized planning of capacities can prevent capacities from remaining unused and fixed costs from being offset by no revenue.

Unleashing personnel capabilities is enabled by AI applications performing analytical and administrative tasks, relieving caregivers’ workload (E8, E10, E11). E7 validates that “our conviction is […] that administrational tasks generate the greatest added value and benefit for doctors and caregivers.” Administrative tasks include the creation of case summaries (use case CA4) or automated de-identification of private health information in electronic health records (use case BR2) [ 54 ]. E8 says that resource optimization enables “more time for direct contact with patients.”

Knowledge discovery

Knowledge discovery follows the business objectives that increase perception and access to novel and previously unrevealed information. AI applications might synthesize and contextualize medical knowledge to create uniform or equalized semantics of information (E5, E11). This semantics enables a translation of knowledge for specific users.

Detection of similarities is enabled by AI applications identifying entities with similar features. AI applications can screen complex and nonlinear databases to identify reoccurring patterns without any a priori understanding of the data (E3). These similarities generate valuable knowledge, which can be applied to enhance scientific research processes such as drug development (use case BR1). In drug development, AI applications can facilitate ligand-based screening to detect new active molecules based on similarities compared with already existing molecular properties. This increases the effectiveness of drug design and reduces risks in clinical trials [ 6 ].

Exploration of new correlations is facilitated by AI applications identifying relationships in data. In diagnostics, AI applications can analyze facial photographs to accurately identify genotype–phenotype correlations and, thus, increase the detection rate of rare diseases (use case DD7). E8 states the potential of AI applications in the field of knowledge discovery: “Well, if you are researching in any medical area, then everybody aims to understand and describe phenomena because science always demands a certain causation.” However, it is crucial to develop transparent and intelligible inferences that are comprehensible for HC professionals and researchers. Exploring new correlations improves diagnoses of rare diseases and ensures earlier treatment.

After describing each business objective and value proposition, we summarize the AI use cases’ contributions to the value propositions in Table  3 .

By revealing 15 business objectives that translate into six value propositions, we contribute to the academic discourse on the value creation of AI (e.g. [ 81 ] and provide prescriptive knowledge on AI applications' value propositions in the HC domain. Our discourse also emphasizes that our findings are not only relevant to the field of value creation research but can also be helpful for adoption research. The value propositions we have identified can be a good starting point to accelerate the adoption of AI in HC, as the understanding of potential value propositions that we foster could mitigate some of the current obstacles to the adoption of AI applications in HC. For example, our findings may help to mitigate the obstacle “added value”, which is presented in the study by Hennrich et al.38 [ 38 ] as users’ concerns that AI might create more burden than benefits.

Further, we deliver valuable implications for practice and provide a comprehensive picture of how organizations in the context of HC can achieve business value with AI applications from a managerial level, which has been missing until now. We guide HC organizations in evaluating their AI applications or those of the competition to assess AI investment decisions and align their AI application portfolio toward an overarching strategy. These results will foster the adoption of AI applications as HC organizations can now understand how they can unfold AI applications’ capabilities into business value. In case a hospital’s major strategy is to reduce patient risks due to limited personal capacities, it might be beneficial for them to invest in AI applications that reduce side effects by calculating medication dosages (use case T2). If an HC organization currently faces issues with overcrowded emergency rooms, the HC organization might acquire AI applications that increase information delivery and help patients decide if and when they should visit the hospital (use case T3) to increase patients’ self-management and, in turn, improve triage. Besides, our findings also offer valuable insights for AI developers. Addressing issues such as transparency and the alignment of AI applications with the needs of HC professionals is crucial. Adapting AI solutions to the specific requirements of the HC sector ensures responsible integration and thus the realization of the expected values.

A closer look at the current challenges in the HC sector reveals that new solutions to mitigate them and improve value creation are needed. Given that a nurse, for example, dedicates a substantial 25% of their working hours to administrative tasks [ 17 ], the rationale behind the respondents’ (E7) recognition of “the greatest added value” in utilizing AI applications for administrative purposes becomes evident. The potential of AI applications in streamlining administrative tasks lies in creating additional time for meaningful patient interactions. Acknowledging the significant impact of the doctor-patient interpersonal relationship on both the patient’s well-being and the processes of diagnosis and healing, as elucidated by Buck et al. [ 82 ] in their interview study, the physicians interviewed emphasized that the mere presence of the doctor in the same room often alleviates the patient’s problems. Consequently, it becomes apparent that the intangible value of AI applications plays a crucial role in the context of HC and is an important factor in the investment decision as to where an AI application should be deployed.

The interviews also indicate that the special context of the HC sector leads to concerns regarding the use of AI applications. For example, one interviewee emphasized a fundamental characteristic of medical staff by pointing out that physicians have a natural desire to understand all phenomena (E8). AI applications, however, are currently struggling with the challenge of transparency. This challenge is described by the so-called black box problem, a phenomenon that makes it impossible to decipher the underlying algorithms that lead to a particular recommendation [ 37 ]. The lack of transparency and the resulting lack of intervention options for medical staff can lead to incorrect decisions by the AI application, which may cause considerable damage. Aware of these risks, physicians are currently struggling with trust issues in AI applications [ 72 ]. The numerous opportunities for value creation through AI applications in HC are offset by the significant risk of causing considerable harm to patients if the technology is not yet fully mature. Ultimately, it remains essential to keep in mind that there are many ethical questions to be answered [ 83 ], and AI applications are still facing many obstacles [ 38 ] that must be overcome in order to realize the expected values and avoid serious harm. One important first step in mitigating the obstacles is disseminating the concerns and risks to relevant stakeholders, emphasizing the urgency for collaborative scientific and public monitoring efforts [ 84 ]. However, keeping these obstacles in mind, by providing prescriptive knowledge, we enhance the understanding of AI’s value creation paths in the HC industry and thus help to drive AI integration forward. For example, looking at the value proposition risk reduced patient care , we demonstrate that this value proposition is determined by four business objectives: precise decision support , detection of misconduct, reduction of side effects, and reduction of invasiveness . Similarly, the AI application’s capability to analyze data more accurately in diagnosis (use case DD1) enables the business objective precise decision support , thereby reducing risks in patient care. Another mechanism can be seen, for example, considering the business objective task execution , which leads to the value proposition process acceleration . The ability of AI applications to rapidly analyze large amounts of data and recognize patterns in biomedical research (use case BR1) allows a faster drug development process.

Further research

By investigating the value creation mechanism of AI applications for HC organizations, we not only make an important contribution to research and practice but also create a valuable foundation for future studies. While we have systematically identified the relations between the business objectives and value propositions, further research is needed to investigate how the business objectives themselves are determined. While the examination of AI capabilities was not the primary research focus, we found first evidence in the use cases that indicates AI technology’s unique capabilities (e.g., to make diagnoses accurate, faster, and more objective) that foster one or several business objectives (e.g., rapid task execution, precise decision support) and unlock one or several value propositions (e.g., Risk-reduced patient care, process acceleration ). In subsequent research, we aim to integrate these into the value creation mechanism by identifying which specific AI capabilities drive business objectives, thereby advancing the understanding of how AI applications in HC create value propositions.

Limitations

This study is subject to certain limitations of methodological and conceptual nature. First, while our methodological approach covers an in-depth analysis of 21 AI use cases, extending the sample of AI use cases would foster the generalizability of the results. This is especially important regarding the latest developments on generative AI and its newcoming use cases. However, our results demonstrate that these AI use cases already provide rich information to derive 15 business objectives, which translate into six value propositions. Second, while many papers assume the potential of AI applications to create value propositions, only a few papers explicitly focus on the value creation and capture mechanisms. To compensate for this paucity of appropriate papers, we used 11 expert interviews to enrich and evaluate the results. Besides, these interviews ensured the practical relevance and reliability of the derived results. Third, we acknowledge limitations of conceptual nature. Our study predominantly takes an optimistic perspective on AI applications in medicine. While we discuss the potential benefits and value propositions in detail, it is important to emphasize that there are still significant barriers and risks currently associated with AI applications that need to be addressed before the identified values can be realized. Furthermore, our investigation is limited because we derive the expected value of AI applications without having extensive real-world use cases to evaluate. It is important to emphasize that our findings are preliminary, and critical reassessment will be essential as the broader implementation of AI applications in medical practice progresses. These limitations emphasize the need for ongoing research and monitoring to understand the true value of AI applications in HC fully.

Conclusions

This study aimed to investigate how AI applications can create value for HC organizations. After elaborating on a diverse and comprehensive set of AI use cases, we are confident that AI applications can create value by making HC, among others, more precise, individualized, self-determined, faster, resource-optimized, and data insight-driven. Especially with regard to the mounting challenges of the industry, such as the aging population and the resulting increase in HC professionals’ workloads, the integration of AI applications and the expected benefits have become more critical than ever. Based on the systematic literature review and expert interviews, we derived 15 business objectives that translate into the following six value propositions that describe how HC organizations can capture the value of AI applications: risk-reduced patient care, advanced patient care, self-management, process acceleration, resource optimization, and knowledge discovery .

By presenting and discussing our results, we enhance the understanding of how HC organizations can unlock AI applications’ value proposition. We provide HC organizations with valuable insights to help them strategically assess their AI applications as well as those deployed by competitors at a management level. Our goal is to facilitate informed decision-making regarding AI investments and enable HC organizations to align their AI application portfolios with a comprehensive and overarching strategy. However, even if various value proposition-creating scenarios exist, AI applications are not yet fully mature in every area or ready for widespread use. Ultimately, it remains essential to take a critical look at which AI applications can be used for which task at which point in time to achieve the promised value. Nonetheless, we are confident that we can shed more light on the value proposition-capturing mechanism and, therefore, support AI application adoption in HC.

Availability of data and materials

The datasets analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Artificial Intelligence

Machine Learning

Fogel AL, Kvedar JC. Artificial intelligence powers digital medicine. NPJ Digit Med. 2018;1:5.

Article   PubMed   PubMed Central   Google Scholar  

Rai A, Constantinides P, Sarker S. Next-Generation Digital Platforms: Toward Human–AI Hybrid. Manag Inf Syst Q. 2019;43(1):iii–ix.

Google Scholar  

Russell S, Norvig P. Artificial Intelligence: A Modern Approach. Pearson Education Limited; 2016.

He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med. 2019;25(1):30–6.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer. 2018;18(8):500–10.

Kadurin A, Aliper A, Kazennov A, Mamoshina P, Vanhaelen Q, Khrabrov K, et al. The cornucopia of meaningful leads: Applying deep adversarial autoencoders for new molecule development in oncology. Oncotarget. 2017;8(7):10883–90.

Article   PubMed   Google Scholar  

Rezazade Mehrizi MH, van Ooijen P, Homan M. Applications of artificial intelligence (AI) in diagnostic radiology: a technography study. Eur Radiol. 2020;31:1805–11.

Dankwa-Mullan I, Rivo M, Sepulveda M, Park Y, Snowdon J, Rhee K. Transforming Diabetes Care Through Artificial Intelligence: The Future Is Here. Popul Health Manag. 2019;22(3):229–42.

Bohr A, Memarzadeh K. The rise of artificial intelligence in healthcare applications. In: Artificial Intelligence in Healthcare. Elsevier; 2020. p. 25–60.

Yu K-H, Beam AL, Kohane IS. Artificial Intelligence in Healthcare. Nat Biomed Eng. 2018;2(10):719–31.

Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2(4):230–43.

Gilvary C, Madhukar N, Elkhader J, Elemento O. The Missing Pieces of Artificial Intelligence in Medicine. Trends Pharmacol Sci. 2019;40(8):555–64.

Article   CAS   PubMed   Google Scholar  

Fernández E. Innovation in Healthcare: Harnessing New Technologies. Journal of the Midwest Association for Information Systems 2017; (2). Available from: http://aisel.aisnet.org/jmwais/vol2017/iss2/8 .

Topol EJ. Deep medicine: How artificial intelligence can make healthcare human again. 1st ed. New York: Basic Books; 2019.

Ngiam KY, Khor IW. Big data and machine learning algorithms for health-care delivery. Lancet Oncol. 2019;20(5):e262–73.

Plastino E, Purdy M. Game changing value from artificial intelligence: eight strategies. Strategy Leadership. 2018;46(1):16–22.

Article   Google Scholar  

Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthcare J. 2019;6(2):94–8.

Meskó B, Görög M. A short guide for medical professionals in the era of artificial intelligence. NPJ Digit Med. 2020;3:126.

Statista. What is the stage of AI adoption in your organization?; 2021. Available from: https://www.statista.com/statistics/1225955/stage-of-ai-adoption-in-healthcare-worldwide/ . Cited 2024 Feb. 8

Garbuio M, Lin N. Artificial Intelligence as a Growth Engine for Health Care Startups: Emerging Business Models. Calif Manage Rev. 2019;61(2):59–83.

Väänänen A, Haataja K, Vehviläinen-Julkunen K, Toivanen P. AI in healthcare A narrative review. F1000Res. 2021;10:6.

Kim H-W, Chan HC, Gupta S. Value-based Adoption of Mobile Internet: An empirical investigation. Decis Support Syst. 2007;43(1):111–26.

Lin TC, Wu S, Hsu JSC, Chou YC. The integration of value-based adoption and expectation–confirmation models An example of IPTV continuance intention. Decision Support Systems. 2012;54(1):63–75.

Jordan MI, Mitchell TM. Machine learning: Trends, perspectives, and prospects. Science. 2015;349(6245):255–60.

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.

Hamet P, Tremblay J. Artificial intelligence in medicine. Metab Clin Exp. 2017;69S:S36–40.

Lidströmer N, Aresu F, Ashrafian H. Basic Concepts of Artificial Intelligence Primed for Clinicians In Artificial Intelligence in Medicine. Cham: Springer; 2022. p. 3–20.

Choi RY, Coyner AS, Kalpathy-Cramer J, Chiang MF, Campbell JP. Introduction to machine learning, neural networks, and deep learning. Transl Vis Sci Technol. 2020;9(2):1–12.

Lorenz U. Reinforcement Learning: Aktuelle Ansätze verstehen - mit Beispielen in Java und Greenfoot. Berlin, Heidelberg: Springer Berlin Heidelberg; 2020.

Kunduru AR. Artificial intelligence advantages in cloud fintech application security. Central Asian J Mathematical Theory Comp Sci. 2023;4(8):48–53.

Zhai X, Chu X, Chai CS, Jong MSY, Istenic A, Spector M, et al. A Review of Artificial Intelligence (AI) in Education from 2010 to 2020. Complexity. 2021;2021:1–18.

Zhao Y, Li T, Zhang X, Zhang C. Artificial intelligence-based fault detection and diagnosis methods for building energy systems: Advantages, challenges and the future. Renew Sustain Energy Rev. 2019;109:85–101.

Secinaro S, Calandra D, Secinaro A, Muthurangu V, Biancone P. The role of artificial intelligence in healthcare: a structured literature review. BMC Med Inform Decis Mak. 2021;21(1):125.

Bellini V, Guzzon M, Bigliardi B, Mordonini M, Filippelli S, Bignami E. Artificial intelligence a new tool in operating room management role of machine learning models in operating room optimization. J Med Syst. 2019;44(1):20.

Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med. 2018;24(11):1716–20.

Roland T. Motion Artifact Suppression for Insulated EMG to Control Myoelectric Prostheses. Sensors (Basel). 2020;20(4):1031.

Amann J, Blasimme A, Vayena E, Frey D, Madai VI. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak. 2020;20(1):310.

Hennrich J, Fuhrmann H, Eymann T. Accelerating the Adoption of Artificial Intelligence Technologies in Radiology: A Comprehensive Overview on Current Obstacles. Proceedings of the 57th Hawaii International Conference on System Sciences 2024.

Asan O, Bayrak AE, Choudhury A. Artificial Intelligence and Human Trust in Healthcare: Focus on Clinicians. J Med Internet Res. 2020;22(6):e15154.

Schryen G. Revisiting IS business value research: what we already know, what we still need to know, and how we can get there. Eur J Inf Syst. 2013;22(2):139–69.

Haddad P, Wickramasinghe N. Conceptualizing Business Value of IT in Healthcare to Design Sustainable e-Health Solutions. Proceedings of Americas Conference on Information Systems 2014.

Strong DM, Volkoff O, Johnson SA, Pelletier LR, Tulu B, Bar-On I, et al. A Theory of Organization-EHR Affordance Actualization. J Assoc Inf Syst. 2014;15(2):53–85.

Hofmann P, Oesterle S, Rust P, Urbach N. Machine Learning Approaches Along the Radiology Value Chain - Rethinking Value Propositions. Proceedings of the European Conference on Information Systems 2019.

Badakhshan P, Wurm B, Grisold T, Geyer-Klingeberg J, Mendling J, Vom Brocke J. Creating business value with process mining. J Strateg Inf Syst. 2022;31(4):101745.

Webster J, Watson RT. Analyzing the Past to Prepare for the Future Writing a Literature Review. MIS Quarterly. 2002;26(2):xiii–xxiii.

Wolfswinkel JF, Furtmueller E, Wilderom CPM. Using Grounded Theory as a Method for Rigorously Reviewing Literature. Eur J Inf Syst. 2013;22(1):45–55.

Siggelkow N. Persuasion With Case Studies. AMJ. 2007;50(1):20–4.

Lakhani P, Prater AB, Hutson RK, Andriole KP, Dreyer KJ, Morey J, et al. Machine Learning in Radiology: Applications Beyond Image Interpretation. J Am Coll Radiol. 2018;15(2):350–9.

Low S-K, Zembutsu H, Nakamura Y. Breast cancer: The translation of big genomic data to cancer precision medicine. Cancer Sci. 2018;109(3):497–506.

Altay EV, Alatas B. Association analysis of Parkinson disease with vocal change characteristics using multi-objective metaheuristic optimization. Med Hypotheses. 2020;141:109722.

Abbasi B, Goldenholz DM. Machine learning applications in epilepsy. Epilepsia. 2019;60(10):2037–47.

Qin B, Quan Q, Wu J, Liang L, Li D. Diagnostic performance of artificial intelligence to detect genetic diseases with facial phenotypes: A protocol for systematic review and meta analysis. Medicine (Baltimore). 2020;99(27):e20989.

Zhavoronkov A, Mamoshina P, Vanhaelen Q, Scheibye-Knudsen M, Moskalev A, Aliper A. Artificial intelligence for aging and longevity research: Recent advances and perspectives. Ageing Res Rev. 2019;49:49–66.

Meystre SM, Friedlin FJ, South BR, Shen S, Samore MH. Automatic de-identification of textual documents in the electronic health record - a review of recent research. BMC Med Res Methodol. 2010;10(70):1–6.

Rhine CL, Neil C, Glidden DT, Cygan KJ, Fredericks AM, Wang J, et al. Future directions for high-throughput splicing assays in precision medicine. Hum Mutat. 2019;40(9):1225–34.

Berlyand Y, Raja AS, Dorner SC, Prabhakar AM, Sonis JD, Gottumukkala RV, et al. How artificial intelligence could transform emergency department operations. Am J Emerg Med. 2018;36(8):1515–7.

Kwon J-M, Kim K-H, Jeon K-H, Park J. Deep learning for predicting in-hospital mortality among heart disease patients based on echocardiography. Echocardiography. 2019;36(2):213–8.

Yang M, Li C, Shen Y, Wu Q, Zhao Z, Chen X. Hierarchical Human-Like Deep Neural Networks for Abstractive Text Summarization. IEEE Trans Neural Netw Learn Syst. 2021;32(6):2744–57.

Hernandez JPT. Network diffusion and technology acceptance of a nurse chatbot for chronic disease self-management support a theoretical perspective. J Med Invest. 2019;66(1.2):24–30.

Bhandari M, Zeffiro T, Reddiboina M. Artificial Intelligence and Robotic Surgery: Current Perspective and Future Directions. Curr Opin Urol. 2020;30(1):48–54.

Padoy N. Machine and deep learning for workflow recognition during surgery. Minim Invasive Ther Allied Technol. 2019;28(2):82–90.

Shepherd M, Abidi SSR, Gao Q, Chen Z, Qi Q, Finley GA. Information Systems and Health Care IX: Accessing Tacit Knowledge and Linking It to the Peer-Reviewed Literature. CAIS 2006; 17.

Wilson EV, Wang W, Sheetz SD. Underpinning a Guiding Theory of Patient-Centered E-Health. CAIS. 2014;34(1):16.

Levy Y, Ellis TJ. A systems approach to conduct an effective literature review in support of information systems research. Informing Sci J. 2006;9:181–211.

Corbin JM, Strauss AL. Basics of qualitative research: Techniques and procedures for developing grounded theory. Fourth edition. Thousand Oaks, Kalifornien: SAGE; 2015.

Glaser BG, Strauss A. Discovery of grounded theory: Strategies for qualitative research. Routledge; 1967.

Feldman S, Horan T. The dynamics of information collaboration: a case study of blended IT value propositions for health information exchange in disability determination. J Assoc Inf Syst. 2011;12(2):189–207.

Schultze U, Avital M. Designing interviews to generate rich data for information systems research. Inf Organ. 2011;21(1):1–16.

Myers MD, Newman M. The qualitative interview in IS research: examining the craft. Inf Organ. 2007;17(1):2–26.

Bhattacherjee A. Social Science Research: Principles, Methods, and Practices. Textbooks Collection 2012; 3.

Moulaei K, Sheikhtaheri A, Fatehi F, Shanbehzadeh M, Bahaadinbeigy K. Patients’ perspectives and preferences toward telemedicine versus in-person visits: a mixed-methods study on 1226 patients. BMC Med Inform Decis Mak. 2023;23(1):261.

Buck C, Hennrich J, Kauffmann A-L. Artificial Intelligence in Radiology – A Qualitative Study on Imaging Specialists’ Perspectives. Proceedings of the 42nd International Conference on Information Systems 2021.

Degnan AJ, Ghobadi EH, Hardy P, Krupinski E, Scali EP, Stratchko L, et al. Perceptual and interpretive error in diagnostic radiology-causes and potential solutions. Acad Radiol. 2019;26(6):833–45.

Pesapane F, Codari M, Sardanelli F. Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine. Eur Radiol Exp. 2018;2(1):35.

Schinkel M, Paranjape K, Nannan Panday RS, Skyttberg N, Nanayakkara PWB. Clinical applications of artificial intelligence in sepsis: A narrative review. Comput Biol Med. 2019;115:103488.

Shamim Nemati, Mohammad Ghassemi, Gari Clifford. Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach. International Conference of the IEEE Engineering in Medicine and Biology Society 2016. Available from: http://ieeexplore.ieee.org/servlet/opac?punumber=7580725 .

Padoy N, Hager GD. Human-Machine Collaborative Surgery Using Learned Methods. Proceedings of IEEE International Conference on Robotics and Automation 2011:5285–92.

Nadarzynski T, Miles O, Cowie A, Ridge D. Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: A mixed-methods study. Digit Health. 2019;5:2055207619871808.

PubMed   PubMed Central   Google Scholar  

Trevitt S, Simpson S, Wood A. Artificial Pancreas device systems for the closed-loop control of type 1 diabetes: what systems are in development? J Diabetes Sci Technol. 2016;10(3):714–23.

Luo L, Zhang F, Yao Y, Gong R, Fu M, Xiao J. Machine learning for identification of surgeries with high risks of cancellation. Health Informatics J. 2018;26(1):141–55.

Shollo A, Hopf K, Thiess T, Müller O. Shifting ML value creation mechanisms: a process model of ML value creation. J Strateg Inf Syst. 2022;31(3):101734.

Buck C, Doctor E, Hennrich J, Jöhnk J, Eymann T. General Practitioners’ Attitudes Toward Artificial Intelligence-Enabled Systems: Interview Study. J Med Internet Res. 2022;24(1):e28916.

Bennett SJ. Transmuting values in artificial intelligence: investigating the motivations and contextual constraints shaping the ethics of artificial intelligence practitioners; 2023.

Baumgartner R, Arora P, Bath C, Burljaev D, Ciereszko K, Custers B, et al. Fair and equitable AI in biomedical research and healthcare: Social science perspectives. Artif Intell Med. 2023;144:102658.

Download references

Acknowledgements

The authors thank all physicians who participated in this study.

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

FIM Research Institute for Information Management, University of Bayreuth, Branch Business and Information Systems Engineering of the Fraunhofer FIT, Wittelsbacherring 10, 95444, Bayreuth, Germany

Jasmin Hennrich, Peter Hofmann & Nils Urbach

University St. Gallen, Dufourstrasse 50, 9000, St. Gallen, Switzerland

Faculty Business and Law, Frankfurt University of Applied Sciences, Nibelungenplatz 1, 60318, Frankfurt Am Main, Germany

Nils Urbach

appliedAI Initiative GmbH, August-Everding-Straße 25, 81671, Munich, Germany

Peter Hofmann

You can also search for this author in PubMed   Google Scholar

Contributions

JH initially composed the introduction, background, and discussion sections; PH contributed to the analysis design, supported data analysis, and critically reviewed the manuscript; ER conducted data collection and analysis and produced the initial draft of the methodology section; NU edited and critically reviewed the manuscript and provided guidance. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jasmin Hennrich .

Ethics declarations

Ethics approval and consent to participate.

The study was based on original data. The authors confirm that all methods were carried out in accordance with relevant guidelines and regulations and confirm that informed consent was obtained from all participants. Ethics approval was granted by the Ethics Committee of the University of Bayreuth (Application-ID 23–032).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hennrich, J., Ritz, E., Hofmann, P. et al. Capturing artificial intelligence applications’ value proposition in healthcare – a qualitative research study. BMC Health Serv Res 24 , 420 (2024). https://doi.org/10.1186/s12913-024-10894-4

Download citation

Received : 26 October 2023

Accepted : 25 March 2024

Published : 03 April 2024

DOI : https://doi.org/10.1186/s12913-024-10894-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence
• Value propositions
• Business objectives

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Corporate activities that influence population health: A scoping review and qualitative synthesis to develop the HEALTH-CORP typology

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• ORCID record for Raquel Burgess
• For correspondence: [email protected]
• Info/History
• Supplementary material
• Preview PDF

Introduction: The concept of the commercial determinants of health (CDH) is used to study the actions (and associated structures) of commercial entities that influence population health and health equity. The aim of this study was to develop a typology that describes the diverse set of activities through which corporations influence population health and health equity across industries. Methods: We conducted a scoping review of articles using CDH terms (n=116) that discuss corporate activities that can influence population health and health equity across 16 industries. We used the qualitative constant comparison method to build a typology called the Corporate Influences on Population Health (HEALTH-CORP) typology. Results: The HEALTH-CORP typology identifies 70 corporate activities that can influence health across industries and categorizes them into seven domains of corporate influence (e.g., political practices, employment practices). We present a model that situates these domains based on their proximity to health outcomes and identify five population groups (e.g., workers, local communities) to consider when evaluating corporate health impacts. Discussion: The HEALTH-CORP typology facilitates an understanding of the diverse set of corporate activities that can influence population health and the population groups affected by these activities. We discuss the utility of these contributions in terms of identifying interventions to address the CDH and advancing efforts to measure and monitor the CDH. We also leverage our findings to identify key gaps in CDH literature and suggest avenues for future research.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

Raquel Burgess was supported by a Doctoral Foreign Study Award provided by the Canadian Institutes of Health Research at the time this research was conducted. Funding was provided by the Yale School of Public Health and the Yale Graduate Student Assembly to present this work at the American Public Health Association Annual Meeting in 2022.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Data Availability

The data for this study are published academic articles which are available from the respective publishers (see Supplementary Material, Appendix 2 for the characteristics of included articles). In addition, we uploaded the following files to Open Science Framework (DOI 10.17605/OSF.IO/TG9S7) to support data availability: 1) a .csv file containing a list of the articles that underwent title and abstract screening in our study and the respective screening decisions that were assigned, and 2) .ris files containing the citations to the respective articles and the assigned screening decisions, which can be uploaded into a reference manager. Interested parties can contact the corresponding author for additional information.

https://osf.io/tg9s7/

View the discussion thread.

Supplementary Material

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager
• Tweet Widget
• Facebook Like
• Google Plus One
• Addiction Medicine (316)
• Allergy and Immunology (620)
• Anesthesia (160)
• Cardiovascular Medicine (2284)
• Dentistry and Oral Medicine (280)
• Dermatology (201)
• Emergency Medicine (370)
• Endocrinology (including Diabetes Mellitus and Metabolic Disease) (805)
• Epidemiology (11593)
• Forensic Medicine (10)
• Gastroenterology (681)
• Genetic and Genomic Medicine (3600)
• Geriatric Medicine (337)
• Health Economics (618)
• Health Informatics (2311)
• Health Policy (916)
• Health Systems and Quality Improvement (865)
• Hematology (335)
• HIV/AIDS (753)
• Infectious Diseases (except HIV/AIDS) (13170)
• Intensive Care and Critical Care Medicine (758)
• Medical Education (360)
• Medical Ethics (100)
• Nephrology (391)
• Neurology (3369)
• Nursing (191)
• Nutrition (508)
• Obstetrics and Gynecology (652)
• Occupational and Environmental Health (647)
• Oncology (1764)
• Ophthalmology (526)
• Orthopedics (210)
• Otolaryngology (284)
• Pain Medicine (223)
• Palliative Medicine (66)
• Pathology (441)
• Pediatrics (1008)
• Pharmacology and Therapeutics (422)
• Primary Care Research (407)
• Psychiatry and Clinical Psychology (3077)
• Public and Global Health (6005)
• Radiology and Imaging (1227)
• Rehabilitation Medicine and Physical Therapy (715)
• Respiratory Medicine (811)
• Rheumatology (367)
• Sexual and Reproductive Health (356)
• Sports Medicine (318)
• Surgery (390)
• Toxicology (50)
• Transplantation (171)
• Urology (142)