case study as a scientific text

Organizing Your Social Sciences Research Assignments

• Annotated Bibliography
• Analyzing a Scholarly Journal Article
• Group Presentations
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• Types of Structured Group Activities
• Group Project Survival Skills
• Leading a Class Discussion
• Multiple Book Review Essay
• Reviewing Collected Works
• Writing a Case Analysis Paper
• Writing a Case Study
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Reflective Paper
• Writing a Research Proposal
• Generative AI and Writing
• Acknowledgments

A case study research paper examines a person, place, event, condition, phenomenon, or other type of subject of analysis in order to extrapolate  key themes and results that help predict future trends, illuminate previously hidden issues that can be applied to practice, and/or provide a means for understanding an important research problem with greater clarity. A case study research paper usually examines a single subject of analysis, but case study papers can also be designed as a comparative investigation that shows relationships between two or more subjects. The methods used to study a case can rest within a quantitative, qualitative, or mixed-method investigative paradigm.

Case Studies. Writing@CSU. Colorado State University; Mills, Albert J. , Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010 ; “What is a Case Study?” In Swanborn, Peter G. Case Study Research: What, Why and How? London: SAGE, 2010.

How to Approach Writing a Case Study Research Paper

General information about how to choose a topic to investigate can be found under the " Choosing a Research Problem " tab in the Organizing Your Social Sciences Research Paper writing guide. Review this page because it may help you identify a subject of analysis that can be investigated using a case study design.

However, identifying a case to investigate involves more than choosing the research problem . A case study encompasses a problem contextualized around the application of in-depth analysis, interpretation, and discussion, often resulting in specific recommendations for action or for improving existing conditions. As Seawright and Gerring note, practical considerations such as time and access to information can influence case selection, but these issues should not be the sole factors used in describing the methodological justification for identifying a particular case to study. Given this, selecting a case includes considering the following:

• The case represents an unusual or atypical example of a research problem that requires more in-depth analysis? Cases often represent a topic that rests on the fringes of prior investigations because the case may provide new ways of understanding the research problem. For example, if the research problem is to identify strategies to improve policies that support girl's access to secondary education in predominantly Muslim nations, you could consider using Azerbaijan as a case study rather than selecting a more obvious nation in the Middle East. Doing so may reveal important new insights into recommending how governments in other predominantly Muslim nations can formulate policies that support improved access to education for girls.
• The case provides important insight or illuminate a previously hidden problem? In-depth analysis of a case can be based on the hypothesis that the case study will reveal trends or issues that have not been exposed in prior research or will reveal new and important implications for practice. For example, anecdotal evidence may suggest drug use among homeless veterans is related to their patterns of travel throughout the day. Assuming prior studies have not looked at individual travel choices as a way to study access to illicit drug use, a case study that observes a homeless veteran could reveal how issues of personal mobility choices facilitate regular access to illicit drugs. Note that it is important to conduct a thorough literature review to ensure that your assumption about the need to reveal new insights or previously hidden problems is valid and evidence-based.
• The case challenges and offers a counter-point to prevailing assumptions? Over time, research on any given topic can fall into a trap of developing assumptions based on outdated studies that are still applied to new or changing conditions or the idea that something should simply be accepted as "common sense," even though the issue has not been thoroughly tested in current practice. A case study analysis may offer an opportunity to gather evidence that challenges prevailing assumptions about a research problem and provide a new set of recommendations applied to practice that have not been tested previously. For example, perhaps there has been a long practice among scholars to apply a particular theory in explaining the relationship between two subjects of analysis. Your case could challenge this assumption by applying an innovative theoretical framework [perhaps borrowed from another discipline] to explore whether this approach offers new ways of understanding the research problem. Taking a contrarian stance is one of the most important ways that new knowledge and understanding develops from existing literature.
• The case provides an opportunity to pursue action leading to the resolution of a problem? Another way to think about choosing a case to study is to consider how the results from investigating a particular case may result in findings that reveal ways in which to resolve an existing or emerging problem. For example, studying the case of an unforeseen incident, such as a fatal accident at a railroad crossing, can reveal hidden issues that could be applied to preventative measures that contribute to reducing the chance of accidents in the future. In this example, a case study investigating the accident could lead to a better understanding of where to strategically locate additional signals at other railroad crossings so as to better warn drivers of an approaching train, particularly when visibility is hindered by heavy rain, fog, or at night.
• The case offers a new direction in future research? A case study can be used as a tool for an exploratory investigation that highlights the need for further research about the problem. A case can be used when there are few studies that help predict an outcome or that establish a clear understanding about how best to proceed in addressing a problem. For example, after conducting a thorough literature review [very important!], you discover that little research exists showing the ways in which women contribute to promoting water conservation in rural communities of east central Africa. A case study of how women contribute to saving water in a rural village of Uganda can lay the foundation for understanding the need for more thorough research that documents how women in their roles as cooks and family caregivers think about water as a valuable resource within their community. This example of a case study could also point to the need for scholars to build new theoretical frameworks around the topic [e.g., applying feminist theories of work and family to the issue of water conservation].

Eisenhardt, Kathleen M. “Building Theories from Case Study Research.” Academy of Management Review 14 (October 1989): 532-550; Emmel, Nick. Sampling and Choosing Cases in Qualitative Research: A Realist Approach . Thousand Oaks, CA: SAGE Publications, 2013; Gerring, John. “What Is a Case Study and What Is It Good for?” American Political Science Review 98 (May 2004): 341-354; Mills, Albert J. , Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010; Seawright, Jason and John Gerring. "Case Selection Techniques in Case Study Research." Political Research Quarterly 61 (June 2008): 294-308.

Structure and Writing Style

The purpose of a paper in the social sciences designed around a case study is to thoroughly investigate a subject of analysis in order to reveal a new understanding about the research problem and, in so doing, contributing new knowledge to what is already known from previous studies. In applied social sciences disciplines [e.g., education, social work, public administration, etc.], case studies may also be used to reveal best practices, highlight key programs, or investigate interesting aspects of professional work.

In general, the structure of a case study research paper is not all that different from a standard college-level research paper. However, there are subtle differences you should be aware of. Here are the key elements to organizing and writing a case study research paper.

I.  Introduction

As with any research paper, your introduction should serve as a roadmap for your readers to ascertain the scope and purpose of your study . The introduction to a case study research paper, however, should not only describe the research problem and its significance, but you should also succinctly describe why the case is being used and how it relates to addressing the problem. The two elements should be linked. With this in mind, a good introduction answers these four questions:

• What is being studied? Describe the research problem and describe the subject of analysis [the case] you have chosen to address the problem. Explain how they are linked and what elements of the case will help to expand knowledge and understanding about the problem.
• Why is this topic important to investigate? Describe the significance of the research problem and state why a case study design and the subject of analysis that the paper is designed around is appropriate in addressing the problem.
• What did we know about this topic before I did this study? Provide background that helps lead the reader into the more in-depth literature review to follow. If applicable, summarize prior case study research applied to the research problem and why it fails to adequately address the problem. Describe why your case will be useful. If no prior case studies have been used to address the research problem, explain why you have selected this subject of analysis.
• How will this study advance new knowledge or new ways of understanding? Explain why your case study will be suitable in helping to expand knowledge and understanding about the research problem.

Each of these questions should be addressed in no more than a few paragraphs. Exceptions to this can be when you are addressing a complex research problem or subject of analysis that requires more in-depth background information.

II.  Literature Review

The literature review for a case study research paper is generally structured the same as it is for any college-level research paper. The difference, however, is that the literature review is focused on providing background information and  enabling historical interpretation of the subject of analysis in relation to the research problem the case is intended to address . This includes synthesizing studies that help to:

• Place relevant works in the context of their contribution to understanding the case study being investigated . This would involve summarizing studies that have used a similar subject of analysis to investigate the research problem. If there is literature using the same or a very similar case to study, you need to explain why duplicating past research is important [e.g., conditions have changed; prior studies were conducted long ago, etc.].
• Describe the relationship each work has to the others under consideration that informs the reader why this case is applicable . Your literature review should include a description of any works that support using the case to investigate the research problem and the underlying research questions.
• Identify new ways to interpret prior research using the case study . If applicable, review any research that has examined the research problem using a different research design. Explain how your use of a case study design may reveal new knowledge or a new perspective or that can redirect research in an important new direction.
• Resolve conflicts amongst seemingly contradictory previous studies . This refers to synthesizing any literature that points to unresolved issues of concern about the research problem and describing how the subject of analysis that forms the case study can help resolve these existing contradictions.
• Point the way in fulfilling a need for additional research . Your review should examine any literature that lays a foundation for understanding why your case study design and the subject of analysis around which you have designed your study may reveal a new way of approaching the research problem or offer a perspective that points to the need for additional research.
• Expose any gaps that exist in the literature that the case study could help to fill . Summarize any literature that not only shows how your subject of analysis contributes to understanding the research problem, but how your case contributes to a new way of understanding the problem that prior research has failed to do.
• Locate your own research within the context of existing literature [very important!] . Collectively, your literature review should always place your case study within the larger domain of prior research about the problem. The overarching purpose of reviewing pertinent literature in a case study paper is to demonstrate that you have thoroughly identified and synthesized prior studies in relation to explaining the relevance of the case in addressing the research problem.

III.  Method

In this section, you explain why you selected a particular case [i.e., subject of analysis] and the strategy you used to identify and ultimately decide that your case was appropriate in addressing the research problem. The way you describe the methods used varies depending on the type of subject of analysis that constitutes your case study.

If your subject of analysis is an incident or event . In the social and behavioral sciences, the event or incident that represents the case to be studied is usually bounded by time and place, with a clear beginning and end and with an identifiable location or position relative to its surroundings. The subject of analysis can be a rare or critical event or it can focus on a typical or regular event. The purpose of studying a rare event is to illuminate new ways of thinking about the broader research problem or to test a hypothesis. Critical incident case studies must describe the method by which you identified the event and explain the process by which you determined the validity of this case to inform broader perspectives about the research problem or to reveal new findings. However, the event does not have to be a rare or uniquely significant to support new thinking about the research problem or to challenge an existing hypothesis. For example, Walo, Bull, and Breen conducted a case study to identify and evaluate the direct and indirect economic benefits and costs of a local sports event in the City of Lismore, New South Wales, Australia. The purpose of their study was to provide new insights from measuring the impact of a typical local sports event that prior studies could not measure well because they focused on large "mega-events." Whether the event is rare or not, the methods section should include an explanation of the following characteristics of the event: a) when did it take place; b) what were the underlying circumstances leading to the event; and, c) what were the consequences of the event in relation to the research problem.

If your subject of analysis is a person. Explain why you selected this particular individual to be studied and describe what experiences they have had that provide an opportunity to advance new understandings about the research problem. Mention any background about this person which might help the reader understand the significance of their experiences that make them worthy of study. This includes describing the relationships this person has had with other people, institutions, and/or events that support using them as the subject for a case study research paper. It is particularly important to differentiate the person as the subject of analysis from others and to succinctly explain how the person relates to examining the research problem [e.g., why is one politician in a particular local election used to show an increase in voter turnout from any other candidate running in the election]. Note that these issues apply to a specific group of people used as a case study unit of analysis [e.g., a classroom of students].

If your subject of analysis is a place. In general, a case study that investigates a place suggests a subject of analysis that is unique or special in some way and that this uniqueness can be used to build new understanding or knowledge about the research problem. A case study of a place must not only describe its various attributes relevant to the research problem [e.g., physical, social, historical, cultural, economic, political], but you must state the method by which you determined that this place will illuminate new understandings about the research problem. It is also important to articulate why a particular place as the case for study is being used if similar places also exist [i.e., if you are studying patterns of homeless encampments of veterans in open spaces, explain why you are studying Echo Park in Los Angeles rather than Griffith Park?]. If applicable, describe what type of human activity involving this place makes it a good choice to study [e.g., prior research suggests Echo Park has more homeless veterans].

If your subject of analysis is a phenomenon. A phenomenon refers to a fact, occurrence, or circumstance that can be studied or observed but with the cause or explanation to be in question. In this sense, a phenomenon that forms your subject of analysis can encompass anything that can be observed or presumed to exist but is not fully understood. In the social and behavioral sciences, the case usually focuses on human interaction within a complex physical, social, economic, cultural, or political system. For example, the phenomenon could be the observation that many vehicles used by ISIS fighters are small trucks with English language advertisements on them. The research problem could be that ISIS fighters are difficult to combat because they are highly mobile. The research questions could be how and by what means are these vehicles used by ISIS being supplied to the militants and how might supply lines to these vehicles be cut off? How might knowing the suppliers of these trucks reveal larger networks of collaborators and financial support? A case study of a phenomenon most often encompasses an in-depth analysis of a cause and effect that is grounded in an interactive relationship between people and their environment in some way.

NOTE:   The choice of the case or set of cases to study cannot appear random. Evidence that supports the method by which you identified and chose your subject of analysis should clearly support investigation of the research problem and linked to key findings from your literature review. Be sure to cite any studies that helped you determine that the case you chose was appropriate for examining the problem.

IV.  Discussion

The main elements of your discussion section are generally the same as any research paper, but centered around interpreting and drawing conclusions about the key findings from your analysis of the case study. Note that a general social sciences research paper may contain a separate section to report findings. However, in a paper designed around a case study, it is common to combine a description of the results with the discussion about their implications. The objectives of your discussion section should include the following:

Reiterate the Research Problem/State the Major Findings Briefly reiterate the research problem you are investigating and explain why the subject of analysis around which you designed the case study were used. You should then describe the findings revealed from your study of the case using direct, declarative, and succinct proclamation of the study results. Highlight any findings that were unexpected or especially profound.

Explain the Meaning of the Findings and Why They are Important Systematically explain the meaning of your case study findings and why you believe they are important. Begin this part of the section by repeating what you consider to be your most important or surprising finding first, then systematically review each finding. Be sure to thoroughly extrapolate what your analysis of the case can tell the reader about situations or conditions beyond the actual case that was studied while, at the same time, being careful not to misconstrue or conflate a finding that undermines the external validity of your conclusions.

Relate the Findings to Similar Studies No study in the social sciences is so novel or possesses such a restricted focus that it has absolutely no relation to previously published research. The discussion section should relate your case study results to those found in other studies, particularly if questions raised from prior studies served as the motivation for choosing your subject of analysis. This is important because comparing and contrasting the findings of other studies helps support the overall importance of your results and it highlights how and in what ways your case study design and the subject of analysis differs from prior research about the topic.

Consider Alternative Explanations of the Findings Remember that the purpose of social science research is to discover and not to prove. When writing the discussion section, you should carefully consider all possible explanations revealed by the case study results, rather than just those that fit your hypothesis or prior assumptions and biases. Be alert to what the in-depth analysis of the case may reveal about the research problem, including offering a contrarian perspective to what scholars have stated in prior research if that is how the findings can be interpreted from your case.

Acknowledge the Study's Limitations You can state the study's limitations in the conclusion section of your paper but describing the limitations of your subject of analysis in the discussion section provides an opportunity to identify the limitations and explain why they are not significant. This part of the discussion section should also note any unanswered questions or issues your case study could not address. More detailed information about how to document any limitations to your research can be found here .

Suggest Areas for Further Research Although your case study may offer important insights about the research problem, there are likely additional questions related to the problem that remain unanswered or findings that unexpectedly revealed themselves as a result of your in-depth analysis of the case. Be sure that the recommendations for further research are linked to the research problem and that you explain why your recommendations are valid in other contexts and based on the original assumptions of your study.

V.  Conclusion

As with any research paper, you should summarize your conclusion in clear, simple language; emphasize how the findings from your case study differs from or supports prior research and why. Do not simply reiterate the discussion section. Provide a synthesis of key findings presented in the paper to show how these converge to address the research problem. If you haven't already done so in the discussion section, be sure to document the limitations of your case study and any need for further research.

The function of your paper's conclusion is to: 1) reiterate the main argument supported by the findings from your case study; 2) state clearly the context, background, and necessity of pursuing the research problem using a case study design in relation to an issue, controversy, or a gap found from reviewing the literature; and, 3) provide a place to persuasively and succinctly restate the significance of your research problem, given that the reader has now been presented with in-depth information about the topic.

Consider the following points to help ensure your conclusion is appropriate:

• If the argument or purpose of your paper is complex, you may need to summarize these points for your reader.
• If prior to your conclusion, you have not yet explained the significance of your findings or if you are proceeding inductively, use the conclusion of your paper to describe your main points and explain their significance.
• Move from a detailed to a general level of consideration of the case study's findings that returns the topic to the context provided by the introduction or within a new context that emerges from your case study findings.

Note that, depending on the discipline you are writing in or the preferences of your professor, the concluding paragraph may contain your final reflections on the evidence presented as it applies to practice or on the essay's central research problem. However, the nature of being introspective about the subject of analysis you have investigated will depend on whether you are explicitly asked to express your observations in this way.

Problems to Avoid

Overgeneralization One of the goals of a case study is to lay a foundation for understanding broader trends and issues applied to similar circumstances. However, be careful when drawing conclusions from your case study. They must be evidence-based and grounded in the results of the study; otherwise, it is merely speculation. Looking at a prior example, it would be incorrect to state that a factor in improving girls access to education in Azerbaijan and the policy implications this may have for improving access in other Muslim nations is due to girls access to social media if there is no documentary evidence from your case study to indicate this. There may be anecdotal evidence that retention rates were better for girls who were engaged with social media, but this observation would only point to the need for further research and would not be a definitive finding if this was not a part of your original research agenda.

Failure to Document Limitations No case is going to reveal all that needs to be understood about a research problem. Therefore, just as you have to clearly state the limitations of a general research study , you must describe the specific limitations inherent in the subject of analysis. For example, the case of studying how women conceptualize the need for water conservation in a village in Uganda could have limited application in other cultural contexts or in areas where fresh water from rivers or lakes is plentiful and, therefore, conservation is understood more in terms of managing access rather than preserving access to a scarce resource.

Failure to Extrapolate All Possible Implications Just as you don't want to over-generalize from your case study findings, you also have to be thorough in the consideration of all possible outcomes or recommendations derived from your findings. If you do not, your reader may question the validity of your analysis, particularly if you failed to document an obvious outcome from your case study research. For example, in the case of studying the accident at the railroad crossing to evaluate where and what types of warning signals should be located, you failed to take into consideration speed limit signage as well as warning signals. When designing your case study, be sure you have thoroughly addressed all aspects of the problem and do not leave gaps in your analysis that leave the reader questioning the results.

Case Studies. Writing@CSU. Colorado State University; Gerring, John. Case Study Research: Principles and Practices . New York: Cambridge University Press, 2007; Merriam, Sharan B. Qualitative Research and Case Study Applications in Education . Rev. ed. San Francisco, CA: Jossey-Bass, 1998; Miller, Lisa L. “The Use of Case Studies in Law and Social Science Research.” Annual Review of Law and Social Science 14 (2018): TBD; Mills, Albert J., Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010; Putney, LeAnn Grogan. "Case Study." In Encyclopedia of Research Design , Neil J. Salkind, editor. (Thousand Oaks, CA: SAGE Publications, 2010), pp. 116-120; Simons, Helen. Case Study Research in Practice . London: SAGE Publications, 2009;  Kratochwill,  Thomas R. and Joel R. Levin, editors. Single-Case Research Design and Analysis: New Development for Psychology and Education .  Hilldsale, NJ: Lawrence Erlbaum Associates, 1992; Swanborn, Peter G. Case Study Research: What, Why and How? London : SAGE, 2010; Yin, Robert K. Case Study Research: Design and Methods . 6th edition. Los Angeles, CA, SAGE Publications, 2014; Walo, Maree, Adrian Bull, and Helen Breen. “Achieving Economic Benefits at Local Events: A Case Study of a Local Sports Event.” Festival Management and Event Tourism 4 (1996): 95-106.

Writing Tip

At Least Five Misconceptions about Case Study Research

Social science case studies are often perceived as limited in their ability to create new knowledge because they are not randomly selected and findings cannot be generalized to larger populations. Flyvbjerg examines five misunderstandings about case study research and systematically "corrects" each one. To quote, these are:

Misunderstanding 1 :  General, theoretical [context-independent] knowledge is more valuable than concrete, practical [context-dependent] knowledge. Misunderstanding 2 :  One cannot generalize on the basis of an individual case; therefore, the case study cannot contribute to scientific development. Misunderstanding 3 :  The case study is most useful for generating hypotheses; that is, in the first stage of a total research process, whereas other methods are more suitable for hypotheses testing and theory building. Misunderstanding 4 :  The case study contains a bias toward verification, that is, a tendency to confirm the researcher’s preconceived notions. Misunderstanding 5 :  It is often difficult to summarize and develop general propositions and theories on the basis of specific case studies [p. 221].

While writing your paper, think introspectively about how you addressed these misconceptions because to do so can help you strengthen the validity and reliability of your research by clarifying issues of case selection, the testing and challenging of existing assumptions, the interpretation of key findings, and the summation of case outcomes. Think of a case study research paper as a complete, in-depth narrative about the specific properties and key characteristics of your subject of analysis applied to the research problem.

Flyvbjerg, Bent. “Five Misunderstandings About Case-Study Research.” Qualitative Inquiry 12 (April 2006): 219-245.

• << Previous: Writing a Case Analysis Paper
• Next: Writing a Field Report >>
• Last Updated: May 7, 2024 9:45 AM
• URL: https://libguides.usc.edu/writingguide/assignments

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Med Libr Assoc
• v.107(1); 2019 Jan

Distinguishing case study as a research method from case reports as a publication type

The purpose of this editorial is to distinguish between case reports and case studies. In health, case reports are familiar ways of sharing events or efforts of intervening with single patients with previously unreported features. As a qualitative methodology, case study research encompasses a great deal more complexity than a typical case report and often incorporates multiple streams of data combined in creative ways. The depth and richness of case study description helps readers understand the case and whether findings might be applicable beyond that setting.

Single-institution descriptive reports of library activities are often labeled by their authors as “case studies.” By contrast, in health care, single patient retrospective descriptions are published as “case reports.” Both case reports and case studies are valuable to readers and provide a publication opportunity for authors. A previous editorial by Akers and Amos about improving case studies addresses issues that are more common to case reports; for example, not having a review of the literature or being anecdotal, not generalizable, and prone to various types of bias such as positive outcome bias [ 1 ]. However, case study research as a qualitative methodology is pursued for different purposes than generalizability. The authors’ purpose in this editorial is to clearly distinguish between case reports and case studies. We believe that this will assist authors in describing and designating the methodological approach of their publications and help readers appreciate the rigor of well-executed case study research.

Case reports often provide a first exploration of a phenomenon or an opportunity for a first publication by a trainee in the health professions. In health care, case reports are familiar ways of sharing events or efforts of intervening with single patients with previously unreported features. Another type of study categorized as a case report is an “N of 1” study or single-subject clinical trial, which considers an individual patient as the sole unit of observation in a study investigating the efficacy or side effect profiles of different interventions. Entire journals have evolved to publish case reports, which often rely on template structures with limited contextualization or discussion of previous cases. Examples that are indexed in MEDLINE include the American Journal of Case Reports , BMJ Case Reports, Journal of Medical Case Reports, and Journal of Radiology Case Reports . Similar publications appear in veterinary medicine and are indexed in CAB Abstracts, such as Case Reports in Veterinary Medicine and Veterinary Record Case Reports .

As a qualitative methodology, however, case study research encompasses a great deal more complexity than a typical case report and often incorporates multiple streams of data combined in creative ways. Distinctions include the investigator’s definitions and delimitations of the case being studied, the clarity of the role of the investigator, the rigor of gathering and combining evidence about the case, and the contextualization of the findings. Delimitation is a term from qualitative research about setting boundaries to scope the research in a useful way rather than describing the narrow scope as a limitation, as often appears in a discussion section. The depth and richness of description helps readers understand the situation and whether findings from the case are applicable to their settings.

CASE STUDY AS A RESEARCH METHODOLOGY

Case study as a qualitative methodology is an exploration of a time- and space-bound phenomenon. As qualitative research, case studies require much more from their authors who are acting as instruments within the inquiry process. In the case study methodology, a variety of methodological approaches may be employed to explain the complexity of the problem being studied [ 2 , 3 ].

Leading authors diverge in their definitions of case study, but a qualitative research text introduces case study as follows:

Case study research is defined as a qualitative approach in which the investigator explores a real-life, contemporary bounded system (a case) or multiple bound systems (cases) over time, through detailed, in-depth data collection involving multiple sources of information, and reports a case description and case themes. The unit of analysis in the case study might be multiple cases (a multisite study) or a single case (a within-site case study). [ 4 ]

Methodologists writing core texts on case study research include Yin [ 5 ], Stake [ 6 ], and Merriam [ 7 ]. The approaches of these three methodologists have been compared by Yazan, who focused on six areas of methodology: epistemology (beliefs about ways of knowing), definition of cases, design of case studies, and gathering, analysis, and validation of data [ 8 ]. For Yin, case study is a method of empirical inquiry appropriate to determining the “how and why” of phenomena and contributes to understanding phenomena in a holistic and real-life context [ 5 ]. Stake defines a case study as a “well-bounded, specific, complex, and functioning thing” [ 6 ], while Merriam views “the case as a thing, a single entity, a unit around which there are boundaries” [ 7 ].

Case studies are ways to explain, describe, or explore phenomena. Comments from a quantitative perspective about case studies lacking rigor and generalizability fail to consider the purpose of the case study and how what is learned from a case study is put into practice. Rigor in case studies comes from the research design and its components, which Yin outlines as (a) the study’s questions, (b) the study’s propositions, (c) the unit of analysis, (d) the logic linking the data to propositions, and (e) the criteria for interpreting the findings [ 5 ]. Case studies should also provide multiple sources of data, a case study database, and a clear chain of evidence among the questions asked, the data collected, and the conclusions drawn [ 5 ].

Sources of evidence for case studies include interviews, documentation, archival records, direct observations, participant-observation, and physical artifacts. One of the most important sources for data in qualitative case study research is the interview [ 2 , 3 ]. In addition to interviews, documents and archival records can be gathered to corroborate and enhance the findings of the study. To understand the phenomenon or the conditions that created it, direct observations can serve as another source of evidence and can be conducted throughout the study. These can include the use of formal and informal protocols as a participant inside the case or an external or passive observer outside of the case [ 5 ]. Lastly, physical artifacts can be observed and collected as a form of evidence. With these multiple potential sources of evidence, the study methodology includes gathering data, sense-making, and triangulating multiple streams of data. Figure 1 shows an example in which data used for the case started with a pilot study to provide additional context to guide more in-depth data collection and analysis with participants.

Key sources of data for a sample case study

VARIATIONS ON CASE STUDY METHODOLOGY

Case study methodology is evolving and regularly reinterpreted. Comparative or multiple case studies are used as a tool for synthesizing information across time and space to research the impact of policy and practice in various fields of social research [ 9 ]. Because case study research is in-depth and intensive, there have been efforts to simplify the method or select useful components of cases for focused analysis. Micro-case study is a term that is occasionally used to describe research on micro-level cases [ 10 ]. These are cases that occur in a brief time frame, occur in a confined setting, and are simple and straightforward in nature. A micro-level case describes a clear problem of interest. Reporting is very brief and about specific points. The lack of complexity in the case description makes obvious the “lesson” that is inherent in the case; although no definitive “solution” is necessarily forthcoming, making the case useful for discussion. A micro-case write-up can be distinguished from a case report by its focus on briefly reporting specific features of a case or cases to analyze or learn from those features.

DATABASE INDEXING OF CASE REPORTS AND CASE STUDIES

Disciplines such as education, psychology, sociology, political science, and social work regularly publish rich case studies that are relevant to particular areas of health librarianship. Case reports and case studies have been defined as publication types or subject terms by several databases that are relevant to librarian authors: MEDLINE, PsycINFO, CINAHL, and ERIC. Library, Information Science & Technology Abstracts (LISTA) does not have a subject term or publication type related to cases, despite many being included in the database. Whereas “Case Reports” are the main term used by MEDLINE’s Medical Subject Headings (MeSH) and PsycINFO’s thesaurus, CINAHL and ERIC use “Case Studies.”

Case reports in MEDLINE and PsycINFO focus on clinical case documentation. In MeSH, “Case Reports” as a publication type is specific to “clinical presentations that may be followed by evaluative studies that eventually lead to a diagnosis” [ 11 ]. “Case Histories,” “Case Studies,” and “Case Study” are all entry terms mapping to “Case Reports”; however, guidance to indexers suggests that “Case Reports” should not be applied to institutional case reports and refers to the heading “Organizational Case Studies,” which is defined as “descriptions and evaluations of specific health care organizations” [ 12 ].

PsycINFO’s subject term “Case Report” is “used in records discussing issues involved in the process of conducting exploratory studies of single or multiple clinical cases.” The Methodology index offers clinical and non-clinical entries. “Clinical Case Study” is defined as “case reports that include disorder, diagnosis, and clinical treatment for individuals with mental or medical illnesses,” whereas “Non-clinical Case Study” is a “document consisting of non-clinical or organizational case examples of the concepts being researched or studied. The setting is always non-clinical and does not include treatment-related environments” [ 13 ].

Both CINAHL and ERIC acknowledge the depth of analysis in case study methodology. The CINAHL scope note for the thesaurus term “Case Studies” distinguishes between the document and the methodology, though both use the same term: “a review of a particular condition, disease, or administrative problem. Also, a research method that involves an in-depth analysis of an individual, group, institution, or other social unit. For material that contains a case study, search for document type: case study.” The ERIC scope note for the thesaurus term “Case Studies” is simple: “detailed analyses, usually focusing on a particular problem of an individual, group, or organization” [ 14 ].

PUBLICATION OF CASE STUDY RESEARCH IN LIBRARIANSHIP

We call your attention to a few examples published as case studies in health sciences librarianship to consider how their characteristics fit with the preceding definitions of case reports or case study research. All present some characteristics of case study research, but their treatment of the research questions, richness of description, and analytic strategies vary in depth and, therefore, diverge at some level from the qualitative case study research approach. This divergence, particularly in richness of description and analysis, may have been constrained by the publication requirements.

As one example, a case study by Janke and Rush documented a time- and context-bound collaboration involving a librarian and a nursing faculty member [ 15 ]. Three objectives were stated: (1) describing their experience of working together on an interprofessional research team, (2) evaluating the value of the librarian role from librarian and faculty member perspectives, and (3) relating findings to existing literature. Elements that signal the qualitative nature of this case study are that the authors were the research participants and their use of the term “evaluation” is reflection on their experience. This reads like a case study that could have been enriched by including other types of data gathered from others engaging with this team to broaden the understanding of the collaboration.

As another example, the description of the academic context is one of the most salient components of the case study written by Clairoux et al., which had the objectives of (1) describing the library instruction offered and learning assessments used at a single health sciences library and (2) discussing the positive outcomes of instruction in that setting [ 16 ]. The authors focus on sharing what the institution has done more than explaining why this institution is an exemplar to explore a focused question or understand the phenomenon of library instruction. However, like a case study, the analysis brings together several streams of data including course attendance, online material page views, and some discussion of results from surveys. This paper reads somewhat in between an institutional case report and a case study.

The final example is a single author reporting on a personal experience of creating and executing the role of research informationist for a National Institutes of Health (NIH)–funded research team [ 17 ]. There is a thoughtful review of the informationist literature and detailed descriptions of the institutional context and the process of gaining access to and participating in the new role. However, the motivating question in the abstract does not seem to be fully addressed through analysis from either the reflective perspective of the author as the research participant or consideration of other streams of data from those involved in the informationist experience. The publication reads more like a case report about this informationist’s experience than a case study that explores the research informationist experience through the selection of this case.

All of these publications are well written and useful for their intended audiences, but in general, they are much shorter and much less rich in depth than case studies published in social sciences research. It may be that the authors have been constrained by word counts or page limits. For example, the submission category for Case Studies in the Journal of the Medical Library Association (JMLA) limited them to 3,000 words and defined them as “articles describing the process of developing, implementing, and evaluating a new service, program, or initiative, typically in a single institution or through a single collaborative effort” [ 18 ]. This definition’s focus on novelty and description sounds much more like the definition of case report than the in-depth, detailed investigation of a time- and space-bound problem that is often examined through case study research.

Problem-focused or question-driven case study research would benefit from the space provided for Original Investigations that employ any type of quantitative or qualitative method of analysis. One of the best examples in the JMLA of an in-depth multiple case study that was authored by a librarian who published the findings from her doctoral dissertation represented all the elements of a case study. In eight pages, she provided a theoretical basis for the research question, a pilot study, and a multiple case design, including integrated data from interviews and focus groups [ 19 ].

We have distinguished between case reports and case studies primarily to assist librarians who are new to research and critical appraisal of case study methodology to recognize the features that authors use to describe and designate the methodological approaches of their publications. For researchers who are new to case research methodology and are interested in learning more, Hancock and Algozzine provide a guide [ 20 ].

We hope that JMLA readers appreciate the rigor of well-executed case study research. We believe that distinguishing between descriptive case reports and analytic case studies in the journal’s submission categories will allow the depth of case study methodology to increase. We also hope that authors feel encouraged to pursue submitting relevant case studies or case reports for future publication.

Editor’s note: In response to this invited editorial, the Journal of the Medical Library Association will consider manuscripts employing rigorous qualitative case study methodology to be Original Investigations (fewer than 5,000 words), whereas manuscripts describing the process of developing, implementing, and assessing a new service, program, or initiative—typically in a single institution or through a single collaborative effort—will be considered to be Case Reports (formerly known as Case Studies; fewer than 3,000 words).

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Case Study | Definition, Examples & Methods

Case Study | Definition, Examples & Methods

Published on 5 May 2022 by Shona McCombes . Revised on 30 January 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organisation, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating, and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyse the case.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Prevent plagiarism, run a free check.

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

Unlike quantitative or experimental research, a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

If you find yourself aiming to simultaneously investigate and solve an issue, consider conducting action research . As its name suggests, action research conducts research and takes action at the same time, and is highly iterative and flexible. 

However, you can also choose a more common or representative case to exemplify a particular category, experience, or phenomenon.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews, observations, and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data .

The aim is to gain as thorough an understanding as possible of the case and its context.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis, with separate sections or chapters for the methods , results , and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyse its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2023, January 30). Case Study | Definition, Examples & Methods. Scribbr. Retrieved 26 May 2024, from https://www.scribbr.co.uk/research-methods/case-studies/

Is this article helpful?

Shona McCombes

Other students also liked, correlational research | guide, design & examples, a quick guide to experimental design | 5 steps & examples, descriptive research design | definition, methods & examples.

Designing and Conducting Case Studies

This guide examines case studies, a form of qualitative descriptive research that is used to look at individuals, a small group of participants, or a group as a whole. Researchers collect data about participants using participant and direct observations, interviews, protocols, tests, examinations of records, and collections of writing samples. Starting with a definition of the case study, the guide moves to a brief history of this research method. Using several well documented case studies, the guide then looks at applications and methods including data collection and analysis. A discussion of ways to handle validity, reliability, and generalizability follows, with special attention to case studies as they are applied to composition studies. Finally, this guide examines the strengths and weaknesses of case studies.

Definition and Overview

Case study refers to the collection and presentation of detailed information about a particular participant or small group, frequently including the accounts of subjects themselves. A form of qualitative descriptive research, the case study looks intensely at an individual or small participant pool, drawing conclusions only about that participant or group and only in that specific context. Researchers do not focus on the discovery of a universal, generalizable truth, nor do they typically look for cause-effect relationships; instead, emphasis is placed on exploration and description.

Case studies typically examine the interplay of all variables in order to provide as complete an understanding of an event or situation as possible. This type of comprehensive understanding is arrived at through a process known as thick description, which involves an in-depth description of the entity being evaluated, the circumstances under which it is used, the characteristics of the people involved in it, and the nature of the community in which it is located. Thick description also involves interpreting the meaning of demographic and descriptive data such as cultural norms and mores, community values, ingrained attitudes, and motives.

Unlike quantitative methods of research, like the survey, which focus on the questions of who, what, where, how much, and how many, and archival analysis, which often situates the participant in some form of historical context, case studies are the preferred strategy when how or why questions are asked. Likewise, they are the preferred method when the researcher has little control over the events, and when there is a contemporary focus within a real life context. In addition, unlike more specifically directed experiments, case studies require a problem that seeks a holistic understanding of the event or situation in question using inductive logic--reasoning from specific to more general terms.

In scholarly circles, case studies are frequently discussed within the context of qualitative research and naturalistic inquiry. Case studies are often referred to interchangeably with ethnography, field study, and participant observation. The underlying philosophical assumptions in the case are similar to these types of qualitative research because each takes place in a natural setting (such as a classroom, neighborhood, or private home), and strives for a more holistic interpretation of the event or situation under study.

Unlike more statistically-based studies which search for quantifiable data, the goal of a case study is to offer new variables and questions for further research. F.H. Giddings, a sociologist in the early part of the century, compares statistical methods to the case study on the basis that the former are concerned with the distribution of a particular trait, or a small number of traits, in a population, whereas the case study is concerned with the whole variety of traits to be found in a particular instance" (Hammersley 95).

Case studies are not a new form of research; naturalistic inquiry was the primary research tool until the development of the scientific method. The fields of sociology and anthropology are credited with the primary shaping of the concept as we know it today. However, case study research has drawn from a number of other areas as well: the clinical methods of doctors; the casework technique being developed by social workers; the methods of historians and anthropologists, plus the qualitative descriptions provided by quantitative researchers like LePlay; and, in the case of Robert Park, the techniques of newspaper reporters and novelists.

Park was an ex-newspaper reporter and editor who became very influential in developing sociological case studies at the University of Chicago in the 1920s. As a newspaper professional he coined the term "scientific" or "depth" reporting: the description of local events in a way that pointed to major social trends. Park viewed the sociologist as "merely a more accurate, responsible, and scientific reporter." Park stressed the variety and value of human experience. He believed that sociology sought to arrive at natural, but fluid, laws and generalizations in regard to human nature and society. These laws weren't static laws of the kind sought by many positivists and natural law theorists, but rather, they were laws of becoming--with a constant possibility of change. Park encouraged students to get out of the library, to quit looking at papers and books, and to view the constant experiment of human experience. He writes, "Go and sit in the lounges of the luxury hotels and on the doorsteps of the flophouses; sit on the Gold Coast settees and on the slum shakedowns; sit in the Orchestra Hall and in the Star and Garter Burlesque. In short, gentlemen [sic], go get the seats of your pants dirty in real research."

But over the years, case studies have drawn their share of criticism. In fact, the method had its detractors from the start. In the 1920s, the debate between pro-qualitative and pro-quantitative became quite heated. Case studies, when compared to statistics, were considered by many to be unscientific. From the 1930's on, the rise of positivism had a growing influence on quantitative methods in sociology. People wanted static, generalizable laws in science. The sociological positivists were looking for stable laws of social phenomena. They criticized case study research because it failed to provide evidence of inter subjective agreement. Also, they condemned it because of the few number of cases studied and that the under-standardized character of their descriptions made generalization impossible. By the 1950s, quantitative methods, in the form of survey research, had become the dominant sociological approach and case study had become a minority practice.

Educational Applications

The 1950's marked the dawning of a new era in case study research, namely that of the utilization of the case study as a teaching method. "Instituted at Harvard Business School in the 1950s as a primary method of teaching, cases have since been used in classrooms and lecture halls alike, either as part of a course of study or as the main focus of the course to which other teaching material is added" (Armisted 1984). The basic purpose of instituting the case method as a teaching strategy was "to transfer much of the responsibility for learning from the teacher on to the student, whose role, as a result, shifts away from passive absorption toward active construction" (Boehrer 1990). Through careful examination and discussion of various cases, "students learn to identify actual problems, to recognize key players and their agendas, and to become aware of those aspects of the situation that contribute to the problem" (Merseth 1991). In addition, students are encouraged to "generate their own analysis of the problems under consideration, to develop their own solutions, and to practically apply their own knowledge of theory to these problems" (Boyce 1993). Along the way, students also develop "the power to analyze and to master a tangled circumstance by identifying and delineating important factors; the ability to utilize ideas, to test them against facts, and to throw them into fresh combinations" (Merseth 1991).

In addition to the practical application and testing of scholarly knowledge, case discussions can also help students prepare for real-world problems, situations and crises by providing an approximation of various professional environments (i.e. classroom, board room, courtroom, or hospital). Thus, through the examination of specific cases, students are given the opportunity to work out their own professional issues through the trials, tribulations, experiences, and research findings of others. An obvious advantage to this mode of instruction is that it allows students the exposure to settings and contexts that they might not otherwise experience. For example, a student interested in studying the effects of poverty on minority secondary student's grade point averages and S.A.T. scores could access and analyze information from schools as geographically diverse as Los Angeles, New York City, Miami, and New Mexico without ever having to leave the classroom.

The case study method also incorporates the idea that students can learn from one another "by engaging with each other and with each other's ideas, by asserting something and then having it questioned, challenged and thrown back at them so that they can reflect on what they hear, and then refine what they say" (Boehrer 1990). In summary, students can direct their own learning by formulating questions and taking responsibility for the study.

Types and Design Concerns

Researchers use multiple methods and approaches to conduct case studies.

Types of Case Studies

Under the more generalized category of case study exist several subdivisions, each of which is custom selected for use depending upon the goals and/or objectives of the investigator. These types of case study include the following:

Illustrative Case Studies These are primarily descriptive studies. They typically utilize one or two instances of an event to show what a situation is like. Illustrative case studies serve primarily to make the unfamiliar familiar and to give readers a common language about the topic in question.

Exploratory (or pilot) Case Studies These are condensed case studies performed before implementing a large scale investigation. Their basic function is to help identify questions and select types of measurement prior to the main investigation. The primary pitfall of this type of study is that initial findings may seem convincing enough to be released prematurely as conclusions.

Cumulative Case Studies These serve to aggregate information from several sites collected at different times. The idea behind these studies is the collection of past studies will allow for greater generalization without additional cost or time being expended on new, possibly repetitive studies.

Critical Instance Case Studies These examine one or more sites for either the purpose of examining a situation of unique interest with little to no interest in generalizability, or to call into question or challenge a highly generalized or universal assertion. This method is useful for answering cause and effect questions.

Identifying a Theoretical Perspective

Much of the case study's design is inherently determined for researchers, depending on the field from which they are working. In composition studies, researchers are typically working from a qualitative, descriptive standpoint. In contrast, physicists will approach their research from a more quantitative perspective. Still, in designing the study, researchers need to make explicit the questions to be explored and the theoretical perspective from which they will approach the case. The three most commonly adopted theories are listed below:

Individual Theories These focus primarily on the individual development, cognitive behavior, personality, learning and disability, and interpersonal interactions of a particular subject.

Organizational Theories These focus on bureaucracies, institutions, organizational structure and functions, or excellence in organizational performance.

Social Theories These focus on urban development, group behavior, cultural institutions, or marketplace functions.

Two examples of case studies are used consistently throughout this chapter. The first, a study produced by Berkenkotter, Huckin, and Ackerman (1988), looks at a first year graduate student's initiation into an academic writing program. The study uses participant-observer and linguistic data collecting techniques to assess the student's knowledge of appropriate discourse conventions. Using the pseudonym Nate to refer to the subject, the study sought to illuminate the particular experience rather than to generalize about the experience of fledgling academic writers collectively.

For example, in Berkenkotter, Huckin, and Ackerman's (1988) study we are told that the researchers are interested in disciplinary communities. In the first paragraph, they ask what constitutes membership in a disciplinary community and how achieving membership might affect a writer's understanding and production of texts. In the third paragraph they state that researchers must negotiate their claims "within the context of his sub specialty's accepted knowledge and methodology." In the next paragraph they ask, "How is literacy acquired? What is the process through which novices gain community membership? And what factors either aid or hinder students learning the requisite linguistic behaviors?" This introductory section ends with a paragraph in which the study's authors claim that during the course of the study, the subject, Nate, successfully makes the transition from "skilled novice" to become an initiated member of the academic discourse community and that his texts exhibit linguistic changes which indicate this transition. In the next section the authors make explicit the sociolinguistic theoretical and methodological assumptions on which the study is based (1988). Thus the reader has a good understanding of the authors' theoretical background and purpose in conducting the study even before it is explicitly stated on the fourth page of the study. "Our purpose was to examine the effects of the educational context on one graduate student's production of texts as he wrote in different courses and for different faculty members over the academic year 1984-85." The goal of the study then, was to explore the idea that writers must be initiated into a writing community, and that this initiation will change the way one writes.

The second example is Janet Emig's (1971) study of the composing process of a group of twelfth graders. In this study, Emig seeks to answer the question of what happens to the self as a result educational stimuli in terms of academic writing. The case study used methods such as protocol analysis, tape-recorded interviews, and discourse analysis.

In the case of Janet Emig's (1971) study of the composing process of eight twelfth graders, four specific hypotheses were made:

• Twelfth grade writers engage in two modes of composing: reflexive and extensive.
• These differences can be ascertained and characterized through having the writers compose aloud their composition process.
• A set of implied stylistic principles governs the writing process.
• For twelfth grade writers, extensive writing occurs chiefly as a school-sponsored activity, or reflexive, as a self-sponsored activity.

In this study, the chief distinction is between the two dominant modes of composing among older, secondary school students. The distinctions are:

• The reflexive mode, which focuses on the writer's thoughts and feelings.
• The extensive mode, which focuses on conveying a message.

Emig also outlines the specific questions which guided the research in the opening pages of her Review of Literature , preceding the report.

Designing a Case Study

After considering the different sub categories of case study and identifying a theoretical perspective, researchers can begin to design their study. Research design is the string of logic that ultimately links the data to be collected and the conclusions to be drawn to the initial questions of the study. Typically, research designs deal with at least four problems:

• What questions to study
• What data are relevant
• What data to collect
• How to analyze that data

In other words, a research design is basically a blueprint for getting from the beginning to the end of a study. The beginning is an initial set of questions to be answered, and the end is some set of conclusions about those questions.

Because case studies are conducted on topics as diverse as Anglo-Saxon Literature (Thrane 1986) and AIDS prevention (Van Vugt 1994), it is virtually impossible to outline any strict or universal method or design for conducting the case study. However, Robert K. Yin (1993) does offer five basic components of a research design:

• A study's questions.
• A study's propositions (if any).
• A study's units of analysis.
• The logic that links the data to the propositions.
• The criteria for interpreting the findings.

In addition to these five basic components, Yin also stresses the importance of clearly articulating one's theoretical perspective, determining the goals of the study, selecting one's subject(s), selecting the appropriate method(s) of collecting data, and providing some considerations to the composition of the final report.

Conducting Case Studies

To obtain as complete a picture of the participant as possible, case study researchers can employ a variety of approaches and methods. These approaches, methods, and related issues are discussed in depth in this section.

Method: Single or Multi-modal?

To obtain as complete a picture of the participant as possible, case study researchers can employ a variety of methods. Some common methods include interviews , protocol analyses, field studies, and participant-observations. Emig (1971) chose to use several methods of data collection. Her sources included conversations with the students, protocol analysis, discrete observations of actual composition, writing samples from each student, and school records (Lauer and Asher 1988).

Berkenkotter, Huckin, and Ackerman (1988) collected data by observing classrooms, conducting faculty and student interviews, collecting self reports from the subject, and by looking at the subject's written work.

A study that was criticized for using a single method model was done by Flower and Hayes (1984). In this study that explores the ways in which writers use different forms of knowing to create space, the authors used only protocol analysis to gather data. The study came under heavy fire because of their decision to use only one method.

Participant Selection

Case studies can use one participant, or a small group of participants. However, it is important that the participant pool remain relatively small. The participants can represent a diverse cross section of society, but this isn't necessary.

For example, the Berkenkotter, Huckin, and Ackerman (1988) study looked at just one participant, Nate. By contrast, in Janet Emig's (1971) study of the composition process of twelfth graders, eight participants were selected representing a diverse cross section of the community, with volunteers from an all-white upper-middle-class suburban school, an all-black inner-city school, a racially mixed lower-middle-class school, an economically and racially mixed school, and a university school.

Often, a brief "case history" is done on the participants of the study in order to provide researchers with a clearer understanding of their participants, as well as some insight as to how their own personal histories might affect the outcome of the study. For instance, in Emig's study, the investigator had access to the school records of five of the participants, and to standardized test scores for the remaining three. Also made available to the researcher was the information that three of the eight students were selected as NCTE Achievement Award winners. These personal histories can be useful in later stages of the study when data are being analyzed and conclusions drawn.

Data Collection

There are six types of data collected in case studies:

• Archival records.
• Interviews.
• Direct observation.
• Participant observation.

In the field of composition research, these six sources might be:

• A writer's drafts.
• School records of student writers.
• Transcripts of interviews with a writer.
• Transcripts of conversations between writers (and protocols).
• Videotapes and notes from direct field observations.
• Hard copies of a writer's work on computer.

Depending on whether researchers have chosen to use a single or multi-modal approach for the case study, they may choose to collect data from one or any combination of these sources.

Protocols, that is, transcriptions of participants talking aloud about what they are doing as they do it, have been particularly common in composition case studies. For example, in Emig's (1971) study, the students were asked, in four different sessions, to give oral autobiographies of their writing experiences and to compose aloud three themes in the presence of a tape recorder and the investigator.

In some studies, only one method of data collection is conducted. For example, the Flower and Hayes (1981) report on the cognitive process theory of writing depends on protocol analysis alone. However, using multiple sources of evidence to increase the reliability and validity of the data can be advantageous.

Case studies are likely to be much more convincing and accurate if they are based on several different sources of information, following a corroborating mode. This conclusion is echoed among many composition researchers. For example, in her study of predrafting processes of high and low-apprehensive writers, Cynthia Selfe (1985) argues that because "methods of indirect observation provide only an incomplete reflection of the complex set of processes involved in composing, a combination of several such methods should be used to gather data in any one study." Thus, in this study, Selfe collected her data from protocols, observations of students role playing their writing processes, audio taped interviews with the students, and videotaped observations of the students in the process of composing.

It can be said then, that cross checking data from multiple sources can help provide a multidimensional profile of composing activities in a particular setting. Sharan Merriam (1985) suggests "checking, verifying, testing, probing, and confirming collected data as you go, arguing that this process will follow in a funnel-like design resulting in less data gathering in later phases of the study along with a congruent increase in analysis checking, verifying, and confirming."

It is important to note that in case studies, as in any qualitative descriptive research, while researchers begin their studies with one or several questions driving the inquiry (which influence the key factors the researcher will be looking for during data collection), a researcher may find new key factors emerging during data collection. These might be unexpected patterns or linguistic features which become evident only during the course of the research. While not bearing directly on the researcher's guiding questions, these variables may become the basis for new questions asked at the end of the report, thus linking to the possibility of further research.

Data Analysis

As the information is collected, researchers strive to make sense of their data. Generally, researchers interpret their data in one of two ways: holistically or through coding. Holistic analysis does not attempt to break the evidence into parts, but rather to draw conclusions based on the text as a whole. Flower and Hayes (1981), for example, make inferences from entire sections of their students' protocols, rather than searching through the transcripts to look for isolatable characteristics.

However, composition researchers commonly interpret their data by coding, that is by systematically searching data to identify and/or categorize specific observable actions or characteristics. These observable actions then become the key variables in the study. Sharan Merriam (1988) suggests seven analytic frameworks for the organization and presentation of data:

• The role of participants.
• The network analysis of formal and informal exchanges among groups.
• Historical.
• Thematical.
• Ritual and symbolism.
• Critical incidents that challenge or reinforce fundamental beliefs, practices, and values.

There are two purposes of these frameworks: to look for patterns among the data and to look for patterns that give meaning to the case study.

As stated above, while most researchers begin their case studies expecting to look for particular observable characteristics, it is not unusual for key variables to emerge during data collection. Typical variables coded in case studies of writers include pauses writers make in the production of a text, the use of specific linguistic units (such as nouns or verbs), and writing processes (planning, drafting, revising, and editing). In the Berkenkotter, Huckin, and Ackerman (1988) study, for example, researchers coded the participant's texts for use of connectives, discourse demonstratives, average sentence length, off-register words, use of the first person pronoun, and the ratio of definite articles to indefinite articles.

Since coding is inherently subjective, more than one coder is usually employed. In the Berkenkotter, Huckin, and Ackerman (1988) study, for example, three rhetoricians were employed to code the participant's texts for off-register phrases. The researchers established the agreement among the coders before concluding that the participant used fewer off-register words as the graduate program progressed.

Composing the Case Study Report

In the many forms it can take, "a case study is generically a story; it presents the concrete narrative detail of actual, or at least realistic events, it has a plot, exposition, characters, and sometimes even dialogue" (Boehrer 1990). Generally, case study reports are extensively descriptive, with "the most problematic issue often referred to as being the determination of the right combination of description and analysis" (1990). Typically, authors address each step of the research process, and attempt to give the reader as much context as possible for the decisions made in the research design and for the conclusions drawn.

This contextualization usually includes a detailed explanation of the researchers' theoretical positions, of how those theories drove the inquiry or led to the guiding research questions, of the participants' backgrounds, of the processes of data collection, of the training and limitations of the coders, along with a strong attempt to make connections between the data and the conclusions evident.

Although the Berkenkotter, Huckin, and Ackerman (1988) study does not, case study reports often include the reactions of the participants to the study or to the researchers' conclusions. Because case studies tend to be exploratory, most end with implications for further study. Here researchers may identify significant variables that emerged during the research and suggest studies related to these, or the authors may suggest further general questions that their case study generated.

For example, Emig's (1971) study concludes with a section dedicated solely to the topic of implications for further research, in which she suggests several means by which this particular study could have been improved, as well as questions and ideas raised by this study which other researchers might like to address, such as: is there a correlation between a certain personality and a certain composing process profile (e.g. is there a positive correlation between ego strength and persistence in revising)?

Also included in Emig's study is a section dedicated to implications for teaching, which outlines the pedagogical ramifications of the study's findings for teachers currently involved in high school writing programs.

Sharan Merriam (1985) also offers several suggestions for alternative presentations of data:

• Prepare specialized condensations for appropriate groups.
• Replace narrative sections with a series of answers to open-ended questions.
• Present "skimmer's" summaries at beginning of each section.
• Incorporate headlines that encapsulate information from text.
• Prepare analytic summaries with supporting data appendixes.
• Present data in colorful and/or unique graphic representations.

Issues of Validity and Reliability

Once key variables have been identified, they can be analyzed. Reliability becomes a key concern at this stage, and many case study researchers go to great lengths to ensure that their interpretations of the data will be both reliable and valid. Because issues of validity and reliability are an important part of any study in the social sciences, it is important to identify some ways of dealing with results.

Multi-modal case study researchers often balance the results of their coding with data from interviews or writer's reflections upon their own work. Consequently, the researchers' conclusions become highly contextualized. For example, in a case study which looked at the time spent in different stages of the writing process, Berkenkotter concluded that her participant, Donald Murray, spent more time planning his essays than in other writing stages. The report of this case study is followed by Murray's reply, wherein he agrees with some of Berkenkotter's conclusions and disagrees with others.

As is the case with other research methodologies, issues of external validity, construct validity, and reliability need to be carefully considered.

Commentary on Case Studies

Researchers often debate the relative merits of particular methods, among them case study. In this section, we comment on two key issues. To read the commentaries, choose any of the items below:

Strengths and Weaknesses of Case Studies

Most case study advocates point out that case studies produce much more detailed information than what is available through a statistical analysis. Advocates will also hold that while statistical methods might be able to deal with situations where behavior is homogeneous and routine, case studies are needed to deal with creativity, innovation, and context. Detractors argue that case studies are difficult to generalize because of inherent subjectivity and because they are based on qualitative subjective data, generalizable only to a particular context.

Flexibility

The case study approach is a comparatively flexible method of scientific research. Because its project designs seem to emphasize exploration rather than prescription or prediction, researchers are comparatively freer to discover and address issues as they arise in their experiments. In addition, the looser format of case studies allows researchers to begin with broad questions and narrow their focus as their experiment progresses rather than attempt to predict every possible outcome before the experiment is conducted.

Emphasis on Context

By seeking to understand as much as possible about a single subject or small group of subjects, case studies specialize in "deep data," or "thick description"--information based on particular contexts that can give research results a more human face. This emphasis can help bridge the gap between abstract research and concrete practice by allowing researchers to compare their firsthand observations with the quantitative results obtained through other methods of research.

Inherent Subjectivity

"The case study has long been stereotyped as the weak sibling among social science methods," and is often criticized as being too subjective and even pseudo-scientific. Likewise, "investigators who do case studies are often regarded as having deviated from their academic disciplines, and their investigations as having insufficient precision (that is, quantification), objectivity and rigor" (Yin 1989). Opponents cite opportunities for subjectivity in the implementation, presentation, and evaluation of case study research. The approach relies on personal interpretation of data and inferences. Results may not be generalizable, are difficult to test for validity, and rarely offer a problem-solving prescription. Simply put, relying on one or a few subjects as a basis for cognitive extrapolations runs the risk of inferring too much from what might be circumstance.

High Investment

Case studies can involve learning more about the subjects being tested than most researchers would care to know--their educational background, emotional background, perceptions of themselves and their surroundings, their likes, dislikes, and so on. Because of its emphasis on "deep data," the case study is out of reach for many large-scale research projects which look at a subject pool in the tens of thousands. A budget request of $10,000 to examine 200 subjects sounds more efficient than a similar request to examine four subjects.

Ethical Considerations

Researchers conducting case studies should consider certain ethical issues. For example, many educational case studies are often financed by people who have, either directly or indirectly, power over both those being studied and those conducting the investigation (1985). This conflict of interests can hinder the credibility of the study.

The personal integrity, sensitivity, and possible prejudices and/or biases of the investigators need to be taken into consideration as well. Personal biases can creep into how the research is conducted, alternative research methods used, and the preparation of surveys and questionnaires.

A common complaint in case study research is that investigators change direction during the course of the study unaware that their original research design was inadequate for the revised investigation. Thus, the researchers leave unknown gaps and biases in the study. To avoid this, researchers should report preliminary findings so that the likelihood of bias will be reduced.

Concerns about Reliability, Validity, and Generalizability

Merriam (1985) offers several suggestions for how case study researchers might actively combat the popular attacks on the validity, reliability, and generalizability of case studies:

• Prolong the Processes of Data Gathering on Site: This will help to insure the accuracy of the findings by providing the researcher with more concrete information upon which to formulate interpretations.
• Employ the Process of "Triangulation": Use a variety of data sources as opposed to relying solely upon one avenue of observation. One example of such a data check would be what McClintock, Brannon, and Maynard (1985) refer to as a "case cluster method," that is, when a single unit within a larger case is randomly sampled, and that data treated quantitatively." For instance, in Emig's (1971) study, the case cluster method was employed, singling out the productivity of a single student named Lynn. This cluster profile included an advanced case history of the subject, specific examination and analysis of individual compositions and protocols, and extensive interview sessions. The seven remaining students were then compared with the case of Lynn, to ascertain if there are any shared, or unique dimensions to the composing process engaged in by these eight students.
• Conduct Member Checks: Initiate and maintain an active corroboration on the interpretation of data between the researcher and those who provided the data. In other words, talk to your subjects.
• Collect Referential Materials: Complement the file of materials from the actual site with additional document support. For example, Emig (1971) supports her initial propositions with historical accounts by writers such as T.S. Eliot, James Joyce, and D.H. Lawrence. Emig also cites examples of theoretical research done with regards to the creative process, as well as examples of empirical research dealing with the writing of adolescents. Specific attention is then given to the four stages description of the composing process delineated by Helmoltz, Wallas, and Cowley, as it serves as the focal point in this study.
• Engage in Peer Consultation: Prior to composing the final draft of the report, researchers should consult with colleagues in order to establish validity through pooled judgment.

Although little can be done to combat challenges concerning the generalizability of case studies, "most writers suggest that qualitative research should be judged as credible and confirmable as opposed to valid and reliable" (Merriam 1985). Likewise, it has been argued that "rather than transplanting statistical, quantitative notions of generalizability and thus finding qualitative research inadequate, it makes more sense to develop an understanding of generalization that is congruent with the basic characteristics of qualitative inquiry" (1985). After all, criticizing the case study method for being ungeneralizable is comparable to criticizing a washing machine for not being able to tell the correct time. In other words, it is unjust to criticize a method for not being able to do something which it was never originally designed to do in the first place.

Annotated Bibliography

Armisted, C. (1984). How Useful are Case Studies. Training and Development Journal, 38 (2), 75-77.

This article looks at eight types of case studies, offers pros and cons of using case studies in the classroom, and gives suggestions for successfully writing and using case studies.

Bardovi-Harlig, K. (1997). Beyond Methods: Components of Second Language Teacher Education . New York: McGraw-Hill.

A compilation of various research essays which address issues of language teacher education. Essays included are: "Non-native reading research and theory" by Lee, "The case for Psycholinguistics" by VanPatten, and "Assessment and Second Language Teaching" by Gradman and Reed.

Bartlett, L. (1989). A Question of Good Judgment; Interpretation Theory and Qualitative Enquiry Address. 70th Annual Meeting of the American Educational Research Association. San Francisco.

Bartlett selected "quasi-historical" methodology, which focuses on the "truth" found in case records, as one that will provide "good judgments" in educational inquiry. He argues that although the method is not comprehensive, it can try to connect theory with practice.

Baydere, S. et. al. (1993). Multimedia conferencing as a tool for collaborative writing: a case study in Computer Supported Collaborative Writing. New York: Springer-Verlag.

The case study by Baydere et. al. is just one of the many essays in this book found in the series "Computer Supported Cooperative Work." Denley, Witefield and May explore similar issues in their essay, "A case study in task analysis for the design of a collaborative document production system."

Berkenkotter, C., Huckin, T., N., & Ackerman J. (1988). Conventions, Conversations, and the Writer: Case Study of a Student in a Rhetoric Ph.D. Program. Research in the Teaching of English, 22, 9-44.

The authors focused on how the writing of their subject, Nate or Ackerman, changed as he became more acquainted or familiar with his field's discourse community.

Berninger, V., W., and Gans, B., M. (1986). Language Profiles in Nonspeaking Individuals of Normal Intelligence with Severe Cerebral Palsy. Augmentative and Alternative Communication, 2, 45-50.

Argues that generalizations about language abilities in patients with severe cerebral palsy (CP) should be avoided. Standardized tests of different levels of processing oral language, of processing written language, and of producing written language were administered to 3 male participants (aged 9, 16, and 40 yrs).

Bockman, J., R., and Couture, B. (1984). The Case Method in Technical Communication: Theory and Models. Texas: Association of Teachers of Technical Writing.

Examines the study and teaching of technical writing, communication of technical information, and the case method in terms of those applications.

Boehrer, J. (1990). Teaching With Cases: Learning to Question. New Directions for Teaching and Learning, 42 41-57.

This article discusses the origins of the case method, looks at the question of what is a case, gives ideas about learning in case teaching, the purposes it can serve in the classroom, the ground rules for the case discussion, including the role of the question, and new directions for case teaching.

Bowman, W. R. (1993). Evaluating JTPA Programs for Economically Disadvantaged Adults: A Case Study of Utah and General Findings . Washington: National Commission for Employment Policy.

"To encourage state-level evaluations of JTPA, the Commission and the State of Utah co-sponsored this report on the effectiveness of JTPA Title II programs for adults in Utah. The technique used is non-experimental and the comparison group was selected from registrants with Utah's Employment Security. In a step-by-step approach, the report documents how non-experimental techniques can be applied and several specific technical issues can be addressed."

Boyce, A. (1993) The Case Study Approach for Pedagogists. Annual Meeting of the American Alliance for Health, Physical Education, Recreation and Dance. (Address). Washington DC.

This paper addresses how case studies 1) bridge the gap between teaching theory and application, 2) enable students to analyze problems and develop solutions for situations that will be encountered in the real world of teaching, and 3) helps students to evaluate the feasibility of alternatives and to understand the ramifications of a particular course of action.

Carson, J. (1993) The Case Study: Ideal Home of WAC Quantitative and Qualitative Data. Annual Meeting of the Conference on College Composition and Communication. (Address). San Diego.

"Increasingly, one of the most pressing questions for WAC advocates is how to keep [WAC] programs going in the face of numerous difficulties. Case histories offer the best chance for fashioning rhetorical arguments to keep WAC programs going because they offer the opportunity to provide a coherent narrative that contextualizes all documents and data, including what is generally considered scientific data. A case study of the WAC program, . . . at Robert Morris College in Pittsburgh demonstrates the advantages of this research method. Such studies are ideal homes for both naturalistic and positivistic data as well as both quantitative and qualitative information."

---. (1991). A Cognitive Process Theory of Writing. College Composition and Communication. 32. 365-87.

No abstract available.

Cromer, R. (1994) A Case Study of Dissociations Between Language and Cognition. Constraints on Language Acquisition: Studies of Atypical Children . Hillsdale: Lawrence Erlbaum Associates, 141-153.

Crossley, M. (1983) Case Study in Comparative and International Education: An Approach to Bridging the Theory-Practice Gap. Proceedings of the 11th Annual Conference of the Australian Comparative and International Education Society. Hamilton, NZ.

Case study research, as presented here, helps bridge the theory-practice gap in comparative and international research studies of education because it focuses on the practical, day-to-day context rather than on the national arena. The paper asserts that the case study method can be valuable at all levels of research, formation, and verification of theories in education.

Daillak, R., H., and Alkin, M., C. (1982). Qualitative Studies in Context: Reflections on the CSE Studies of Evaluation Use . California: EDRS

The report shows how the Center of the Study of Evaluation (CSE) applied qualitative techniques to a study of evaluation information use in local, Los Angeles schools. It critiques the effectiveness and the limitations of using case study, evaluation, field study, and user interview survey methodologies.

Davey, L. (1991). The Application of Case Study Evaluations. ERIC/TM Digest.

This article examines six types of case studies, the type of evaluation questions that can be answered, the functions served, some design features, and some pitfalls of the method.

Deutch, C. E. (1996). A course in research ethics for graduate students. College Teaching, 44, 2, 56-60.

This article describes a one-credit discussion course in research ethics for graduate students in biology. Case studies are focused on within the four parts of the course: 1) major issues, 2 )practical issues in scholarly work, 3) ownership of research results, and 4) training and personal decisions.

DeVoss, G. (1981). Ethics in Fieldwork Research. RIE 27p. (ERIC)

This article examines four of the ethical problems that can happen when conducting case study research: acquiring permission to do research, knowing when to stop digging, the pitfalls of doing collaborative research, and preserving the integrity of the participants.

Driscoll, A. (1985). Case Study of a Research Intervention: the University of Utah’s Collaborative Approach . San Francisco: Far West Library for Educational Research Development.

Paper presented at the annual meeting of the American Association of Colleges of Teacher Education, Denver, CO, March 1985. Offers information of in-service training, specifically case studies application.

Ellram, L. M. (1996). The Use of the Case Study Method in Logistics Research. Journal of Business Logistics, 17, 2, 93.

This article discusses the increased use of case study in business research, and the lack of understanding of when and how to use case study methodology in business.

Emig, J. (1971) The Composing Processes of Twelfth Graders . Urbana: NTCE.

This case study uses observation, tape recordings, writing samples, and school records to show that writing in reflexive and extensive situations caused different lengths of discourse and different clusterings of the components of the writing process.

Feagin, J. R. (1991). A Case For the Case Study . Chapel Hill: The University of North Carolina Press.

This book discusses the nature, characteristics, and basic methodological issues of the case study as a research method.

Feldman, H., Holland, A., & Keefe, K. (1989) Language Abilities after Left Hemisphere Brain Injury: A Case Study of Twins. Topics in Early Childhood Special Education, 9, 32-47.

"Describes the language abilities of 2 twin pairs in which 1 twin (the experimental) suffered brain injury to the left cerebral hemisphere around the time of birth and1 twin (the control) did not. One pair of twins was initially assessed at age 23 mo. and the other at about 30 mo.; they were subsequently evaluated in their homes 3 times at about 6-mo intervals."

Fidel, R. (1984). The Case Study Method: A Case Study. Library and Information Science Research, 6.

The article describes the use of case study methodology to systematically develop a model of online searching behavior in which study design is flexible, subject manner determines data gathering and analyses, and procedures adapt to the study's progressive change.

Flower, L., & Hayes, J. R. (1984). Images, Plans and Prose: The Representation of Meaning in Writing. Written Communication, 1, 120-160.

Explores the ways in which writers actually use different forms of knowing to create prose.

Frey, L. R. (1992). Interpreting Communication Research: A Case Study Approach Englewood Cliffs, N.J.: Prentice Hall.

The book discusses research methodologies in the Communication field. It focuses on how case studies bridge the gap between communication research, theory, and practice.

Gilbert, V. K. (1981). The Case Study as a Research Methodology: Difficulties and Advantages of Integrating the Positivistic, Phenomenological and Grounded Theory Approaches . The Annual Meeting of the Canadian Association for the Study of Educational Administration. (Address) Halifax, NS, Can.

This study on an innovative secondary school in England shows how a "low-profile" participant-observer case study was crucial to the initial observation, the testing of hypotheses, the interpretive approach, and the grounded theory.

Gilgun, J. F. (1994). A Case for Case Studies in Social Work Research. Social Work, 39, 4, 371-381.

This article defines case study research, presents guidelines for evaluation of case studies, and shows the relevance of case studies to social work research. It also looks at issues such as evaluation and interpretations of case studies.

Glennan, S. L., Sharp-Bittner, M. A. & Tullos, D. C. (1991). Augmentative and Alternative Communication Training with a Nonspeaking Adult: Lessons from MH. Augmentative and Alternative Communication, 7, 240-7.

"A response-guided case study documented changes in a nonspeaking 36-yr-old man's ability to communicate using 3 trained augmentative communication modes. . . . Data were collected in videotaped interaction sessions between the nonspeaking adult and a series of adult speaking."

Graves, D. (1981). An Examination of the Writing Processes of Seven Year Old Children. Research in the Teaching of English, 15, 113-134.

Hamel, J. (1993). Case Study Methods . Newbury Park: Sage. .

"In a most economical fashion, Hamel provides a practical guide for producing theoretically sharp and empirically sound sociological case studies. A central idea put forth by Hamel is that case studies must "locate the global in the local" thus making the careful selection of the research site the most critical decision in the analytic process."

Karthigesu, R. (1986, July). Television as a Tool for Nation-Building in the Third World: A Post-Colonial Pattern, Using Malaysia as a Case-Study. International Television Studies Conference. (Address). London, 10-12.

"The extent to which Television Malaysia, as a national mass media organization, has been able to play a role in nation building in the post-colonial period is . . . studied in two parts: how the choice of a model of nation building determines the character of the organization; and how the character of the organization influences the output of the organization."

Kenny, R. (1984). Making the Case for the Case Study. Journal of Curriculum Studies, 16, (1), 37-51.

The article looks at how and why the case study is justified as a viable and valuable approach to educational research and program evaluation.

Knirk, F. (1991). Case Materials: Research and Practice. Performance Improvement Quarterly, 4 (1 ), 73-81.

The article addresses the effectiveness of case studies, subject areas where case studies are commonly used, recent examples of their use, and case study design considerations.

Klos, D. (1976). Students as Case Writers. Teaching of Psychology, 3.2, 63-66.

This article reviews a course in which students gather data for an original case study of another person. The task requires the students to design the study, collect the data, write the narrative, and interpret the findings.

Leftwich, A. (1981). The Politics of Case Study: Problems of Innovation in University Education. Higher Education Review, 13.2, 38-64.

The article discusses the use of case studies as a teaching method. Emphasis is on the instructional materials, interdisciplinarity, and the complex relationships within the university that help or hinder the method.

Mabrito, M. (1991, Oct.). Electronic Mail as a Vehicle for Peer Response: Conversations of High and Low Apprehensive Writers. Written Communication, 509-32.

McCarthy, S., J. (1955). The Influence of Classroom Discourse on Student Texts: The Case of Ella . East Lansing: Institute for Research on Teaching.

A look at how students of color become marginalized within traditional classroom discourse. The essay follows the struggles of one black student: Ella.

Matsuhashi, A., ed. (1987). Writing in Real Time: Modeling Production Processes Norwood, NJ: Ablex Publishing Corporation.

Investigates how writers plan to produce discourse for different purposes to report, to generalize, and to persuade, as well as how writers plan for sentence level units of language. To learn about planning, an observational measure of pause time was used" (ERIC).

Merriam, S. B. (1985). The Case Study in Educational Research: A Review of Selected Literature. Journal of Educational Thought, 19.3, 204-17.

The article examines the characteristics of, philosophical assumptions underlying the case study, the mechanics of conducting a case study, and the concerns about the reliability, validity, and generalizability of the method.

---. (1988). Case Study Research in Education: A Qualitative Approach San Francisco: Jossey Bass.

Merry, S. E., & Milner, N. eds. (1993). The Possibility of Popular Justice: A Case Study of Community Mediation in the United States . Ann Arbor: U of Michigan.

". . . this volume presents a case study of one experiment in popular justice, the San Francisco Community Boards. This program has made an explicit claim to create an alternative justice, or new justice, in the midst of a society ordered by state law. The contributors to this volume explore the history and experience of the program and compare it to other versions of popular justice in the United States, Europe, and the Third World."

Merseth, K. K. (1991). The Case for Cases in Teacher Education. RIE. 42p. (ERIC).

This monograph argues that the case method of instruction offers unique potential for revitalizing the field of teacher education.

Michaels, S. (1987). Text and Context: A New Approach to the Study of Classroom Writing. Discourse Processes, 10, 321-346.

"This paper argues for and illustrates an approach to the study of writing that integrates ethnographic analysis of classroom interaction with linguistic analysis of written texts and teacher/student conversational exchanges. The approach is illustrated through a case study of writing in a single sixth grade classroom during a single writing assignment."

Milburn, G. (1995). Deciphering a Code or Unraveling a Riddle: A Case Study in the Application of a Humanistic Metaphor to the Reporting of Social Studies Teaching. Theory and Research in Education, 13.

This citation serves as an example of how case studies document learning procedures in a senior-level economics course.

Milley, J. E. (1979). An Investigation of Case Study as an Approach to Program Evaluation. 19th Annual Forum of the Association for Institutional Research. (Address). San Diego.

The case study method merged a narrative report focusing on the evaluator as participant-observer with document review, interview, content analysis, attitude questionnaire survey, and sociogram analysis. Milley argues that case study program evaluation has great potential for widespread use.

Minnis, J. R. (1985, Sept.). Ethnography, Case Study, Grounded Theory, and Distance Education Research. Distance Education, 6.2.

This article describes and defines the strengths and weaknesses of ethnography, case study, and grounded theory.

Nunan, D. (1992). Collaborative language learning and teaching . New York: Cambridge University Press.

Included in this series of essays is Peter Sturman’s "Team Teaching: a case study from Japan" and David Nunan’s own "Toward a collaborative approach to curriculum development: a case study."

Nystrand, M., ed. (1982). What Writers Know: The Language, Process, and Structure of Written Discourse . New York: Academic Press.

Owenby, P. H. (1992). Making Case Studies Come Alive. Training, 29, (1), 43-46. (ERIC)

This article provides tips for writing more effective case studies.

---. (1981). Pausing and Planning: The Tempo of Writer Discourse Production. Research in the Teaching of English, 15 (2),113-34.

Perl, S. (1979). The Composing Processes of Unskilled College Writers. Research in the Teaching of English, 13, 317-336.

"Summarizes a study of five unskilled college writers, focusing especially on one of the five, and discusses the findings in light of current pedagogical practice and research design."

Pilcher J. and A. Coffey. eds. (1996). Gender and Qualitative Research . Brookfield: Aldershot, Hants, England.

This book provides a series of essays which look at gender identity research, qualitative research and applications of case study to questions of gendered pedagogy.

Pirie, B. S. (1993). The Case of Morty: A Four Year Study. Gifted Education International, 9 (2), 105-109.

This case study describes a boy from kindergarten through third grade with above average intelligence but difficulty in learning to read, write, and spell.

Popkewitz, T. (1993). Changing Patterns of Power: Social Regulation and Teacher Education Reform. Albany: SUNY Press.

Popkewitz edits this series of essays that address case studies on educational change and the training of teachers. The essays vary in terms of discipline and scope. Also, several authors include case studies of educational practices in countries other than the United States.

---. (1984). The Predrafting Processes of Four High- and Four Low Apprehensive Writers. Research in the Teaching of English, 18, (1), 45-64.

Rasmussen, P. (1985, March) A Case Study on the Evaluation of Research at the Technical University of Denmark. International Journal of Institutional Management in Higher Education, 9 (1).

This is an example of a case study methodology used to evaluate the chemistry and chemical engineering departments at the University of Denmark.

Roth, K. J. (1986). Curriculum Materials, Teacher Talk, and Student Learning: Case Studies in Fifth-Grade Science Teaching . East Lansing: Institute for Research on Teaching.

Roth offers case studies on elementary teachers, elementary school teaching, science studies and teaching, and verbal learning.

Selfe, C. L. (1985). An Apprehensive Writer Composes. When a Writer Can't Write: Studies in Writer's Block and Other Composing-Process Problems . (pp. 83-95). Ed. Mike Rose. NMY: Guilford.

Smith-Lewis, M., R. and Ford, A. (1987). A User's Perspective on Augmentative Communication. Augmentative and Alternative Communication, 3, 12-7.

"During a series of in-depth interviews, a 25-yr-old woman with cerebral palsy who utilized augmentative communication reflected on the effectiveness of the devices designed for her during her school career."

St. Pierre, R., G. (1980, April). Follow Through: A Case Study in Metaevaluation Research . 64th Annual Meeting of the American Educational Research Association. (Address).

The three approaches to metaevaluation are evaluation of primary evaluations, integrative meta-analysis with combined primary evaluation results, and re-analysis of the raw data from a primary evaluation.

Stahler, T., M. (1996, Feb.) Early Field Experiences: A Model That Worked. ERIC.

"This case study of a field and theory class examines a model designed to provide meaningful field experiences for preservice teachers while remaining consistent with the instructor's beliefs about the role of teacher education in preparing teachers for the classroom."

Stake, R. E. (1995). The Art of Case Study Research. Thousand Oaks: Sage Publications.

This book examines case study research in education and case study methodology.

Stiegelbauer, S. (1984) Community, Context, and Co-curriculum: Situational Factors Influencing School Improvements in a Study of High Schools. Presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Discussion of several case studies: one looking at high school environments, another examining educational innovations.

Stolovitch, H. (1990). Case Study Method. Performance And Instruction, 29, (9), 35-37.

This article describes the case study method as a form of simulation and presents guidelines for their use in professional training situations.

Thaller, E. (1994). Bibliography for the Case Method: Using Case Studies in Teacher Education. RIE. 37 p.

This bibliography presents approximately 450 citations on the use of case studies in teacher education from 1921-1993.

Thrane, T. (1986). On Delimiting the Senses of Near-Synonyms in Historical Semantics: A Case Study of Adjectives of 'Moral Sufficiency' in the Old English Andreas. Linguistics Across Historical and Geographical Boundaries: In Honor of Jacek Fisiak on the Occasion of his Fiftieth Birthday . Berlin: Mouton de Gruyter.

United Nations. (1975). Food and Agriculture Organization. Report on the FAO/UNFPA Seminar on Methodology, Research and Country: Case Studies on Population, Employment and Productivity . Rome: United Nations.

This example case study shows how the methodology can be used in a demographic and psychographic evaluation. At the same time, it discusses the formation and instigation of the case study methodology itself.

Van Vugt, J. P., ed. (1994). Aids Prevention and Services: Community Based Research . Westport: Bergin and Garvey.

"This volume has been five years in the making. In the process, some of the policy applications called for have met with limited success, such as free needle exchange programs in a limited number of American cities, providing condoms to prison inmates, and advertisements that depict same-sex couples. Rather than dating our chapters that deal with such subjects, such policy applications are verifications of the type of research demonstrated here. Furthermore, they indicate the critical need to continue community based research in the various communities threatened by acquired immuno-deficiency syndrome (AIDS) . . . "

Welch, W., ed. (1981, May). Case Study Methodology in Educational Evaluation. Proceedings of the Minnesota Evaluation Conference. Minnesota. (Address).

The four papers in these proceedings provide a comprehensive picture of the rationale, methodology, strengths, and limitations of case studies.

Williams, G. (1987). The Case Method: An Approach to Teaching and Learning in Educational Administration. RIE, 31p.

This paper examines the viability of the case method as a teaching and learning strategy in instructional systems geared toward the training of personnel of the administration of various aspects of educational systems.

Yin, R. K. (1993). Advancing Rigorous Methodologies: A Review of 'Towards Rigor in Reviews of Multivocal Literatures.' Review of Educational Research, 61, (3).

"R. T. Ogawa and B. Malen's article does not meet its own recommended standards for rigorous testing and presentation of its own conclusions. Use of the exploratory case study to analyze multivocal literatures is not supported, and the claim of grounded theory to analyze multivocal literatures may be stronger."

---. (1989). Case Study Research: Design and Methods. London: Sage Publications Inc.

This book discusses in great detail, the entire design process of the case study, including entire chapters on collecting evidence, analyzing evidence, composing the case study report, and designing single and multiple case studies.

Related Links

Consider the following list of related Web sites for more information on the topic of case study research. Note: although many of the links cover the general category of qualitative research, all have sections that address issues of case studies.

• Sage Publications on Qualitative Methodology: Search here for a comprehensive list of new books being published about "Qualitative Methodology" http://www.sagepub.co.uk/
• The International Journal of Qualitative Studies in Education: An on-line journal "to enhance the theory and practice of qualitative research in education." On-line submissions are welcome. http://www.tandf.co.uk/journals/tf/09518398.html
• Qualitative Research Resources on the Internet: From syllabi to home pages to bibliographies. All links relate somehow to qualitative research. http://www.nova.edu/ssss/QR/qualres.html

Becker, Bronwyn, Patrick Dawson, Karen Devine, Carla Hannum, Steve Hill, Jon Leydens, Debbie Matuskevich, Carol Traver, & Mike Palmquist. (2005). Case Studies. Writing@CSU . Colorado State University. https://writing.colostate.edu/guides/guide.cfm?guideid=60

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

What Is a Case Study?

Weighing the pros and cons of this method of research

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Cara Lustik is a fact-checker and copywriter.

Verywell / Colleen Tighe

• Pros and Cons

What Types of Case Studies Are Out There?

Where do you find data for a case study, how do i write a psychology case study.

A case study is an in-depth study of one person, group, or event. In a case study, nearly every aspect of the subject's life and history is analyzed to seek patterns and causes of behavior. Case studies can be used in many different fields, including psychology, medicine, education, anthropology, political science, and social work.

The point of a case study is to learn as much as possible about an individual or group so that the information can be generalized to many others. Unfortunately, case studies tend to be highly subjective, and it is sometimes difficult to generalize results to a larger population.

While case studies focus on a single individual or group, they follow a format similar to other types of psychology writing. If you are writing a case study, we got you—here are some rules of APA format to reference.  

At a Glance

A case study, or an in-depth study of a person, group, or event, can be a useful research tool when used wisely. In many cases, case studies are best used in situations where it would be difficult or impossible for you to conduct an experiment. They are helpful for looking at unique situations and allow researchers to gather a lot of˜ information about a specific individual or group of people. However, it's important to be cautious of any bias we draw from them as they are highly subjective.

What Are the Benefits and Limitations of Case Studies?

A case study can have its strengths and weaknesses. Researchers must consider these pros and cons before deciding if this type of study is appropriate for their needs.

One of the greatest advantages of a case study is that it allows researchers to investigate things that are often difficult or impossible to replicate in a lab. Some other benefits of a case study:

• Allows researchers to capture information on the 'how,' 'what,' and 'why,' of something that's implemented
• Gives researchers the chance to collect information on why one strategy might be chosen over another
• Permits researchers to develop hypotheses that can be explored in experimental research

On the other hand, a case study can have some drawbacks:

• It cannot necessarily be generalized to the larger population
• Cannot demonstrate cause and effect
• It may not be scientifically rigorous
• It can lead to bias

Researchers may choose to perform a case study if they want to explore a unique or recently discovered phenomenon. Through their insights, researchers develop additional ideas and study questions that might be explored in future studies.

It's important to remember that the insights from case studies cannot be used to determine cause-and-effect relationships between variables. However, case studies may be used to develop hypotheses that can then be addressed in experimental research.

Case Study Examples

There have been a number of notable case studies in the history of psychology. Much of  Freud's work and theories were developed through individual case studies. Some great examples of case studies in psychology include:

• Anna O : Anna O. was a pseudonym of a woman named Bertha Pappenheim, a patient of a physician named Josef Breuer. While she was never a patient of Freud's, Freud and Breuer discussed her case extensively. The woman was experiencing symptoms of a condition that was then known as hysteria and found that talking about her problems helped relieve her symptoms. Her case played an important part in the development of talk therapy as an approach to mental health treatment.
• Phineas Gage : Phineas Gage was a railroad employee who experienced a terrible accident in which an explosion sent a metal rod through his skull, damaging important portions of his brain. Gage recovered from his accident but was left with serious changes in both personality and behavior.
• Genie : Genie was a young girl subjected to horrific abuse and isolation. The case study of Genie allowed researchers to study whether language learning was possible, even after missing critical periods for language development. Her case also served as an example of how scientific research may interfere with treatment and lead to further abuse of vulnerable individuals.

Such cases demonstrate how case research can be used to study things that researchers could not replicate in experimental settings. In Genie's case, her horrific abuse denied her the opportunity to learn a language at critical points in her development.

This is clearly not something researchers could ethically replicate, but conducting a case study on Genie allowed researchers to study phenomena that are otherwise impossible to reproduce.

There are a few different types of case studies that psychologists and other researchers might use:

• Collective case studies : These involve studying a group of individuals. Researchers might study a group of people in a certain setting or look at an entire community. For example, psychologists might explore how access to resources in a community has affected the collective mental well-being of those who live there.
• Descriptive case studies : These involve starting with a descriptive theory. The subjects are then observed, and the information gathered is compared to the pre-existing theory.
• Explanatory case studies : These   are often used to do causal investigations. In other words, researchers are interested in looking at factors that may have caused certain things to occur.
• Exploratory case studies : These are sometimes used as a prelude to further, more in-depth research. This allows researchers to gather more information before developing their research questions and hypotheses .
• Instrumental case studies : These occur when the individual or group allows researchers to understand more than what is initially obvious to observers.
• Intrinsic case studies : This type of case study is when the researcher has a personal interest in the case. Jean Piaget's observations of his own children are good examples of how an intrinsic case study can contribute to the development of a psychological theory.

The three main case study types often used are intrinsic, instrumental, and collective. Intrinsic case studies are useful for learning about unique cases. Instrumental case studies help look at an individual to learn more about a broader issue. A collective case study can be useful for looking at several cases simultaneously.

The type of case study that psychology researchers use depends on the unique characteristics of the situation and the case itself.

There are a number of different sources and methods that researchers can use to gather information about an individual or group. Six major sources that have been identified by researchers are:

• Archival records : Census records, survey records, and name lists are examples of archival records.
• Direct observation : This strategy involves observing the subject, often in a natural setting . While an individual observer is sometimes used, it is more common to utilize a group of observers.
• Documents : Letters, newspaper articles, administrative records, etc., are the types of documents often used as sources.
• Interviews : Interviews are one of the most important methods for gathering information in case studies. An interview can involve structured survey questions or more open-ended questions.
• Participant observation : When the researcher serves as a participant in events and observes the actions and outcomes, it is called participant observation.
• Physical artifacts : Tools, objects, instruments, and other artifacts are often observed during a direct observation of the subject.

If you have been directed to write a case study for a psychology course, be sure to check with your instructor for any specific guidelines you need to follow. If you are writing your case study for a professional publication, check with the publisher for their specific guidelines for submitting a case study.

Here is a general outline of what should be included in a case study.

Section 1: A Case History

This section will have the following structure and content:

Background information : The first section of your paper will present your client's background. Include factors such as age, gender, work, health status, family mental health history, family and social relationships, drug and alcohol history, life difficulties, goals, and coping skills and weaknesses.

Description of the presenting problem : In the next section of your case study, you will describe the problem or symptoms that the client presented with.

Describe any physical, emotional, or sensory symptoms reported by the client. Thoughts, feelings, and perceptions related to the symptoms should also be noted. Any screening or diagnostic assessments that are used should also be described in detail and all scores reported.

Your diagnosis : Provide your diagnosis and give the appropriate Diagnostic and Statistical Manual code. Explain how you reached your diagnosis, how the client's symptoms fit the diagnostic criteria for the disorder(s), or any possible difficulties in reaching a diagnosis.

Section 2: Treatment Plan

This portion of the paper will address the chosen treatment for the condition. This might also include the theoretical basis for the chosen treatment or any other evidence that might exist to support why this approach was chosen.

• Cognitive behavioral approach : Explain how a cognitive behavioral therapist would approach treatment. Offer background information on cognitive behavioral therapy and describe the treatment sessions, client response, and outcome of this type of treatment. Make note of any difficulties or successes encountered by your client during treatment.
• Humanistic approach : Describe a humanistic approach that could be used to treat your client, such as client-centered therapy . Provide information on the type of treatment you chose, the client's reaction to the treatment, and the end result of this approach. Explain why the treatment was successful or unsuccessful.
• Psychoanalytic approach : Describe how a psychoanalytic therapist would view the client's problem. Provide some background on the psychoanalytic approach and cite relevant references. Explain how psychoanalytic therapy would be used to treat the client, how the client would respond to therapy, and the effectiveness of this treatment approach.
• Pharmacological approach : If treatment primarily involves the use of medications, explain which medications were used and why. Provide background on the effectiveness of these medications and how monotherapy may compare with an approach that combines medications with therapy or other treatments.

This section of a case study should also include information about the treatment goals, process, and outcomes.

When you are writing a case study, you should also include a section where you discuss the case study itself, including the strengths and limitiations of the study. You should note how the findings of your case study might support previous research. 

In your discussion section, you should also describe some of the implications of your case study. What ideas or findings might require further exploration? How might researchers go about exploring some of these questions in additional studies?

Need More Tips?

Here are a few additional pointers to keep in mind when formatting your case study:

• Never refer to the subject of your case study as "the client." Instead, use their name or a pseudonym.
• Read examples of case studies to gain an idea about the style and format.
• Remember to use APA format when citing references .

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach .  BMC Med Res Methodol . 2011;11:100.

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach . BMC Med Res Methodol . 2011 Jun 27;11:100. doi:10.1186/1471-2288-11-100

Gagnon, Yves-Chantal.  The Case Study as Research Method: A Practical Handbook . Canada, Chicago Review Press Incorporated DBA Independent Pub Group, 2010.

Yin, Robert K. Case Study Research and Applications: Design and Methods . United States, SAGE Publications, 2017.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons

Margin Size

• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

1.1: Case Study: Why Should You Learn About Science?

• Last updated
• Save as PDF
• Page ID 22423

• Suzanne Wakim & Mandeep Grewal
• Butte College

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

\( \newcommand{\Span}{\mathrm{span}}\)

\( \newcommand{\id}{\mathrm{id}}\)

\( \newcommand{\kernel}{\mathrm{null}\,}\)

\( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\)

\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\)

\( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

\( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vectorC}[1]{\textbf{#1}} \)

\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

Case Study: To Give a Shot or Not

Elena and Daris are expecting their first child. They are excited for the baby to arrive, but they are nervous as well. Will the baby be healthy? Will they be good parents? In addition to these big concerns, it seems like there are a million decisions to be made. Will Elena breastfeed or will they use formula? Will they buy a crib or let the baby sleep in their bed?

Elena goes online to try to find some answers. She finds a website from an author who writes books on parenting. On this site, she reads an article that argues that children should not be given many of the standard childhood vaccines, including the measles, mumps, and rubella (MMR) vaccine.

The article claims that the MMR vaccine has been proven to cause autism and gives examples of three children who came down with autism-like symptoms shortly after their first MMR vaccination at one year of age. The author believes that the recent increase in the incidence of children diagnosed with autism spectrum disorders is due to the fact that the number of vaccinations given in childhood has increased.

Elena is concerned. She does not want to create lifelong challenges for their child. Besides, aren’t diseases like measles, mumps, and rubella basically eradicated by now? Why should they risk the health of their baby by injecting them with vaccines for diseases that are a thing of the past?

Once baby Juan is born, Elena brings them to the pediatrician’s office. Dr. Rodriguez says Juan needs some shots. Elena is reluctant and shares what she has read online. Dr. Rodriguez assures Elena that the study that originally claimed a link between the MMR vaccine and autism has been found to be fraudulent and that vaccines have repeatedly been demonstrated to be safe and effective in peer-reviewed studies.

Although Elena trusts their doctor, she is not fully convinced. What about the increase in the number of children with autism and the cases where symptoms of autism appeared after MMR vaccination? Elena has a tough decision to make, but a better understanding of science can help her. In this chapter, you will learn about what science is (and what it is not), how it works, and how it relates to human health.

Chapter Overview: The Nature and Process of Science

In the rest of the chapter, you'll learn much more about science, including how scientists think and how they advance scientific knowledge. Specifically, you'll learn that:

• Science is a distinctive way of gaining knowledge about the natural world that is based on evidence and logic. Scientists assume that nature can be understood with systematic study; that scientific ideas are open to revision, although sound scientific ideas can withstand repeated testing; and that science is limited in the types of questions it can answer.
• A scientific theory is at the pinnacle of explanations in science. A theory is a broad explanation for many phenomena that is widely accepted because it is supported by a great deal of evidence. An example of a theory in human biology is the germ theory of disease. It took more than two centuries of research to provide enough evidence that microorganisms ("germs") cause disease for this explanation to become widely accepted and attain the status of a theory.
• The process of science is epitomized by scientific investigation. This is a procedure for gathering evidence to test a hypothesis. A scientific investigation typically involves steps such as asking a question based on observations and formulating a hypothesis as a testable answer to the question. It also generally involves collecting data as evidence for or against the hypothesis, drawing conclusions, and communicating results. In reality, the process of science is not simple and straightforward. The process actually tends to be nonlinear, iterative, creative, and unpredictable. "Doing" science can be very exciting!
• Scientific experiments are a special type of scientific investigation, in which variables are manipulated by the researcher to test expected outcomes. Experiments are performed under controlled conditions to mitigate the effects of other variables on the outcome variable. Experiments provide the best evidence that one variable causes another variable in scientific research. An example of an experiment in human biology is the astounding public health experiment to test Salk's polio vaccine that was undertaken in 1953. Some 600,000 children received a vaccine injection; another 600,000 received a placebo injection of useless salt water. The vaccine group had a significant drop in polio cases relative to the placebo group, providing support for the hypothesis that the vaccine prevented the disease.
• Many questions in human biology are not amenable to experimental research. Consider the question: "Does smoking cause lung cancer?" It would not be ethical to deliberately experiment with human subjects by exposing them to harmful tobacco smoke in order to see whether they develop lung cancer. For questions like this, observational studies are done to look for correlations between variables. For example, Doll and Hill gathered information on past smoking habits from a large sample of lung cancer patients and another large sample of controls without lung cancer. Smoking and lung cancer were found to be correlated. Correlation does not imply causation, but it can be a big hint!
• Research involving human subjects presents special challenges to scientists. Until the 1970s, there were few ethical guidelines for researchers to follow when studying human subjects. A shamefully unethical syphilis study called the Tuskegee study changed all that. The Tuskegee study was conducted on African-American men in Alabama from 1932 to 1972. This study was done to see the progression of syphilis. In this study, the control group with the disease was not treated for syphilis. When details of the study were leaked to the media, the public was outraged and the U.S. Congress got involved. In 1974, Congress passed important legislation to protect human subjects in scientific research projects. Chief among the protections was the necessity of informed consent.

As you read this chapter, think about the following questions:

• What do you think about the quality of Elena’s online source of information about vaccines compared to Dr. Rodriguez’s sources?
• Do you think the arguments presented here that claim that the MMR vaccine causes autism are scientifically valid? Could there be alternative explanations for the observations?
• Why do you think diseases like measles, polio, and mumps are rare these days, and why are we still vaccinating for these diseases?

Attributions

• Pregnant woman by Petar Milošević licenced CC BY-SA 4.0 via Wikimedia Commons
• Text adapted from Human Biology by CK-12 licensed CC BY-NC 3.0

Why Write a Scientific Argument?

A scientific argument must persuade the reader that the evidence presented and the data cited are strong enough to support a theory, model, or proposed action. To "win" the argument, writers must make it easy for readers to understand their main points and the data that support them. Therefore, writing a scientific argument (also known as a scientific paper) can be thought of as a process for learning rather than a way of documenting what has been learned.

In producing a scientific paper, writers develop their own understanding of several interrelated concepts, write and refine precise descriptions of materials and processes that are involved, decide which material is irrelevant and should be left out, choose or develop figures that convey meaning most clearly and annotate them effectively, and build a body of evidence from which a solid conclusion follows. The ability to tap into rich sources of data, yet constrain the range of information ultimately used in making the argument is another skill that scientific writers develop. By laying out thought processes in an organized manner and producing a paper that follows the style and structure of established journals in the field, writers gain critical-thinking skills, develop communication skills, and generate a product that compares with those presented by practicing scientists.

« Previous Page       Next Page »

• Writing to Support the Theory of Plate Tectonics
• Teaching Notes
• Case Study: Why Write a Scientific Argument?
• Step-by-Step Instructions
• Going Further
• Tools and Data

• About this Site
• Accessibility

Citing and Terms of Use

Material on this page is offered under a Creative Commons license unless otherwise noted below.

Show terms of use for text on this page »

Show terms of use for media on this page »

• None found in this page
• Initial Publication Date: September 29, 2008
• Short URL: https://serc.carleton.edu/27840 What's This?

• Publications
• Conferences & Events
• Professional Learning
• Science Standards
• Awards & Competitions
• Instructional Materials
• Free Resources
• American Rescue Plan
• For Preservice Teachers

NCCSTS Case Collection

• Science and STEM Education Jobs
• Interactive eBooks+
• Digital Catalog
• Regional Product Representatives
• e-Newsletters
• Bestselling Books
• Latest Books
• Popular Book Series
• Prospective Authors
• Web Seminars
• Exhibits & Sponsorship
• Conference Reviewers
• National Conference • Denver 24
• Leaders Institute 2024
• National Conference • New Orleans 24
• Submit a Proposal
• Latest Resources
• Professional Learning Units & Courses
• For Districts
• Online Course Providers
• Schools & Districts
• College Professors & Students
• The Standards
• Teachers and Admin
• eCYBERMISSION
• Toshiba/NSTA ExploraVision
• Junior Science & Humanities Symposium
• Teaching Awards
• Climate Change
• Earth & Space Science
• New Science Teachers
• Early Childhood
• Middle School
• High School
• Postsecondary
• Informal Education
• Journal Articles
• Lesson Plans
• e-newsletters
• Science & Children
• Science Scope
• The Science Teacher
• Journal of College Sci. Teaching
• Connected Science Learning
• NSTA Reports
• Next-Gen Navigator
• Science Update
• Teacher Tip Tuesday
• Trans. Sci. Learning

MyNSTA Community

• My Collections

Case Study Listserv

Permissions & Guidelines

Submit a Case Study

Resources & Publications

Enrich your students’ educational experience with case-based teaching

The NCCSTS Case Collection, created and curated by the National Center for Case Study Teaching in Science, on behalf of the University at Buffalo, contains over a thousand peer-reviewed case studies on a variety of topics in all areas of science.

Cases (only) are freely accessible; subscription is required for access to teaching notes and answer keys.

Subscribe Today

Browse Case Studies

Latest Case Studies

Development of the NCCSTS Case Collection was originally funded by major grants to the University at Buffalo from the National Science Foundation , The Pew Charitable Trusts , and the U.S. Department of Education .

• Browse All Articles
• Newsletter Sign-Up

• 23 Apr 2024
• In Practice

Getting to Net Zero: The Climate Standards and Ecosystem the World Needs Now

What can companies and regulators do as climate predictions grow grimmer? They should measure impact, strengthen environmental institutions, and look to cities to lead, say Robert Kaplan, Shirley Lu, and Rosabeth Moss Kanter.

• 22 Apr 2024
• Research & Ideas

When Does Impact Investing Make the Biggest Impact?

More investors want to back businesses that contribute to social change, but are impact funds the only approach? Research by Shawn Cole, Leslie Jeng, Josh Lerner, Natalia Rigol, and Benjamin Roth challenges long-held assumptions about impact investing and reveals where such funds make the biggest difference.

• 18 Mar 2024

When It Comes to Climate Regulation, Energy Companies Take a More Nuanced View

Many assume that major oil and gas companies adamantly oppose climate-friendly regulation, but that's not true. A study of 30 years of corporate advocacy by Jonas Meckling finds that energy companies have backed clean-energy efforts when it aligns with their business interests.

• 12 Mar 2024

How Used Products Can Unlock New Markets: Lessons from Apple's Refurbished iPhones

The idea of reselling old smartphones might have seemed risky for a company known for high-end devices, but refurbished products have become a major profit stream for Apple and an environmental victory. George Serafeim examines Apple's circular model in a case study, and offers insights for other industries.

• 27 Feb 2024
• Cold Call Podcast

How Could Harvard Decarbonize Its Supply Chain?

Harvard University aims to be fossil-fuel neutral by 2026 and totally free of fossil fuels by 2050. As part of this goal, the university is trying to decarbonize its supply chain and considers replacing cement with a low-carbon substitute called Pozzotive®, made with post-consumer recycled glass. A successful pilot project could jump start Harvard’s initiative to reduce embodied carbon emissions, but it first needs credible information about the magnitude and validity of potential carbon reductions. Harvard Business School professor emeritus Robert Kaplan and assistant professor Shirley Lu discuss the flow of emissions along the supply chain of Harvard University’s construction projects, the different methods of measuring carbon emissions, including the E-liability approach, and the opportunity to leverage blockchain technology to facilitate the flow of comparable and reliable emissions information in the case, “Harvard University and Urban Mining Industries: Decarbonizing the Supply Chain.”

• 30 Jan 2024

Can Second-Generation Ethanol Production Help Decarbonize the World?

Raízen, a bioenergy company headquartered in São Paulo, is Brazil’s leader in sugar and ethanol production and the world’s leading ethanol trader. Since its creation in 2011, the company had primarily produced first-generation ethanol (E1G) from sugarcane, a crop that can also be used to produce sugar. In 2015, Raízen also started to produce second-generation ethanol (E2G), a biofuel derived from residual and waste materials, such as cane bagasse and straw – which don’t compete with food production. The company’s growth strategy focused on developing and boosting a low carbon portfolio that focused on E2G, based on the belief that Raízen—and Brazil—could help the world decarbonize and profit from the energy transition. Paula Kovarsky, Raízen’s chief strategy and sustainability officer, was confident the company could become a global green energy champion. But after the board’s approval for the first round of E2G investments, she faced a complex challenge: how to expand the market for second-generation ethanol and other sugar-cane waste biofuels, in order to ensure Raízen’s long-term growth? Harvard Business School professor Gunnar Trumbull and Kovarsky discuss the company’s strategy for bringing second-generation ethanol to the world in the case, “Raízen: Helping to Decarbonize the World?”

• 29 Jan 2024

Do Disasters Rally Support for Climate Action? It's Complicated.

Reactions to devastating wildfires in the Amazon show the contrasting realities for people living in areas vulnerable to climate change. Research by Paula Rettl illustrates the political ramifications that arise as people weigh the economic tradeoffs of natural disasters.

• 17 Jan 2024

Are Companies Getting Away with 'Cheap Talk' on Climate Goals?

Many companies set emissions targets with great fanfare—and never meet them, says research by Shirley Lu and colleagues. But what if investors held businesses accountable for achieving their climate plans?

• 09 Jan 2024

Could Clean Hydrogen Become Affordable at Scale by 2030?

The cost to produce hydrogen could approach the $1-per-kilogram target set by US regulators by 2030, helping this cleaner energy source compete with fossil fuels, says research by Gunther Glenk and colleagues. But planned global investments in hydrogen production would need to come to fruition to reach full potential.

• 02 Jan 2024

Should Businesses Take a Stand on Societal Issues?

Should businesses take a stand for or against particular societal issues? And how should leaders determine when and how to engage on these sensitive matters? Harvard Business School Senior Lecturer Hubert Joly, who led the electronics retailer Best Buy for almost a decade, discusses examples of corporate leaders who had to determine whether and how to engage with humanitarian crises, geopolitical conflict, racial justice, climate change, and more in the case, “Deciding When to Engage on Societal Issues.”

10 Trends to Watch in 2024

Employees may seek new approaches to balance, even as leaders consider whether to bring more teams back to offices or make hybrid work even more flexible. These are just a few trends that Harvard Business School faculty members will be following during a year when staffing, climate, and inclusion will likely remain top of mind.

• 19 Sep 2023

What Chandrayaan-3 Says About India's Entrepreneurial Approach to Space

India reached an unexplored part of the moon despite its limited R&D funding compared with NASA and SpaceX. Tarun Khanna discusses the significance of the landing, and the country's advancements in data and digital technology.

• 12 Sep 2023
• What Do You Think?

Who Gets the Loudest Voice in DEI Decisions?

Business leaders are wrestling with how to manage their organizations' commitment to diversity, equity, and inclusion. If you were a CEO, which constituency would you consider most: your employees, customers, or investors? asks James Heskett. Open for comment; 0 Comments.

• 26 Jul 2023

STEM Needs More Women. Recruiters Often Keep Them Out

Tech companies and programs turn to recruiters to find top-notch candidates, but gender bias can creep in long before women even apply, according to research by Jacqueline Ng Lane and colleagues. She highlights several tactics to make the process more equitable.

• 18 Jul 2023

Will Global Demand for Oil Peak This Decade?

The International Energy Agency expects the world's oil demand to start to ebb in the coming years. However, Joseph Lassiter and Lauren Cohen say the outlook will likely be more complex, especially as poor and fast-growing regions seek energy sources for their economies.

• 28 Apr 2023

Sweden’s Northvolt Electric Battery Maker: A Startup with a Mission

In Stockholm, Sweden an upstart battery maker, Northvolt, is trying to recreate the value chain for European car manufacturers making the switch to EVs. With two founders from Tesla and two experienced financiers at the helm, the company seems bound for success. But can they partner with government, scale fast enough, and truly be part of the climate solution? Harvard Business School professor George Serafeim discusses what it takes to scale a business—the right people, in the right place, at the right time—with the aim of providing a climate solution in the case, “Northvolt, Building Batteries to Fight Climate Change.” As part of a new first-year MBA course at Harvard Business School, this case examines the central question: what is the social purpose of the firm?

• 18 Apr 2023

What Happens When Banks Ditch Coal: The Impact Is 'More Than Anyone Thought'

Bank divestment policies that target coal reduced carbon dioxide emissions, says research by Boris Vallée and Daniel Green. Could the finance industry do even more to confront climate change?

• 11 Apr 2023

A Rose by Any Other Name: Supply Chains and Carbon Emissions in the Flower Industry

Headquartered in Kitengela, Kenya, Sian Flowers exports roses to Europe. Because cut flowers have a limited shelf life and consumers want them to retain their appearance for as long as possible, Sian and its distributors used international air cargo to transport them to Amsterdam, where they were sold at auction and trucked to markets across Europe. But when the Covid-19 pandemic caused huge increases in shipping costs, Sian launched experiments to ship roses by ocean using refrigerated containers. The company reduced its costs and cut its carbon emissions, but is a flower that travels halfway around the world truly a “low-carbon rose”? Harvard Business School professors Willy Shih and Mike Toffel debate these questions and more in their case, “Sian Flowers: Fresher by Sea?”

• 28 Mar 2023

BMW’s Decarbonization Strategy: Sustainable for the Environment and the Bottom Line

In mid-2022, automakers, consumers, regulators, and investors were focusing on the transition from internal combustion engine (ICE) vehicles to electric vehicles (EV). While this would reduce tail-pipe emissions, it ignored the fact that the production of EVs—and especially their batteries—increases emissions in the supply chain. Many automakers were announcing deadlines by which they would stop selling ICE vehicles altogether, buoyed by investment analysts and favorable press. But BMW decided to focus on lifecycle emissions and pursued a flexible powertrain strategy by offering vehicles with several options: gasoline and diesel-fueled ICE, plug-in hybrid electric vehicles, and battery electric vehicles. That approach received a frostier reception in the stock market. Assistant Professor Shirley Lu discusses how BMW plans to convince stakeholders that its strategy is good for both the environment and the company’s financial performance in the case, “Driving Decarbonization at BMW.”

• 23 Mar 2023

As Climate Fears Mount, More Investors Turn to 'ESG' Funds Despite Few Rules

Regulations and ratings remain murky, but that's not deterring climate-conscious investors from paying more for funds with an ESG label. Research by Mark Egan and Malcolm Baker sizes up the premium these funds command. Is it time for more standards in impact investing?

“Explain” in scientific discourse

• Published: 25 April 2012
• Volume 190 , pages 1383–1405, ( 2013 )

Cite this article

• James A. Overton 1  

891 Accesses

28 Citations

1 Altmetric

Explore all metrics

The philosophical literature on scientific explanation contains a striking diversity of accounts. I use novel empirical methods to address this fragmentation and assess the importance and generality of explanation in science. My evidence base is a set of 781 articles from one year of the journal Science , and I begin by applying text mining techniques to discover patterns in the usage of “explain” and other words of philosophical interest. I then use random sampling from the data set to develop and test a classification scheme for scientific explanation. My results show that explanation and inference to the best explanation are ubiquitous in science, that they occur across a wide range of scientific disciplines, and that they are a goal of scientific practise. These explanations and inferences to the best explanation come in a diversity forms, which at least partially justifies the fragmentation of philosophical accounts. I draw two methodological lessons: first that text mining can enhance traditional conceptual analysis by establishing facts about word usage; and second that random sampling of cases can increase our confidence that a philosophical account applies in general. These empirical techniques supplement traditional philosophical methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Philosophical reasoning about science: a quantitative, digital study

The Automated Laplacean Demon: How ML Challenges Our Views on Prediction and Explanation

Non-causal explanations in the humanities: some examples.

Batterman R. W. (2002) The devil in the details: Asymptotic reasoning in explanation, reduction, and emergence. Oxford University Press, Oxford

Google Scholar  

Bechtel W., Abrahamsen A. (2005) Explanation: A mechanist alternative. Studies in History and Philosophy of Biological and Biomedical Sciences 36: 421–441

Article   Google Scholar  

Bokulich A. (2009) How scientific models can explain. Synthese 180(1): 33–45

Cárdenas M., Gosling W., Sherlock S., Poole I., Pennington R., Mothes P. (2011) The response of vegetation on the Andean flank in western Amazonia to Pleistocene climate change. Science 331(6020): 1055–1058

Das S., Eder S., Schauer S., Diwoky C., Temmel H., Guertl B. et al (2011) Adipose triglyceride lipase contributes to cancer-associated cachexia. Science, 333(6039): 233

Dowe P. (2000) Physical causation. Cambridge University Press, Cambridge

Book   Google Scholar  

Faith J., McNulty N., Rey F., Gordon J. (2011) Predicting a human gut microbiota’s response to diet in gnotobiotic mice. Science 333(6038): 101

Friedman M. (1974) Explanation and scientific understanding. The Journal of Philosophy 71(1): 5–19

Gerson A., Guglielmo C. (2011) Flight at low ambient humidity increases protein catabolism in migratory birds. Science 333(6048): 1434–1436

Gupta S., Luo X., Mohanty B., Ritter H., Xu N. (2011) Scale for the phase diagram of quantum chromodynamics. Science 332(6037): 1525

Hempel C. G. (1965) Aspects of scientific explanation: And other essays in the philosophy of science. Free Press, New York

Hempel C. G., Oppenheim P. (1948) Studies in the logic of explanation. Philosophy of Science 15(2): 135–175

Herd C., Blinova A., Simkus D., Huang Y., Tarozo R., Alexander C. et al (2011) Origin and evolution of prebiotic organic matter as inferred from the Tagish Lake meteorite. Science 332(6035): 1304

Kiers E., Duhamel M., Beesetty Y., Mensah J., Franken O., Verbruggen E. et al (2011) Reciprocal rewards stabilize cooperation in the mycorrhizal symbiosis. Science 333(6044): 880

Kitcher P. (1989) Explanatory unification and the causal structure of the world. In: Kitcher P, Salmon W.C. (eds) Scientific explanation. Minnesota studies in the philosophy of science. University of Minnesota Press, Minneapolis, pp 410–505

Lopez-Garcia C., Klein A., Simons B., Winton D. (2010) Intestinal stem cell replacement follows a pattern of neutral drift. Science 330(6005): 822

Machamer P. K., Darden L., Craver C. F. (2000) Thinking about mechanisms. Philosophy of Science 67(1): 1–25

Mitchell S. D. (1997) Pragmatic laws. Philosophy of Science 64: 468–479

Mitchell S. D. (2009) Unsimple truths: Science, complexity, and policy. University of Chicago Press, Chicago

Olsen A., Ninnemann U. (2010) Large δ 13 C gradients in the preindustrial North Atlantic revealed. Science 330(6004): 658

Overton, J. A. (2011). Mechanisms, types, and abstractions. Philosophy of Science, 78(5), 941–954. http://www.jstor.org/stable/10.1086/662274 .

Rust M., Golden S., O’Shea E. (2011) Light-driven changes in energy metabolism directly entrain the cyanobacterial circadian oscillator. Science 331(6014): 220

Salmon W. C. (1971) Statistical explanation. In: Salmon W. C. (eds) Statistical explanation and statistical relevance. University of Pittsburgh Press, Pittsburgh, pp 29–87

Salmon W. C. (1984) Scientific explanation and the causal structure of the world. Princeton University Press, Princeton, NJ

Setiawan J., Klement R., Henning T., Rix H., Rochau B., Rodmann J. et al (2010) A giant planet around a metal-poor star of extragalactic origin. Science 330(6011): 1642

van Fraassen B. C. (1980) The scientific image. Clarendon Press, Oxford

Woods R., Barrick J., Cooper T., Shrestha U., Kauth M., Lenski R. (2011) Second-order selection for evolvability in a large Escherichia coli population. Science 331(6023): 1433

Woodward J. (2003) Making things happen: A theory of causal explanation. Oxford University Press, Oxford

Woodward, J. (2011). Scientific explanation. In E. N. Zalta (Ed.) The Stanford encyclopedia of philosophy . Metaphysics Research Lab, winter 2011 edition.

Download references

Author information

Authors and affiliations.

The University of Western Ontario, Stevenson Hall, London, ON, N6A 5B8, Canada

James A. Overton

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to James A. Overton .

Rights and permissions

Reprints and permissions

About this article

Overton, J.A. “Explain” in scientific discourse. Synthese 190 , 1383–1405 (2013). https://doi.org/10.1007/s11229-012-0109-8

Download citation

Received : 04 January 2012

Accepted : 04 April 2012

Published : 25 April 2012

Issue Date : May 2013

DOI : https://doi.org/10.1007/s11229-012-0109-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Philosophy of science
• Explanation
• Text mining
• Methodology
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 27 June 2011

The case study approach

• Sarah Crowe 1 ,
• Kathrin Cresswell 2 ,
• Ann Robertson 2 ,
• Guro Huby 3 ,
• Anthony Avery 1 &
• Aziz Sheikh 2  

BMC Medical Research Methodology volume  11 , Article number:  100 ( 2011 ) Cite this article

782k Accesses

1038 Citations

37 Altmetric

Metrics details

The case study approach allows in-depth, multi-faceted explorations of complex issues in their real-life settings. The value of the case study approach is well recognised in the fields of business, law and policy, but somewhat less so in health services research. Based on our experiences of conducting several health-related case studies, we reflect on the different types of case study design, the specific research questions this approach can help answer, the data sources that tend to be used, and the particular advantages and disadvantages of employing this methodological approach. The paper concludes with key pointers to aid those designing and appraising proposals for conducting case study research, and a checklist to help readers assess the quality of case study reports.

Peer Review reports

Introduction

The case study approach is particularly useful to employ when there is a need to obtain an in-depth appreciation of an issue, event or phenomenon of interest, in its natural real-life context. Our aim in writing this piece is to provide insights into when to consider employing this approach and an overview of key methodological considerations in relation to the design, planning, analysis, interpretation and reporting of case studies.

The illustrative 'grand round', 'case report' and 'case series' have a long tradition in clinical practice and research. Presenting detailed critiques, typically of one or more patients, aims to provide insights into aspects of the clinical case and, in doing so, illustrate broader lessons that may be learnt. In research, the conceptually-related case study approach can be used, for example, to describe in detail a patient's episode of care, explore professional attitudes to and experiences of a new policy initiative or service development or more generally to 'investigate contemporary phenomena within its real-life context' [ 1 ]. Based on our experiences of conducting a range of case studies, we reflect on when to consider using this approach, discuss the key steps involved and illustrate, with examples, some of the practical challenges of attaining an in-depth understanding of a 'case' as an integrated whole. In keeping with previously published work, we acknowledge the importance of theory to underpin the design, selection, conduct and interpretation of case studies[ 2 ]. In so doing, we make passing reference to the different epistemological approaches used in case study research by key theoreticians and methodologists in this field of enquiry.

This paper is structured around the following main questions: What is a case study? What are case studies used for? How are case studies conducted? What are the potential pitfalls and how can these be avoided? We draw in particular on four of our own recently published examples of case studies (see Tables 1 , 2 , 3 and 4 ) and those of others to illustrate our discussion[ 3 – 7 ].

What is a case study?

A case study is a research approach that is used to generate an in-depth, multi-faceted understanding of a complex issue in its real-life context. It is an established research design that is used extensively in a wide variety of disciplines, particularly in the social sciences. A case study can be defined in a variety of ways (Table 5 ), the central tenet being the need to explore an event or phenomenon in depth and in its natural context. It is for this reason sometimes referred to as a "naturalistic" design; this is in contrast to an "experimental" design (such as a randomised controlled trial) in which the investigator seeks to exert control over and manipulate the variable(s) of interest.

Stake's work has been particularly influential in defining the case study approach to scientific enquiry. He has helpfully characterised three main types of case study: intrinsic , instrumental and collective [ 8 ]. An intrinsic case study is typically undertaken to learn about a unique phenomenon. The researcher should define the uniqueness of the phenomenon, which distinguishes it from all others. In contrast, the instrumental case study uses a particular case (some of which may be better than others) to gain a broader appreciation of an issue or phenomenon. The collective case study involves studying multiple cases simultaneously or sequentially in an attempt to generate a still broader appreciation of a particular issue.

These are however not necessarily mutually exclusive categories. In the first of our examples (Table 1 ), we undertook an intrinsic case study to investigate the issue of recruitment of minority ethnic people into the specific context of asthma research studies, but it developed into a instrumental case study through seeking to understand the issue of recruitment of these marginalised populations more generally, generating a number of the findings that are potentially transferable to other disease contexts[ 3 ]. In contrast, the other three examples (see Tables 2 , 3 and 4 ) employed collective case study designs to study the introduction of workforce reconfiguration in primary care, the implementation of electronic health records into hospitals, and to understand the ways in which healthcare students learn about patient safety considerations[ 4 – 6 ]. Although our study focusing on the introduction of General Practitioners with Specialist Interests (Table 2 ) was explicitly collective in design (four contrasting primary care organisations were studied), is was also instrumental in that this particular professional group was studied as an exemplar of the more general phenomenon of workforce redesign[ 4 ].

What are case studies used for?

According to Yin, case studies can be used to explain, describe or explore events or phenomena in the everyday contexts in which they occur[ 1 ]. These can, for example, help to understand and explain causal links and pathways resulting from a new policy initiative or service development (see Tables 2 and 3 , for example)[ 1 ]. In contrast to experimental designs, which seek to test a specific hypothesis through deliberately manipulating the environment (like, for example, in a randomised controlled trial giving a new drug to randomly selected individuals and then comparing outcomes with controls),[ 9 ] the case study approach lends itself well to capturing information on more explanatory ' how ', 'what' and ' why ' questions, such as ' how is the intervention being implemented and received on the ground?'. The case study approach can offer additional insights into what gaps exist in its delivery or why one implementation strategy might be chosen over another. This in turn can help develop or refine theory, as shown in our study of the teaching of patient safety in undergraduate curricula (Table 4 )[ 6 , 10 ]. Key questions to consider when selecting the most appropriate study design are whether it is desirable or indeed possible to undertake a formal experimental investigation in which individuals and/or organisations are allocated to an intervention or control arm? Or whether the wish is to obtain a more naturalistic understanding of an issue? The former is ideally studied using a controlled experimental design, whereas the latter is more appropriately studied using a case study design.

Case studies may be approached in different ways depending on the epistemological standpoint of the researcher, that is, whether they take a critical (questioning one's own and others' assumptions), interpretivist (trying to understand individual and shared social meanings) or positivist approach (orientating towards the criteria of natural sciences, such as focusing on generalisability considerations) (Table 6 ). Whilst such a schema can be conceptually helpful, it may be appropriate to draw on more than one approach in any case study, particularly in the context of conducting health services research. Doolin has, for example, noted that in the context of undertaking interpretative case studies, researchers can usefully draw on a critical, reflective perspective which seeks to take into account the wider social and political environment that has shaped the case[ 11 ].

How are case studies conducted?

Here, we focus on the main stages of research activity when planning and undertaking a case study; the crucial stages are: defining the case; selecting the case(s); collecting and analysing the data; interpreting data; and reporting the findings.

Defining the case

Carefully formulated research question(s), informed by the existing literature and a prior appreciation of the theoretical issues and setting(s), are all important in appropriately and succinctly defining the case[ 8 , 12 ]. Crucially, each case should have a pre-defined boundary which clarifies the nature and time period covered by the case study (i.e. its scope, beginning and end), the relevant social group, organisation or geographical area of interest to the investigator, the types of evidence to be collected, and the priorities for data collection and analysis (see Table 7 )[ 1 ]. A theory driven approach to defining the case may help generate knowledge that is potentially transferable to a range of clinical contexts and behaviours; using theory is also likely to result in a more informed appreciation of, for example, how and why interventions have succeeded or failed[ 13 ].

For example, in our evaluation of the introduction of electronic health records in English hospitals (Table 3 ), we defined our cases as the NHS Trusts that were receiving the new technology[ 5 ]. Our focus was on how the technology was being implemented. However, if the primary research interest had been on the social and organisational dimensions of implementation, we might have defined our case differently as a grouping of healthcare professionals (e.g. doctors and/or nurses). The precise beginning and end of the case may however prove difficult to define. Pursuing this same example, when does the process of implementation and adoption of an electronic health record system really begin or end? Such judgements will inevitably be influenced by a range of factors, including the research question, theory of interest, the scope and richness of the gathered data and the resources available to the research team.

Selecting the case(s)

The decision on how to select the case(s) to study is a very important one that merits some reflection. In an intrinsic case study, the case is selected on its own merits[ 8 ]. The case is selected not because it is representative of other cases, but because of its uniqueness, which is of genuine interest to the researchers. This was, for example, the case in our study of the recruitment of minority ethnic participants into asthma research (Table 1 ) as our earlier work had demonstrated the marginalisation of minority ethnic people with asthma, despite evidence of disproportionate asthma morbidity[ 14 , 15 ]. In another example of an intrinsic case study, Hellstrom et al.[ 16 ] studied an elderly married couple living with dementia to explore how dementia had impacted on their understanding of home, their everyday life and their relationships.

For an instrumental case study, selecting a "typical" case can work well[ 8 ]. In contrast to the intrinsic case study, the particular case which is chosen is of less importance than selecting a case that allows the researcher to investigate an issue or phenomenon. For example, in order to gain an understanding of doctors' responses to health policy initiatives, Som undertook an instrumental case study interviewing clinicians who had a range of responsibilities for clinical governance in one NHS acute hospital trust[ 17 ]. Sampling a "deviant" or "atypical" case may however prove even more informative, potentially enabling the researcher to identify causal processes, generate hypotheses and develop theory.

In collective or multiple case studies, a number of cases are carefully selected. This offers the advantage of allowing comparisons to be made across several cases and/or replication. Choosing a "typical" case may enable the findings to be generalised to theory (i.e. analytical generalisation) or to test theory by replicating the findings in a second or even a third case (i.e. replication logic)[ 1 ]. Yin suggests two or three literal replications (i.e. predicting similar results) if the theory is straightforward and five or more if the theory is more subtle. However, critics might argue that selecting 'cases' in this way is insufficiently reflexive and ill-suited to the complexities of contemporary healthcare organisations.

The selected case study site(s) should allow the research team access to the group of individuals, the organisation, the processes or whatever else constitutes the chosen unit of analysis for the study. Access is therefore a central consideration; the researcher needs to come to know the case study site(s) well and to work cooperatively with them. Selected cases need to be not only interesting but also hospitable to the inquiry [ 8 ] if they are to be informative and answer the research question(s). Case study sites may also be pre-selected for the researcher, with decisions being influenced by key stakeholders. For example, our selection of case study sites in the evaluation of the implementation and adoption of electronic health record systems (see Table 3 ) was heavily influenced by NHS Connecting for Health, the government agency that was responsible for overseeing the National Programme for Information Technology (NPfIT)[ 5 ]. This prominent stakeholder had already selected the NHS sites (through a competitive bidding process) to be early adopters of the electronic health record systems and had negotiated contracts that detailed the deployment timelines.

It is also important to consider in advance the likely burden and risks associated with participation for those who (or the site(s) which) comprise the case study. Of particular importance is the obligation for the researcher to think through the ethical implications of the study (e.g. the risk of inadvertently breaching anonymity or confidentiality) and to ensure that potential participants/participating sites are provided with sufficient information to make an informed choice about joining the study. The outcome of providing this information might be that the emotive burden associated with participation, or the organisational disruption associated with supporting the fieldwork, is considered so high that the individuals or sites decide against participation.

In our example of evaluating implementations of electronic health record systems, given the restricted number of early adopter sites available to us, we sought purposively to select a diverse range of implementation cases among those that were available[ 5 ]. We chose a mixture of teaching, non-teaching and Foundation Trust hospitals, and examples of each of the three electronic health record systems procured centrally by the NPfIT. At one recruited site, it quickly became apparent that access was problematic because of competing demands on that organisation. Recognising the importance of full access and co-operative working for generating rich data, the research team decided not to pursue work at that site and instead to focus on other recruited sites.

Collecting the data

In order to develop a thorough understanding of the case, the case study approach usually involves the collection of multiple sources of evidence, using a range of quantitative (e.g. questionnaires, audits and analysis of routinely collected healthcare data) and more commonly qualitative techniques (e.g. interviews, focus groups and observations). The use of multiple sources of data (data triangulation) has been advocated as a way of increasing the internal validity of a study (i.e. the extent to which the method is appropriate to answer the research question)[ 8 , 18 – 21 ]. An underlying assumption is that data collected in different ways should lead to similar conclusions, and approaching the same issue from different angles can help develop a holistic picture of the phenomenon (Table 2 )[ 4 ].

Brazier and colleagues used a mixed-methods case study approach to investigate the impact of a cancer care programme[ 22 ]. Here, quantitative measures were collected with questionnaires before, and five months after, the start of the intervention which did not yield any statistically significant results. Qualitative interviews with patients however helped provide an insight into potentially beneficial process-related aspects of the programme, such as greater, perceived patient involvement in care. The authors reported how this case study approach provided a number of contextual factors likely to influence the effectiveness of the intervention and which were not likely to have been obtained from quantitative methods alone.

In collective or multiple case studies, data collection needs to be flexible enough to allow a detailed description of each individual case to be developed (e.g. the nature of different cancer care programmes), before considering the emerging similarities and differences in cross-case comparisons (e.g. to explore why one programme is more effective than another). It is important that data sources from different cases are, where possible, broadly comparable for this purpose even though they may vary in nature and depth.

Analysing, interpreting and reporting case studies

Making sense and offering a coherent interpretation of the typically disparate sources of data (whether qualitative alone or together with quantitative) is far from straightforward. Repeated reviewing and sorting of the voluminous and detail-rich data are integral to the process of analysis. In collective case studies, it is helpful to analyse data relating to the individual component cases first, before making comparisons across cases. Attention needs to be paid to variations within each case and, where relevant, the relationship between different causes, effects and outcomes[ 23 ]. Data will need to be organised and coded to allow the key issues, both derived from the literature and emerging from the dataset, to be easily retrieved at a later stage. An initial coding frame can help capture these issues and can be applied systematically to the whole dataset with the aid of a qualitative data analysis software package.

The Framework approach is a practical approach, comprising of five stages (familiarisation; identifying a thematic framework; indexing; charting; mapping and interpretation) , to managing and analysing large datasets particularly if time is limited, as was the case in our study of recruitment of South Asians into asthma research (Table 1 )[ 3 , 24 ]. Theoretical frameworks may also play an important role in integrating different sources of data and examining emerging themes. For example, we drew on a socio-technical framework to help explain the connections between different elements - technology; people; and the organisational settings within which they worked - in our study of the introduction of electronic health record systems (Table 3 )[ 5 ]. Our study of patient safety in undergraduate curricula drew on an evaluation-based approach to design and analysis, which emphasised the importance of the academic, organisational and practice contexts through which students learn (Table 4 )[ 6 ].

Case study findings can have implications both for theory development and theory testing. They may establish, strengthen or weaken historical explanations of a case and, in certain circumstances, allow theoretical (as opposed to statistical) generalisation beyond the particular cases studied[ 12 ]. These theoretical lenses should not, however, constitute a strait-jacket and the cases should not be "forced to fit" the particular theoretical framework that is being employed.

When reporting findings, it is important to provide the reader with enough contextual information to understand the processes that were followed and how the conclusions were reached. In a collective case study, researchers may choose to present the findings from individual cases separately before amalgamating across cases. Care must be taken to ensure the anonymity of both case sites and individual participants (if agreed in advance) by allocating appropriate codes or withholding descriptors. In the example given in Table 3 , we decided against providing detailed information on the NHS sites and individual participants in order to avoid the risk of inadvertent disclosure of identities[ 5 , 25 ].

What are the potential pitfalls and how can these be avoided?

The case study approach is, as with all research, not without its limitations. When investigating the formal and informal ways undergraduate students learn about patient safety (Table 4 ), for example, we rapidly accumulated a large quantity of data. The volume of data, together with the time restrictions in place, impacted on the depth of analysis that was possible within the available resources. This highlights a more general point of the importance of avoiding the temptation to collect as much data as possible; adequate time also needs to be set aside for data analysis and interpretation of what are often highly complex datasets.

Case study research has sometimes been criticised for lacking scientific rigour and providing little basis for generalisation (i.e. producing findings that may be transferable to other settings)[ 1 ]. There are several ways to address these concerns, including: the use of theoretical sampling (i.e. drawing on a particular conceptual framework); respondent validation (i.e. participants checking emerging findings and the researcher's interpretation, and providing an opinion as to whether they feel these are accurate); and transparency throughout the research process (see Table 8 )[ 8 , 18 – 21 , 23 , 26 ]. Transparency can be achieved by describing in detail the steps involved in case selection, data collection, the reasons for the particular methods chosen, and the researcher's background and level of involvement (i.e. being explicit about how the researcher has influenced data collection and interpretation). Seeking potential, alternative explanations, and being explicit about how interpretations and conclusions were reached, help readers to judge the trustworthiness of the case study report. Stake provides a critique checklist for a case study report (Table 9 )[ 8 ].

Conclusions

The case study approach allows, amongst other things, critical events, interventions, policy developments and programme-based service reforms to be studied in detail in a real-life context. It should therefore be considered when an experimental design is either inappropriate to answer the research questions posed or impossible to undertake. Considering the frequency with which implementations of innovations are now taking place in healthcare settings and how well the case study approach lends itself to in-depth, complex health service research, we believe this approach should be more widely considered by researchers. Though inherently challenging, the research case study can, if carefully conceptualised and thoughtfully undertaken and reported, yield powerful insights into many important aspects of health and healthcare delivery.

Yin RK: Case study research, design and method. 2009, London: Sage Publications Ltd., 4

Google Scholar  

Keen J, Packwood T: Qualitative research; case study evaluation. BMJ. 1995, 311: 444-446.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Sheikh A, Halani L, Bhopal R, Netuveli G, Partridge M, Car J, et al: Facilitating the Recruitment of Minority Ethnic People into Research: Qualitative Case Study of South Asians and Asthma. PLoS Med. 2009, 6 (10): 1-11.

Article   Google Scholar  

Pinnock H, Huby G, Powell A, Kielmann T, Price D, Williams S, et al: The process of planning, development and implementation of a General Practitioner with a Special Interest service in Primary Care Organisations in England and Wales: a comparative prospective case study. Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO). 2008, [ http://www.sdo.nihr.ac.uk/files/project/99-final-report.pdf ]

Robertson A, Cresswell K, Takian A, Petrakaki D, Crowe S, Cornford T, et al: Prospective evaluation of the implementation and adoption of NHS Connecting for Health's national electronic health record in secondary care in England: interim findings. BMJ. 2010, 41: c4564-

Pearson P, Steven A, Howe A, Sheikh A, Ashcroft D, Smith P, the Patient Safety Education Study Group: Learning about patient safety: organisational context and culture in the education of healthcare professionals. J Health Serv Res Policy. 2010, 15: 4-10. 10.1258/jhsrp.2009.009052.

Article   PubMed   Google Scholar  

van Harten WH, Casparie TF, Fisscher OA: The evaluation of the introduction of a quality management system: a process-oriented case study in a large rehabilitation hospital. Health Policy. 2002, 60 (1): 17-37. 10.1016/S0168-8510(01)00187-7.

Stake RE: The art of case study research. 1995, London: Sage Publications Ltd.

Sheikh A, Smeeth L, Ashcroft R: Randomised controlled trials in primary care: scope and application. Br J Gen Pract. 2002, 52 (482): 746-51.

PubMed   PubMed Central   Google Scholar  

King G, Keohane R, Verba S: Designing Social Inquiry. 1996, Princeton: Princeton University Press

Doolin B: Information technology as disciplinary technology: being critical in interpretative research on information systems. Journal of Information Technology. 1998, 13: 301-311. 10.1057/jit.1998.8.

George AL, Bennett A: Case studies and theory development in the social sciences. 2005, Cambridge, MA: MIT Press

Eccles M, the Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG): Designing theoretically-informed implementation interventions. Implementation Science. 2006, 1: 1-8. 10.1186/1748-5908-1-1.

Article   PubMed Central   Google Scholar  

Netuveli G, Hurwitz B, Levy M, Fletcher M, Barnes G, Durham SR, Sheikh A: Ethnic variations in UK asthma frequency, morbidity, and health-service use: a systematic review and meta-analysis. Lancet. 2005, 365 (9456): 312-7.

Sheikh A, Panesar SS, Lasserson T, Netuveli G: Recruitment of ethnic minorities to asthma studies. Thorax. 2004, 59 (7): 634-

CAS   PubMed   PubMed Central   Google Scholar  

Hellström I, Nolan M, Lundh U: 'We do things together': A case study of 'couplehood' in dementia. Dementia. 2005, 4: 7-22. 10.1177/1471301205049188.

Som CV: Nothing seems to have changed, nothing seems to be changing and perhaps nothing will change in the NHS: doctors' response to clinical governance. International Journal of Public Sector Management. 2005, 18: 463-477. 10.1108/09513550510608903.

Lincoln Y, Guba E: Naturalistic inquiry. 1985, Newbury Park: Sage Publications

Barbour RS: Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?. BMJ. 2001, 322: 1115-1117. 10.1136/bmj.322.7294.1115.

Mays N, Pope C: Qualitative research in health care: Assessing quality in qualitative research. BMJ. 2000, 320: 50-52. 10.1136/bmj.320.7226.50.

Mason J: Qualitative researching. 2002, London: Sage

Brazier A, Cooke K, Moravan V: Using Mixed Methods for Evaluating an Integrative Approach to Cancer Care: A Case Study. Integr Cancer Ther. 2008, 7: 5-17. 10.1177/1534735407313395.

Miles MB, Huberman M: Qualitative data analysis: an expanded sourcebook. 1994, CA: Sage Publications Inc., 2

Pope C, Ziebland S, Mays N: Analysing qualitative data. Qualitative research in health care. BMJ. 2000, 320: 114-116. 10.1136/bmj.320.7227.114.

Cresswell KM, Worth A, Sheikh A: Actor-Network Theory and its role in understanding the implementation of information technology developments in healthcare. BMC Med Inform Decis Mak. 2010, 10 (1): 67-10.1186/1472-6947-10-67.

Article   PubMed   PubMed Central   Google Scholar  

Malterud K: Qualitative research: standards, challenges, and guidelines. Lancet. 2001, 358: 483-488. 10.1016/S0140-6736(01)05627-6.

Article   CAS   PubMed   Google Scholar  

Yin R: Case study research: design and methods. 1994, Thousand Oaks, CA: Sage Publishing, 2

Yin R: Enhancing the quality of case studies in health services research. Health Serv Res. 1999, 34: 1209-1224.

Green J, Thorogood N: Qualitative methods for health research. 2009, Los Angeles: Sage, 2

Howcroft D, Trauth E: Handbook of Critical Information Systems Research, Theory and Application. 2005, Cheltenham, UK: Northampton, MA, USA: Edward Elgar

Book   Google Scholar  

Blakie N: Approaches to Social Enquiry. 1993, Cambridge: Polity Press

Doolin B: Power and resistance in the implementation of a medical management information system. Info Systems J. 2004, 14: 343-362. 10.1111/j.1365-2575.2004.00176.x.

Bloomfield BP, Best A: Management consultants: systems development, power and the translation of problems. Sociological Review. 1992, 40: 533-560.

Shanks G, Parr A: Positivist, single case study research in information systems: A critical analysis. Proceedings of the European Conference on Information Systems. 2003, Naples

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/11/100/prepub

Download references

Acknowledgements

We are grateful to the participants and colleagues who contributed to the individual case studies that we have drawn on. This work received no direct funding, but it has been informed by projects funded by Asthma UK, the NHS Service Delivery Organisation, NHS Connecting for Health Evaluation Programme, and Patient Safety Research Portfolio. We would also like to thank the expert reviewers for their insightful and constructive feedback. Our thanks are also due to Dr. Allison Worth who commented on an earlier draft of this manuscript.

Author information

Authors and affiliations.

Division of Primary Care, The University of Nottingham, Nottingham, UK

Sarah Crowe & Anthony Avery

Centre for Population Health Sciences, The University of Edinburgh, Edinburgh, UK

Kathrin Cresswell, Ann Robertson & Aziz Sheikh

School of Health in Social Science, The University of Edinburgh, Edinburgh, UK

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sarah Crowe .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors' contributions

AS conceived this article. SC, KC and AR wrote this paper with GH, AA and AS all commenting on various drafts. SC and AS are guarantors.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Crowe, S., Cresswell, K., Robertson, A. et al. The case study approach. BMC Med Res Methodol 11 , 100 (2011). https://doi.org/10.1186/1471-2288-11-100

Download citation

Received : 29 November 2010

Accepted : 27 June 2011

Published : 27 June 2011

DOI : https://doi.org/10.1186/1471-2288-11-100

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Case Study Approach
• Electronic Health Record System
• Case Study Design
• Case Study Site
• Case Study Report

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Help | Advanced Search

Computer Science > Computation and Language

Title: fine grained human evaluation for english-to-chinese machine translation: a case study on scientific text.

Abstract: Recent research suggests that neural machine translation (MT) in the news domain has reached human-level performance, but for other professional domains, it is far below the level. In this paper, we conduct a fine-grained systematic human evaluation for four widely used Chinese-English NMT systems on scientific abstracts which are collected from published journals and books. Our human evaluation results show that all the systems return with more than 10\% error rates on average, which requires much post editing effort for real academic use. Furthermore, we categorize six main error types and and provide some real examples. Our findings emphasise the needs that research attention in the MT community should be shifted from short text generic translation to professional machine translation and build large scale bilingual corpus for these specific domains.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

DBLP - CS Bibliography

Bibtex formatted citation.

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 16 May 2024

Finding the truth in science

• Elisabeth M. Bik 1  

Nature Reviews Cancer ( 2024 ) Cite this article

1868 Accesses

182 Altmetric

Metrics details

In a ‘publish or perish’ culture, some scientists may resort to questionable research practices or even fraud. Scientific paper mills and artificial intelligence increasingly threaten the pursuit of truth in science. Structural changes, including heightened scrutiny of papers and authorship and better funding, are needed to ensure scientific integrity.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 print issues and online access

195,33 € per year

only 16,28 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Bik, E. M., Casadevall, A. & Fang, F. C. The prevalence of inappropriate image duplication in biomedical research publications. mBio 7 , https://doi.org/10.1128/mbio.00809-16 (2016).

Van Noorden, R. How big is science’s fake-paper problem? Nature 623 , 466–467 (2023).

Article   PubMed   Google Scholar  

Byrne, J. A. et al. Protection of the human gene research literature from contract cheating organizations known as research paper mills. Nucleic Acids Res. 50 , 12058–12070 (2022).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Sanderson, K. Science’s fake-paper problem: high-profile effort will tackle paper mills. Nature 626 , 17–18 (2024).

Article   CAS   PubMed   Google Scholar  

Banks, M. How fishy email addresses tipped off a sleuth to a paper mill. Retraction Watch https://go.nature.com/44jQcqt (6 February 2023).

Else, H. What makes an undercover science sleuth tick? Fake-paper detective speaks out. Nature https://www.nature.com/articles/d41586-022-02099-8 (4 August 2022).

Larousserie, D., G. Cabanac, le Sisyphe de la dépollution de la science. Le Monde https://go.nature.com/3UEjM6C (22 April 2024).

Oza, A. AI beats human sleuth at finding problematic images in research papers. Nature https://www.nature.com/articles/d41586-023-02920-y (3 October 2023).

Oransky, I. How to find evidence of paper mills using peer review comments. Retraction Watch https://go.nature.com/3we9X6e (21 February 2022).

Kincaid, E. Meet a sleuth whose work has resulted in more than 850 retractions. Retraction Watch https://go.nature.com/4bhPKLD (25 October 2022).

Montanari, S. Retraction, She Wrote: Dorothy Bishop’s life after research. The Transmitter https://go.nature.com/3UpkJ1k (12 June 2023).

Piper, K. Is it defamation to point out scientific research fraud? Vox https://go.nature.com/4biP09a (9 August 2023).

Download references

Author information

Authors and affiliations.

Harbers Bik, San Francisco, CA, USA

Elisabeth M. Bik

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Elisabeth M. Bik .

Ethics declarations

Competing interests.

E.M.B. is an independent consultant who has received payments from scientific publishers and institutions to investigate particular cases of research misconduct, or for giving talks and workshops about research integrity and misconduct. In addition, she receives donations to support her work through Patreon.com.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Bik, E.M. Finding the truth in science. Nat Rev Cancer (2024). https://doi.org/10.1038/s41568-024-00702-w

Download citation

Published : 16 May 2024

DOI : https://doi.org/10.1038/s41568-024-00702-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Study protocol
• Open access
• Published: 24 May 2024

The effect of audit and feedback and implementation support on guideline adherence and patient outcomes in cardiac rehabilitation: a study protocol for an open-label cluster-randomized effectiveness-implementation hybrid trial

• Halldóra Ögmundsdóttir Michelsen 1 , 2 ,
• Matthias Lidin 3 , 4 ,
• Maria Bäck 5 , 6 ,
• Therese Scott Duncan 7 ,
• Björn Ekman 1 ,
• Emil Hagström 8 , 9 ,
• Maria Hägglund 7 ,
• Bertil Lindahl 8 , 9 ,
• Mona Schlyter 10 &
• Margrét Leósdóttir 1 , 10  

Implementation Science volume  19 , Article number:  35 ( 2024 ) Cite this article

Metrics details

Providing secondary prevention through structured and comprehensive cardiac rehabilitation programmes to patients after a myocardial infarction (MI) reduces mortality and morbidity and improves health-related quality of life. Cardiac rehabilitation has the highest recommendation in current guidelines. While treatment target attainment rates at Swedish cardiac rehabilitation centres is among the highest in Europe, there are considerable differences in service delivery and variations in patient-level outcomes between centres. In this trial, we aim to study whether centre-level guideline adherence and patient-level outcomes across Swedish cardiac rehabilitation centres can be improved through a) regular audit and feedback of cardiac rehabilitation structure and processes through a national quality registry and b) supporting cardiac rehabilitation centres in implementing guidelines on secondary prevention. Furthermore, we aim to evaluate the implementation process and costs.

The study is an open-label cluster-randomized effectiveness-implementation hybrid trial including all 78 cardiac rehabilitation centres (attending to approximately 10 000 MI patients/year) that report to the SWEDEHEART registry. The centres will be randomized 1:1:1 to three clusters: 1) reporting cardiac rehabilitation structure and process variables to SWEDEHEART every six months (audit intervention) and being offered implementation support to implement guidelines on secondary prevention (implementation support intervention); 2) audit intervention only; or 3) no intervention offered. Baseline cardiac rehabilitation structure and process variables will be collected. The primary outcome is an adherence score measuring centre-level adherence to secondary prevention guidelines. Secondary outcomes include patient-level secondary prevention risk factor goal attainment at one-year after MI and major adverse coronary outcomes for up to five-years post-MI. Implementation outcomes include barriers and facilitators to guideline adherence evaluated using semi-structured focus-group interviews and relevant questionnaires, as well as costs and cost-effectiveness assessed by a comparative health economic evaluation.

Optimizing cardiac rehabilitation centres’ delivery of services to meet standards set in guidelines may lead to improvement in cardiovascular risk factors, including lifestyle factors, and ultimately a decrease in morbidity and mortality after MI.

Trial registration

ClinicalTrials.gov. Identifier: NCT05889416 . Registered 2023-03-23.

Contributions to the literature

Structured implementation support to improve guideline adherence has not previously been assessed within cardiac rehabilitation.

The study is a national open-label cluster-randomized effectiveness-implementation hybrid trial including all cardiac rehabilitation centres that report to a national quality registry.

The aim is to evaluate whether regularly reporting cardiac rehabilitation structure and process variables through a national quality registry and/or offering cardiac rehabilitation centres structured support to implement guidelines on secondary prevention will lead to an increase in centre-level adherence to guidelines on secondary prevention and improve patient-level outcomes.

Cardiovascular risk factor reduction and the fostering of a healthy lifestyle after an acute myocardial infarction (MI) are the most effective interventions to prevent recurrent coronary events [ 1 ]. Administering these interventions via structured and comprehensive cardiac rehabilitation (CR) programmes reduces mortality, morbidity, unplanned hospital admissions, and improves health-related quality of life [ 2 , 3 , 4 ]. CR is a complex intervention, combining the optimal use of cardio-protective medication, exercise training, patient education, and behavioural modification to improve lifestyle, and psychosocial counselling [ 5 , 6 ]. Patient participation in CR after an MI is given the highest recommendation and level of evidence in current guidelines on cardiovascular disease (CVD) prevention [ 1 ]. However, referral rates to CR programmes are generally low and patients´ treatment target attainment is sub-optimal [ 7 ]. While treatment target attainment rates at Swedish CR centres is on average among the highest in Europe, there are considerable differences in CR service delivery between centres [ 8 , 9 , 10 , 11 ] and consequently, large variations in patient-level outcomes between centres [ 12 ].

The SWEDEHEART registry is a nationwide quality registry that records baseline characteristics, treatments, and outcomes of patients with MI admitted to coronary care units in Sweden [ 13 ]. Follow-up data describing secondary preventive patient-level outcomes have been collected in the CR part of the registry (SWEDEHEART-CR) since 2005 [ 14 ]. However, only a handful of variables monitoring centre-level structure and processes are included in the registry.

In 2019, a National Working Group on Secondary Prevention was commissioned by the Swedish Association of Local Authorities and Regions to author National Guidelines on Secondary Prevention for patients with coronary artery disease, aiming to decrease the variation in secondary prevention delivery and outcomes in Sweden. The guidelines were published in February 2022 [ 15 ]. In parallel with the release of CR guidelines, the SWEDEHEART-CR Working Group proposed incorporating variables into the registry to assess the recommended structure and processes outlined in the guidelines [ 16 ].

The Swedish healthcare system is highly decentralized, where overall healthcare policies and guidelines are set by national regulating agencies, and the responsibility for providing and funding services, lies with 21 autonomous regions [ 17 ]. Each regional authority is responsible for implementing the 2022 National Guidelines on Secondary Prevention on a local level. Implementing guidelines is, however, a complex process faced with many challenges, and often revised guidelines result in little or no change in clinical practice [ 18 , 19 ]. To address the disparity between policies and clinical practice, the aims of the study are following:

Primary aim

To prospectively study whether a) audit and feedback of CR structure and processes within the SWEDEHEART registry and b) supporting CR centres in implementing CR guidelines can increase centre-level guideline adherence.

Secondary aims

To cross-sectionally evaluate the association between centre-level adherence to guidelines and patient-level outcomes.

To prospectively study whether audit and feedback of CR structure and processes within the SWEDEHEART-CR registry can improve patient-level outcomes.

To prospectively evaluate whether supporting CR centres in implementing CR guidelines can improve patient-level outcomes.

To qualitatively evaluate barriers and facilitators to guideline implementation.

To evaluate cost and cost-effectiveness of the implementation support.

Study setting and recruitment

The study started in October 2023 and will include all CR centres ( n  = 78) which report to the SWEDEHEART-CR registry. The sole exclusion criterion for CR centres is unwillingness to participate. The inclusion criteria for patients are 1) having a diagnosis of a type 1 MI (caused by atherosclerotic plaque rupture or coronary artery thrombosis) and 2) age 18–79 years at discharge from MI hospitalization. There are no exclusion criteria for patients.

Study design

The study is an open-label cluster-randomized effectiveness-implementation hybrid trial. The effectiveness-implementation hybrid design allows for testing an implementation strategy while observing the intervention´s impact on patient outcomes [ 20 ]. For the implementation support, the Consolidated Framework for Implementation Research (CFIR) model will be used to guide the design [ 21 , 22 ]. Normalization Process Theory (NPT) will be used to guide the exploration of the implementation process [ 20 ].

Randomization

Centres will be randomized 1:1:1 to three clusters (A, B and C). Randomization will be stratified by geographical healthcare district (two districts in each stratum). Randomization will be performed by an independent organisation not involved in the study to avoid site-selection bias.

The interventions

First, baseline structure and process variables will be administered through the SWEDEHEART-CR registry at all CR centres. Second, two clusters will receive one of two interventions (A and B) and one cluster will receive no intervention (control cluster) (C) (Fig.  1 ):

Reprting CR structure and process variables through the registry every six months and being offered structured implementation support (audit intervention + implementation support intervention).

Reporting CR structure and process variables through the registry every six months but no structured implementation support being offered (audit intervention).

No reporting of CR structure and process variables, and no structured implementation support offered (control).

Overview, process and timeline of the study

The audit intervention

The audit intervention (clusters A and B) involves, on centre-level, reporting 30 variables on CR structure and processes, measuring adherence to the National Guidelines, to the SWEDEHEART registry. The complete list of CR structure and process variables is available in Additional file 1.

The implementation support intervention

For the implementation support intervention (cluster A) facilitators from the research team will provide hands-on support and guidance to the CR team as they work to implement changes [ 23 ]. Facilitators are physicians and nurses experienced in the field of cardiac rehabilitation. In the recruitment step, the centre directors or key stakeholders will be contacted by the principal investigator to offer study participation by means of an e-mail. Upon acceptance to participate, the time to start the intervention will be determined, a timeline established, and the CR centre director will be asked to allocate time in the schedule for the relevant CR staff to work on the study. The implementation support will be conducted in four steps: 1) evaluation of centre practice, 2) identification of areas in need of improvement, 3) implementation of change, and 4) follow-up (Table  1 ). The implementation objects are work routines or CR programme components listed in Additional file 1 identified as sub-optimally implemented at the respective CR centre. Implementation strategies (defined by results from the Expert Recommendations for Implementing Change (ERIC) project) used will be identifying and preparing champions, identify barriers and facilitators, facilitation, distribution of educational materials, conducting educational meetings, organizing clinician implementation team meetings, and, providing local and technical assistance [ 23 , 24 , 25 , 26 , 27 , 28 ].

The implementation support will be provided during a two-day visit from the facilitators, two follow-up phone calls during the first month after the visit, up to three educational meetings (number as requested by centre personnel), and a digital follow-up meeting 4 months after the initial visit. Additional phone or video calls, and emails will be provided as needed.

Primary effectiveness outcome

The primary endpoint is an “adherence score”, measuring adherence to the National Guidelines on Secondary Prevention [ 15 ]. The adherence score is partly derived from the 30 variables capturing guideline-directed CR structure and processes incorporated into the SWEDEHEART-CR registry in October 2023. Permissible values are yes/partly/no/unknown. Additionally, the adherence score will include nine process variables that measure a) patient attendance in various CR components (number of patients attending a CR programme component divided by the number of patients eligible for participation), and b) time between MI hospitalization discharge and start of different CR programme components, already audited in SWEDEHEART:

Attendance in CR components

Proportion of patients attending an initial CR assessment (nurse visit).

Proportion of patients attending an individual visit to a physiotherapist after discharge before starting an exercise-based CR (EBCR) programme (a pre-exercise screening visit).

Proportion of patients completing a 3-month EBCR programme.

Proportion of patients attending an individual close-out visit to a physiotherapist after completing an EBCR programme (a post-exercise assessment visit).

Proportion of patients attending a patient education programme.

Proportion of patients attending a close-out CR visit with a nurse at one-year after MI.

Time (days) to start of different components of CR

Time from hospital discharge to initial CR assessment (nurse visit).

Time from hospital discharge to the pre-exercise screening visit to a physiotherapist.

Time from pre-exercise screening visit to start of EBCR programme.

The responses will be pooled at each site into an adherence score for each CR centre. The contribution of each variable to the score will be weighed depending on the importance of each measurement, based on level of recommendation in European guidelines [ 1 ].

Secondary effectiveness outcomes

Table  2 displays secondary outcomes, encompassing both short- and long-term patient outcomes and implementation outcomes. Implementation outcomes will be based on responses gathered through the customized Normalisation Measure Development (NoMAD) questionnaire [ 29 ] and insights obtained from focus group interviews, as well as a cost-effectiveness analysis.

Sample size calculations

As the number of CR centres is fixed, sample size calculations have been performed to estimate the difference in mean adherence score the study will be able to reveal. For this purpose, in a feasibility analysis the CR structure and process variables were collected from 8 CR centres of different sizes and geographical locations. The mean adherence score was 30.8 (standard deviation [± 2.4]) out of a maximum available score of 39. For the audit intervention, given the following presumptions:

23 out of possible 26 CR centres in each group (minimal drop-out is anticipated).

Power of 80%.

An alpha value of 0.05.

The study will have the power to identify a difference of ± 2.0 in adherence score between centres in cluster B (randomized to having variables on CR structure and processes incorporated in the SWEDEHEART registry) and cluster C (no new variables incorporated).

For the implementation support intervention, only centres in the lower 2 tertiles of the adherence score at baseline (approximately 17 centres) will be offered implementation support. Given the following presumptions:

14 out of possible 17 centres in each group (minimal drop-out is anticipated).

The study will have the power to identify a difference of ± 2.5 in adherence score between centres in cluster A (randomized to receiving implementation support) and cluster B (no implementation support).

The timeline

The audit intervention started in October 2023 and will continue for 3 years. An interim analysis will be conducted two years after the start of the intervention. If the interim analysis shows the primary endpoint to be met (a difference of at least ± 2.0 in adherence score) the intervention will be terminated. Otherwise, the intervention will be continued until October 2026 and thereby uphold.

The implementation support intervention will start Q2 2024. Centres randomized to the implementation intervention will receive implementation support consecutively over a period of 18 months. The order in which centres will receive implementation support will depend on the centres’ possibilities and the research team´s capacity. The study outline is displayed in Table  3 .

Data analysis

In the primary outcome analysis of the audit intervention, CR centres in clusters B and C will be compared. All CR centres that have responded to the CR structure and process variables will be included in the analysis. For the primary outcome analysis of the implementation support intervention CR centres in clusters A and B will be compared. All CR centres that have i) provided answers to the structure and process variables and that ii) have scores in the lower two tertiles of the adherence score at baseline will be compared. Our analyses will assume intention-to-treat principles by treating intervention assignment as randomized regardless of whether the intervention had uptake within the practice. If not all centres in cluster A accept implementation assistance, a per-protocol analysis will be performed including only those centres in study group A that accept implementation assistance.

For the secondary analyses on patient outcomes, all patients that attended at least two follow-up visits within CR will be included. For the outcome analysis of the audit intervention patients followed at CR centres in clusters B and C will be compared. For the outcome analysis of the implementation support intervention patients followed at CR centres in clusters A and B will be compared. A per-protocol analysis will be performed including only patients belonging to CR centres in cluster A that accept implementation assistance.

Quantitative data

For baseline characteristics descriptive statistics will be used (means +/-standard deviation, medians [quartile 1, quartile 3], proportions [%] and ranges). For the primary outcome analysis, given randomized treatment assignment, total adherence scores at end of follow-up as well as change in scores between baseline and follow-up will be compared using linear regression analysis. In the case of unequal randomization concerning CR centre size (small < 75 patients/year, medium 75–150 patients/year or large > 150 patients per year), geography (6 geographical districts in Sweden, two in each stratum) or the centres belonging to a university hospital (yes/no), adjusted multivariable analysis will be performed. Results will be reported as relative treatment effects (odds ratios) with 95% confidence intervals. For secondary outcome analysis on short-term patient outcomes, the same statistical methods will be used, applying linear (continuous) or logistic (binary) regression analyses. For long-term outcomes (major adverse cardiovascular events [MACE] and total mortality) Cox proportional hazards regression models will be performed, reporting hazard ratios with 95% confidence intervals. In case of missing data, imputation will be considered. For all quantitative analyses a two-sided test of statistical significance will be used with an alpha level of 0.05.

Qualitative data

For the implementation support intervention (cluster A), semi structured focus groups interviews will be conducted on each intervention site. A focus group will include 3–4 members of the CR team and a facilitator with experience of qualitative interviews will conduct the interviews. The interviews will assess the perception of contributing organizational, contextual, and structural factors that impact successful/unsuccessful uptake of the guidelines. Depending on number of centres accepting participation, approximately 10–12 interviews will be performed (pre- and post-implementation support intervention). Interviews will be conducted face-to-face or virtually via videoconferencing depending on the CR team´s and the facilitator´s availability and preference. To ensure anonymity, any identifiable information shared by participants will be dissociated from individual identities before analysis. The interview questions will be designed using the CFIR interview guide [ 30 ]. The interviews will be audio-recorded, transcribed verbatim, and analysed with descriptive, qualitative content analysis with an inductive and manifest approach according to Graneheim and Lundman [ 31 , 32 ]. CFIR definitions and coding guidelines will be used to assist with coding of qualitative data [ 26 ].

Cost and cost-effectiveness analysis

A comparative health economic evaluation of the implementation and the usual care models will be conducted. Generally, an economic evaluation serves to provide decision makers with relevant information about the value for money as to alternative treatment models. The economic evaluation will adopt generally accepted methods for such types of analyses, including the estimation of all relevant costs and benefits from both a health system and a societal perspective. Data on both direct and indirect costs of the two models will be collected through a survey where all centres will fill out the resources needed to ensure the implementation of the intervention.

Based on the cost estimates and the effect of the implementation assistance on patient outcomes, the economic evaluation will then be able to conduct a cost-effectiveness analysis (CEA). The CEA responds to the key policy question of the cost of the measured effects. Such information contributes to making informed priority decisions under fixed budget constraints in healthcare services. In addition, the incremental cost-effectiveness ratio (ICER) will be computed to assess the added costs relative to the added effects (benefits) of the intervention model compared with usual care:

where t = treatment option and uc = usual care option. The ICER shows the additional (incremental) costs of implementing the treatment model compared with the usual care option. Consequently, the ICER responds to the related policy question of how much more will be achieved for how much more resources (costs) compared with the current situation. The effect measures include those identified above under Study objectives (Sect. 4): guidelines adherence and patient-level outcomes.

Ethical considerations and withdrawal criteria

The study will be performed in compliance with the study protocol, the Declaration of Helsinki, and current national and international regulations governing this clinical trial. The study has been approved by the Swedish Ethical Review Authority (Registration number: 2023-03217-01) and is registered at ClinicalTrials.gov (identifier: NCT05889416).

The SWEDEHEART registry is sanctioned by Swedish law, stating that all patients are informed of their inclusion and their right to opt-out and have their data erased at any time without a specific reason [ 13 ]. Opt-out is extremely rare, counting fewer than ten cases per year.

All centres report data to the SWEDEHEART registry on a voluntary basis. For the audit intervention an opt-out approach will be applied, i.e., CR centres not willing to provide answers to the new variables will be asked to convey this to the registry. Otherwise, if they submit answers to the new variables, they will be included in the analysis.

For the implementation support intervention, a perquisite for participation is a verbal consent from a centre director or key stakeholder followed by a signed Letter of Intent from the CR centre director.

Data protection

The SWEDEHEART registry data is collected through an interactive web-based IT-platform, developed and maintained by Uppsala Clinical Research centres (UCR), Uppsala, Sweden. The data is electronically transferred to UCR in encrypted format and stored on a central server.

To ensure correct data matching and analysis, centre-level data (structure and process variables) will be requested in an identifiable form (i.e., name of CR centre is linked to data). All patient data from the SWEDEHEART registry and other national registries is, however, delivered pseudonymized to researchers, only containing a study identification number for each patient. As patient data contains information on at which centre the patients had their follow-up, matching to centre-level data will be done by CR centre. A patient-level identification key (study identification number linked to personal identification number) will be stored at the National Board of Health and Welfare, to allow for delivery of long-term outcome data – in the case of this study for up to 5 years.

All electronic study data delivered to the research team will be stored in a locked data storage requiring double identification for access. Only members of the research team will have access to data. Data processing will be performed in accordance with the provisions of the General Data Protection Regulation (GDPR) and other relevant legislation. No data will be shared outside of Sweden.

Referral rates to CR programmes after MI are generally low, and patients’ achievement of secondary preventive treatment targets is sub-optimal [ 7 ]. Despite well-defined frameworks for optimal CR in modern cardiology, there is considerable heterogeneity in the delivery of CR services across programmes and in patient outcomes [ 6 , 9 , 10 , 12 ].

Quality registries offer unique possibilities to assess and compare the quality of care for patients. Monitoring the quality of healthcare is also crucial to reveal discrepancies between evidence-based recommended treatments and the actual care provided in clinical practice [ 33 ]. The audit and feedback process provided by registries has also been suggested to stimulate a question-behaviour effect among healthcare providers, even though scientific evidence is sparse [ 34 , 35 ].

With the aim to decrease disparities in CR service delivery on a national level, Swedish Guidelines on Secondary Prevention were recently published. However, passively publishing guidelines is generally ineffective and leads to little change in clinical practice [ 19 , 36 ]. If CR centres can optimize their delivery of services to meet standards set in guidelines, we may improve risk factor management and lifestyle, and decrease morbidity and mortality post-MI. This study has the potential to reveal barriers and facilitating factors that affect adoption to guidelines. Identifying barriers and facilitators opens opportunities to develop effective strategies and interventions to overcome them. This, in turn, may lead to a reduction in disparities between different centres promoting health equity.

Strengths and limitations

A strength of our study is that patient-level data will be retrieved from coherent and consistent quality registries with national representability. On centre-level, the research team has a vast collegial network in Sweden, which is anticipated to aid in cultivating trust and engagement in the study. As for all implementation interventions, possible recruitment and retention barriers might be encountered, with possible lack of local leadership support and resistance to change in practice. At the same time, the facilitators will closely collaborate with CR teams to evaluate potential opportunities and obstacles. They will provide support and aim to establish trusting relationships with members of the teams. In the event of disruptions, their role will involve assisting the teams in effectively navigating and resolving these challenges.

In addition to the potential impact on development of CVD across a large patient population, the work of implementing best practice to CR centres may engage local leaders in prioritizing CR and preventive patient care, increase CR centres awareness of their ability to change and adapt, and increase their collegial network within the field of preventive cardiology.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the PI on reasonable request.

Abbreviations

Cost effectiveness analysis

• Cardiac rehabilitation

Cardiovascular disease

Exercise-based cardiac rehabilitation

Expert Recommendations for Implementing Change

EuroQoL Visual Analogue Scale

Incremental cost-effectiveness ratio

Low-density lipoprotein cholesterol

Major adverse coronary events

• Myocardial infarction

Normalisation Measure Development

Normalization Process Theory

Swedish Web-system for Enhancement and Development of Evidence-based care in Heart Disease Evaluated According to Recommended Therapies

Visseren FLJ, Mach F, Smulders YM, Carballo D, Koskinas KC, Back M, et al. [2021 ESC guidelines on cardiovascular disease prevention in clinical practice]. G Ital Cardiol (Rome). 2022;23(6 Suppl 1):e3–115.

PubMed   Google Scholar  

Salzwedel A, Jensen K, Rauch B, Doherty P, Metzendorf MI, Hackbusch M, Völler H, Schmid JP, Davos CH. Effectiveness of comprehensive cardiac rehabilitation in coronary artery disease patients treated according to contemporary evidence based medicine: Update of the Cardiac Rehabilitation Outcome Study (CROS-II). Eur J Prev Cardiol. 2020;27(16):1756–74. https://doi.org/10.1177/2047487320905719 .

van Halewijn G, Deckers J, Tay HY, van Domburg R, Kotseva K, Wood D. Lessons from contemporary trials of cardiovascular prevention and rehabilitation: a systematic review and meta-analysis. Int J Cardiol. 2017;232:294–303.

Article   PubMed   Google Scholar  

Dibben GO, Faulkner J, Oldridge N, Rees K, Thompson DR, Zwisler AD, et al. Exercise-based cardiac rehabilitation for coronary heart disease: a meta-analysis. Eur Heart J. 2023;44(6):452–69.

Article   PubMed   PubMed Central   Google Scholar  

Ambrosetti M, Abreu A, Corra U, Davos CH, Hansen D, Frederix I, et al. Secondary prevention through comprehensive cardiovascular rehabilitation: from knowledge to implementation. 2020 update. A position paper from the Secondary Prevention and Rehabilitation Section of the European Association of Preventive Cardiology. Eur J Prev Cardiol. 2021;28(5):460–95.

Abreu A, Frederix I, Dendale P, Janssen A, Doherty P, Piepoli MF, Völler H, Davos CH; Secondary Prevention and Rehabilitation Section of EAPC Reviewers. Standardization and quality improvement of secondary prevention through cardiovascular rehabilitation programmes in Europe: The avenue towards EAPC accreditation programme: A position statement of the Secondary Prevention and Rehabilitation Section of the European Association of Preventive Cardiology (EAPC). Eur J Prev Cardiol. 2021;28(5):496−509. https://doi.org/10.1177/2047487320924912.

Kotseva K, De Backer G, De Bacquer D, Ryden L, Hoes A, Grobbee D, et al. Lifestyle and impact on cardiovascular risk factor control in coronary patients across 27 countries: results from the European Society of Cardiology ESC-EORP EUROASPIRE V registry. Eur J Prev Cardiol. 2019;26(8):824–35.

Abreu A, Frederix I, Dendale P, Janssen A, Doherty P, Piepoli MF, et al. Standardization and quality improvement of secondary prevention through cardiovascular rehabilitation programmes in Europe: the avenue towards EAPC accreditation programme: a position statement of the secondary prevention and rehabilitation section of the European Association of Preventive Cardiology (EAPC). Eur J Prev Cardiol. 2021;28(5):496–509.

Ambrosetti M, Abreu A, Corrà U, Davos CH, Hansen D, Frederix I, Iliou MC, Pedretti RFE, Schmid JP, Vigorito C, Voller H, Wilhelm M, Piepoli MF, Bjarnason-Wehrens B, Berger T, Cohen-Solal A, Cornelissen V, Dendale P, Doehner W, Gaita D, Gevaert AB, Kemps H, Kraenkel N, Laukkanen J, Mendes M, Niebauer J, Simonenko M, Zwisler AO. Secondary prevention through comprehensive cardiovascular rehabilitation: From knowledge to implementation. 2020 update. A position paper from the Secondary Prevention and Rehabilitation Section of the European Association of Preventive Cardiology. Eur J Prev Cardiol. 2021;28(5):460–95. https://doi.org/10.1177/2047487320913379 .

Ogmundsdottir Michelsen H, Sjolin I, Schlyter M, Hagstrom E, Kiessling A, Henriksson P, et al. Cardiac rehabilitation after acute myocardial infarction in Sweden - evaluation of programme characteristics and adherence to European guidelines: the Perfect Cardiac Rehabilitation (Perfect-CR) study. Eur J Prev Cardiol. 2020;27(1):18–27.

Wittboldt S, Leosdottir M, Ravn Fischer A, Ekman B, Back M. Exercise-based cardiac rehabilitation after acute myocardial infarction in Sweden - standards, costs, and adherence to European guidelines (the Perfect-CR study). Physiother Theory Pract. 2024;40(2):366–76.

Vasko P, Alfredsson J, Back M, Dahlbom L, Engblom H, Erlinge D, et al. SWEDEHEART årsrapport 2022. Report No.: 2000 – 1843. Uppsala: Uppsala Clinical Research Center (UCR); 2023.

Jernberg T, Attebring MF, Hambraeus K, Ivert T, James S, Jeppsson A, et al. The Swedish web-system for enhancement and development of evidence-based care in heart disease evaluated according to recommended therapies (SWEDEHEART). Heart. 2010;96(20):1617–21.

Back M, Leosdottir M, Hagstrom E, Norhammar A, Hag E, Jernberg T, et al. The SWEDEHEART secondary prevention and cardiac rehabilitation registry (SWEDEHEART CR registry). Eur Heart J Qual Care Clin Outcomes. 2021;7(5):431–7.

Back M, Bergstrom O, Ekstrom M, Forslund T, Hag E, Johansson P, et al. Riktlinje för sekundärprevention vid kranskärlssjukdom. sjukvård NsfkH-o, editor, Vol. 1. Uppsala: Sveriges regioner i samverkan; 2022.

Leosdottir M, Bäck M, Dahlbom L, Ekstrom M, Lindahl B, Hagstrom E. Data standards for cardiac rehabilitation processes and structures. Uppsala: the SWEDEHEART cardiac rehabilitation (SWEDEHEART-CR) registry; 2023.

Socialstyrelsen. About the Swedish healthcare system 2019-04-25 [updated 2020-09-01. https://www.socialstyrelsen.se/en/about-us/healthcare-for-visitors-to-sweden/about-the-swedish-healthcare-system/ .

Astley CM, Neubeck L, Gallagher R, Berry N, Du H, Hill MN, et al. Cardiac rehabilitation: unraveling the complexity of referral and current models of delivery. J Cardiovasc Nurs. 2017;32(3):236–43.

Kotseva K. Implementation of cardiovascular disease prevention guidelines in clinical practice-can we do better? Curr Treat Options Cardiovasc Med. 2015;17(12):58.

Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010;8:63.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated consolidated framework for implementation research based on user feedback. Implement Sci. 2022;17(1):75.

Flodgren G, O’Brien MA, Parmelli E, Grimshaw JM. Local opinion leaders: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2019;6(6):CD000125.

Forsetlund L, O’Brien MA, Forsen L, Reinar LM, Okwen MP, Horsley T, et al. Continuing education meetings and workshops: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2021;9(9):CD003030.

Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50.

Liddy C, Hogg W, Singh J, Taljaard M, Russell G, Deri Armstrong C, et al. A real-world stepped wedge cluster randomized trial of practice facilitation to improve cardiovascular care. Implement Sci. 2015;10:150.

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med. 2012;10(1):63–74.

Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, Steen IN, McColl EM, Dickinson C, Rapley T, Nomad. Implementation measure based on Normalization Process Theory. [Measurement instrument]. 2015 [ http://www.normalizationprocess.org .

CFIR Guide [Available from. https://cfirguide.org/guide/app/#/ .

Graneheim UH, Lindgren BM, Lundman B. Methodological challenges in qualitative content analysis: a discussion paper. Nurse Educ Today. 2017;56:29–34.

Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105–12.

Article   CAS   PubMed   Google Scholar  

Aktaa S, Batra G, Wallentin L, Baigent C, Erlinge D, James S, et al. European society of cardiology methodology for the development of quality indicators for the quantification of cardiovascular care and outcomes. Eur Heart J Qual Care Clin Outcomes. 2022;8(1):4–13.

Cappadona I, Corallo F, Cardile D, Ielo A, Bramanti P, Lo Buono V, Ciurleo R, D'Aleo G, De Cola MC. Audit as a Tool for Improving the Quality of Stroke Care: A Review. Int J Environ Res Public Health. 2023;20(5):4490. https://doi.org/10.3390/ijerph20054490 .

Rodrigues AM, O’Brien N, French DP, Glidewell L, Sniehotta FF. The question-behavior effect: genuine effect or spurious phenomenon? A systematic review of randomized controlled trials with meta-analyses. Health Psychol. 2015;34(1):61–78.

Gotham HJ. Advancing the implementation of evidence-based practices into ClinicalPractice: how do we get there from Here? Prof Psychol: Res Pract. 2006;37(6):606–6013.

Article   Google Scholar  

Download references

Acknowledgements

Not applicable.

Open access funding provided by Lund University. This study is funded by The Swedish Research Council for Health, Working Life and Welfare grant (Grant ID: 2019 − 00365) and by The Swedish Heart Lung Foundation (Grant ID: 20190431). The funding body has no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Author information

Authors and affiliations.

Department of Clinical Sciences Malmö, Lund University, Malmö, Sweden

Halldóra Ögmundsdóttir Michelsen, Björn Ekman & Margrét Leósdóttir

Department of Emergency medicine and Geriatrics, Helsingborg Hospital, Helsingborg, Sweden

Halldóra Ögmundsdóttir Michelsen

Department of Medicine, Karolinska Institute, Solna, Stockholm, Sweden

Matthias Lidin

Department of Cardiology, Heart, Vascular and Neuro Theme, Karolinska University Hospital, Stockholm, Sweden

Department of Occupational Therapy and Physiotherapy, Sahlgrenska University Hospital, Gothenburg, Sweden

Department of Health, Medicine and Caring Sciences, Unit of Physiotherapy, Linköping University, Linköping, Sweden

Department of Women’s and Children’s Health, Uppsala University, Uppsala, Sweden

Therese Scott Duncan & Maria Hägglund

Department of Medical Sciences, Cardiology, Uppsala University, Uppsala, Sweden

Emil Hagström & Bertil Lindahl

Uppsala Clinical Research Centre, Uppsala, Sweden

Department of Cardiology, Skåne University Hospital, Malmö, Sweden

Mona Schlyter & Margrét Leósdóttir

You can also search for this author in PubMed   Google Scholar

Contributions

HÖM: Conceptualization, Methodology, Writing - Original Draft, Writing - Review & Editing. MLi: Conceptualization, Methodology, Writing - Review & Editing. MB: Methodology, Writing - Review & Editing. TSD: Methodology, Writing - Review & Editing. EH: Methodology, Writing - Review & Editing, MH: Methodology, Writing - Review & Editing. BE: Methodology, Writing - Review & Editing, BL: Methodology, Writing - Review & Editing, MS: Writing - Review & Editing. MLe: Conceptualisation, Methodology, Writing - Review & Editing, Supervision, Project administration, Funding acquisition.  All authors have approved the submitted version.  All authors have agreed both to be personally accountable for the author’s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.

Corresponding author

Correspondence to Halldóra Ögmundsdóttir Michelsen .

Ethics declarations

Ethics approval and consent to participate.

The study has been approved by the Swedish Ethical Review Authority (Registration number: 2023-03217-01).

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., supplementary material 4., supplementary material 5., supplementary material 6., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Michelsen, H.Ö., Lidin, M., Bäck, M. et al. The effect of audit and feedback and implementation support on guideline adherence and patient outcomes in cardiac rehabilitation: a study protocol for an open-label cluster-randomized effectiveness-implementation hybrid trial. Implementation Sci 19 , 35 (2024). https://doi.org/10.1186/s13012-024-01366-8

Download citation

Received : 15 March 2024

Accepted : 15 May 2024

Published : 24 May 2024

DOI : https://doi.org/10.1186/s13012-024-01366-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Cost-effectiveness
• Implementation support
• Secondary prevention

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Advertisement

• Previous Article
• Next Article

PEER REVIEW

1. introduction, 2. data and methods, 4. discussion and conclusions, author contributions, competing interests, funding information, data and code availability, large-scale text analysis using generative language models: a case study in discovering public value expressions in ai patents.

Co-first authors.

Handling Editor: Vincent Larivière

• Funder(s):  Partnership for the Organization of Innovation and New Technologies
• Funder(s):  Social Sciences and Humanities Research Council of Canada
• Award Id(s): 895-2018-1006
• Principal Award Recipient(s): Sergio   Pelaez , Philip   Shapira
• Funder(s):  Biotechnology and Biological Sciences Research Council
• Award Id(s): BB/W013770/1
• Principal Award Recipient(s): Philip   Shapira
• Funder(s):  Snap Research Fellowship
• Principal Award Recipient(s): Gaurav   Verma
• Cite Icon Cite
• Open the PDF for in another window
• Permissions
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• Search Site

Sergio Pelaez , Gaurav Verma , Barbara Ribeiro , Philip Shapira; Large-scale text analysis using generative language models: A case study in discovering public value expressions in AI patents. Quantitative Science Studies 2024; 5 (1): 153–169. doi: https://doi.org/10.1162/qss_a_00285

Download citation file:

• Ris (Zotero)
• Reference Manager

We put forward a novel approach using a generative language model (GPT-4) to produce labels and rationales for large-scale text analysis. The approach is used to discover public value expressions in patents. Using text (5.4 million sentences) for 154,934 US AI patent documents from the United States Patent and Trademark Office (USPTO), we design a semi-automated, human-supervised framework for identifying and labeling public value expressions in these sentences. A GPT-4 prompt is developed that includes definitions, guidelines, examples, and rationales for text classification. We evaluate the labels and rationales produced by GPT-4 using BLEU scores and topic modeling, finding that they are accurate, diverse, and faithful. GPT-4 achieved an advanced recognition of public value expressions from our framework, which it also uses to discover unseen public value expressions. The GPT-produced labels are used to train BERT-based classifiers and predict sentences on the entire database, achieving high F1 scores for the 3-class (0.85) and 2-class classification (0.91) tasks. We discuss the implications of our approach for conducting large-scale text analyses with complex and abstract concepts. With careful framework design and interactive human oversight, we suggest that generative language models can offer significant assistance in producing labels and rationales.

https://www.webofscience.com/api/gateway/wos/peer-review/10.1162/qss_a_00285

Supervised machine learning (ML) relies on high-quality training data labeled by humans. However, the process of obtaining and validating human annotations, especially for complex and abstract concepts, is often overlooked and underemphasized in ML research and education. This can lead to costly and unreliable data that affect the performance and validity of ML models ( Geiger, Yu et al., 2020 ). To address this gap, this paper develops a novel approach to aid the labeling of complex and abstract concepts using generative language models (GLMs). Although our method is relevant for varied disciplines and studies undertaking text mining and content analysis, we demonstrate its application in the context of science and innovation policy analysis. Specifically, we put forward a semiautomated approach to identifying public values associated with inventions in artificial intelligence (AI) through the analysis of text in AI patent documents.

Previous quantitative research in science and innovation policy has focused on the use of bibliometric and text-mining methods of sources that include scientific publications and patents. Trends and patterns in research and innovation performance have been discerned using a range of text processing and content analysis techniques ( Antons, Grünwald et al., 2020 ; Porter & Cunningham, 2004 ). More recently, studies have used Bidirectional Encoder Representations from Transformers (BERT)–based or similar techniques to categorize research documents according to their respective subject areas based on abstract texts. Examples include tagging publications by their discipline and keywords ( Färber & Ao, 2022 ), identifying AI patent documents ( Giczy, Pairolero, & Toole, 2022 ) and research papers ( Sachini, Sioumalas-Christodoulou et al., 2022 ), and obtaining multiple category label predictions for social science journal articles ( Eykens, Guns, & Engels, 2021 ). Seed and antiseed sets of examples of labels for different categories are required to build a model using these methods. These methods have been applied to large-scale data sets. Qualitative methods have also been used to code and classify unstructured text of scientific publications, patents, and other innovation policy documents ( Ribeiro & Shapira, 2020 ), providing nuanced analyses but with human limitations on the volume of data that can be analyzed.

In this study, we deploy an approach that uses a recently available GLM to analyze unstructured patent text. In our case, we apply the approach to discover and classify text in AI patents that conveys attention to public values. It is a semiautomated approach that involves human input and review to identify concepts that are complex and context dependent but uses a GLM and ML to accelerate and scale up classification processes. The approach addresses the shortcomings of human-based labeling, such as its high demands on time and resources, yet also supports collaborative annotation and enhances reflexivity in abductive research.

1.1. Public Value Expressions in Patent Documents

In public policy and administration, public values (PVs) have been characterized as enduring beliefs that are founded on an ideal of human society. As a result, they offer direction, meaning, and legitimacy to collective action ( Rutgers, 2015 ). This concept has been put forward as an improvement over the traditional notion of public interest, which was viewed as ambiguous and lacking practicality as a guide. It was also proposed as an alternative theory to the market failure framework, which was deemed to offer a narrow and sometimes contradictory policy justification. In this sense, PVs are portrayed to be more concrete and practical relative to the idea of public interest while providing broader possibilities for policy deliberation and justification relative to the market failure framework ( Bozeman, 2002 ).

Efforts to identify PVs have involved investigating, for example, scholarly literature, government documents, cultural artifacts, or opinion polls ( Bozeman & Sarewitz, 2011 ). We frame the written articulation of a PV as a public value expression (PVE) where it indicates societal benefits that are promised to or for people, organizations, or ecosystems. A PVE is a signal—it does not mean that the PV will necessarily be realized but it does suggest that there is an idea or intent to do so. Discovering and analyzing such signals, in the context of science and innovation policy, helps understand the potential pathways and directions emphasized and promised by researchers and inventors. At the same time, identifying PVEs presents an operationalization challenge that needs to be addressed if they are to be used as an analytical tool for policy deliberation and justification. PVEs can be difficult to identify, as they are contextual, change over time, and require distinctions that are not always evident or easy to discern ( Fukumoto & Bozeman, 2019 ).

Patent documents can be a valuable source of PVEs ( Ribeiro & Shapira, 2020 ). Patents are manifested in written text, where inventors and legal representatives elaborate on their inventions and the context in which they operate. These descriptions provide valuable opportunities to identify and analyze PVEs. Additionally, given their radical novelty, fast growth, prominent impact, and uncertainty ( Rotolo, Hicks, & Martin, 2015 ), emerging and general-purpose technologies, such as AI, which are often the subject of patent applications, can serve as a springboard for more comprehensive discussions on society’s PVs. Because emerging technologies can change society in fundamental ways, it is important to reflect on the extent to which these changes align with our collective values and goals. This can be done by understanding how PVs are mobilized in narratives around these technologies. While doing so, we can gain insights into the intended societal benefits of emerging technologies as well as anticipate their potential negative impacts ( Buhmann & Fieseler, 2021 ).

Patent texts are, therefore, valuable materials for studying PVEs because, at one level, they contain detailed descriptions of inventions and the processes used to create them, which can provide insights into the current state of the art, emerging trends, commercial uses, and broader impacts of technologies. At another level, in their background section, patent texts offer clues regarding the social and economic context in which the technology was developed as well as its intended impacts in such a context. The arrival and accessibility of GLMs that can address complex and ambiguous topics, abridge them for human understanding, and organize them effectively, presents an opportunity to explore a new approach to discovering PVEs in patent texts. We describe this approach in the following sections, using the case of AI patents.

The approach used in this study proceeds through four key stages. First, we create a database of AI documents and identify potential PVEs through keyword filtering. Next, we develop a framework for sentence labeling that involves both human input and AI annotation (using a GLM). Third, we estimate BLEU scores and perform topic modeling to evaluate the faithfulness, diversity, and discovery capabilities of the AI annotator’s output. Finally, we use these annotated labels to train an open-source classifier and apply it to all records. These steps are illustrated in Figure 1 and explained in detail in the following sections.

Schematic of main stages of the study approach.

2.1. Data: Obtaining AI Patent Documents and PVEs

Our process for obtaining a set of patent documents involves two steps. First, we employ a search strategy developed by Liu, Shapira et al. (2021) that uses AI-related keywords, cooperative patent classification (CPC) codes, and international patent classification (IPC) codes to retrieve an extensive collection of U.S. patent applications and granted patents filed between 2005 and 2022. To execute the search, we entered a Boolean query into InnovationQ+® 1 , a patent search tool, resulting in the retrieval of patent ID numbers and metadata at the invention level. This yielded 198,456 patent documents, which included a single record for each simple family and granted patents that overrode applications. Second, we used the bulk download option of PatentsView, provided by the United States Patent and Trademark Office (USPTO) 2 , to extract background, summary (description), and abstract text. We merged the IDs generated in the first step with the text retrieved in the second step to obtain a final set of 154,934 U.S. patent documents related to AI.

The patent documents were split into individual sentences to facilitate their analysis and categorization through ML classifiers. The 154,934 patent documents yielded 5.4 million sentences. To find PVEs within such a large and sparse corpus, we required a method to obtain a manageable subset of sentences with a high density of PVEs to select a sample to annotate. To do so, we implemented a keyword filtering approach. This involved creating a list of single words, bigrams, and trigrams. Initial PV keywords were sampled from a narrative review of relevant documents, both peer-reviewed and nonpeer-reviewed, focusing on the benefits and risks of AI. An important part of this literature is reflected in publications related to the governance of this technology. Therefore, we conducted a search for guidelines published by public and private organizations, as well as journal articles analyzing and proposing frameworks for managing the impacts of AI. Our final list of terms ( N = 320) represented a broad range of topics related to both the positive and adverse impacts of AI, as discussed in the available literature. By filtering sentences containing these terms, we were able to retrieve a smaller subset of sentences ( N = 73,813) that potentially contained PVEs.

From this pool of sentences, reflecting practical and technical factors such as cost, time, and saturation in classification performance, we extracted a training and evaluation set of 10,000 sentences. Instead of randomly drawing sentences, we created a system to rank the importance of PV keywords, grouping them into four categories (1 to 4) based on their ability to capture relevant PVEs. On one end, keywords classified as category 1 were ranked very low in their ability to capture PVEs because they were considered ambiguous. That is, they had the potential to retrieve technical or private value sentences in addition to PVEs. Some examples of category 1 keywords include “risk management” and “emotional state.” Additionally, category 1 contained short-tail keywords that retrieved a disproportionately large number of sentences, such as “health care” and “education.” Keywords in category 4, on the other end, were characterized by their high ability to capture PVEs. These tended to be long-tail and to retrieve a small number of sentences. Examples of keywords in category 4 are “explainable artificial intelligence,” “human safety,” “privacy by design,” and “societal concern.” We used these categories to oversample sentences that were more relevant and undersample those that were less relevant. Specifically, we randomly sampled 4.5% of sentences from category 1, 14% from category 2, 65% from category 3, and 100% from category 4. By doing so, we ensured our training and evaluation set contained a diverse sample of sentences.

2.2. Framework: A Method for Designing Instructions for Human and AI Annotators

After obtaining the training and evaluation set, the next step was to generate labels for each sentence (i.e., classify them as PVE (or something else)). We developed a multistage process to develop a framework for labeling PVEs in patent documents. In the initial stage, we employed an abductive approach—a group of three researchers with experience in text mining of patent documents, PV theory, and emerging technologies went through the sentences and labeled them as either being PVEs or not. We achieved 66%, 56%, and 78% agreement rates in three batches of 50 sentences. The aim was to train ourselves in identifying PVEs in AI patents by observing patterns in disagreements and discussing our labeling decisions. In the second stage, we manually classified sentences on a larger scale, labeling 1,000 sentences with an agreement rate of 79%, leading to a new round of discussion.

A key learning from our discussions was that Although the three coders had many areas of agreement about PVEs, in other cases there were differences in the interpretation of PV concepts. This resulted in the rise of differences in labeling sentences. Our individual understanding of PV theories was often implicit and challenging to articulate to one another. Therefore, our labeling decisions were nonstandard and sometimes contradictory. Additionally, in the process of repeated coding, we disagreed with our past selves, failed to be consistent across sentences, and provided varied justifications for our choices. As explained below, we imputed that one of the key reasons for this behavior was the lack of a consolidated paradigm in PV theory, which required us to develop our own framework to support the labeling.

There may be other reasons that generally explain the challenges of labeling complex and abstract topics, such as PVs. In such cases, human labeling capabilities can be diminished by cognitive limitations, such as working memory constraints, mental fatigue from repetitive and lengthy coding tasks, attentional biases, and inflexibility (i.e., once we learn a heuristic to label, we get stuck with it, even if it is wrong). Cognitive biases can also play a role. For instance, a tendency to rely too heavily on initial information (anchoring bias) could result in misclassifying later sentences. The impact of labeling discrepancies could be significant given a small set of labelers. Earlier research in fields such as political science, for example, has relied on crowdsourcing to deploy large-scale, qualitative analyses of text to overcome the challenges of bias and inconsistent coding. In this context, the deployment of an increased number of labelers was justified as a way to compensate for individual errors ( Benoit, Conway et al., 2016 ).

To overcome the variability of individual coding, we progressed to a third stage, where we provided written justifications for decisions regarding 100 labeled sentences. Justifications made our disagreements and the limitations of PV theory more visible. We discussed how PV theory differed from similar frameworks, such as public interest. We also debated whether PVEs and private value statements were mutually exclusive, the assumptions that were acceptable to infer a PVE from an ambiguous sentence, the role of the target beneficiary in determining whether a sentence was a PVE, the unequal standards to annotate sentences across PV themes, and the consequences of assigning hierarchies to PVEs. Notably, some of these debates are at the center of current discussions in PV theory, as highlighted, for example, in Fukumoto and Bozeman (2019) . Our efforts to provide justifications and address contested points led us to create a set of guidelines that encompassed positive heuristics for identifying a sentence as a PVE and negative heuristics for determining when not to label a sentence as a PVE.

a concrete and comprehensive definition of PV;

a categorization of sentences as Direct-PVE (i.e., the sentence demonstrates that the invention addresses a PV), Contextual-PVE (i.e., the sentence demonstrates awareness of a PV, but does not provide enough information to link it with the invention), or No-PVE;

a reduced number of two positive heuristics and three negative heuristics;

four examples for each heuristic; and

rationales for each example.

This framework is included in the Supplementary material .

The high-level form of the framework, which most effectively communicates with humans, must be adapted to function as a prompt with instructions for the AI annotator. We provided system instructions to the model to clarify its role and supply necessary definitions to aid categorization. In Figure 2 , we show the instructions provided to the GPT-4 model and the role that different components play. Before specifying the definitions crafted as part of the developed framework, we specify the role of the GPT-4 agent (see “Task specification” in the figure). Subsequently, we use the definitions developed as part of our framework ( Supplementary material ). The definitions are concise and actionable, and can include both inclusionary as well as exclusionary identifiers for each of the categories. Furthermore, based on our early experiments, we added statements to correct the behavior of the GLM. In the illustrated example, it is to stop the model from making assumptions and inferences based on its inherent knowledge of a language (see “Iterative refinement of definitions to correct behavior” in the figure). Finally, we note that some definitions can refer to the properties discussed as part of other definitions, as long as the entire set of instructions provided to the model is kept self-contained. As GLMs demonstrate long-form context modeling abilities, they can use the considerations discussed under the definition of “Public Value Expression” to infer “Direct-PVE. ”

Illustration of the instructions provided to GPT-4 and the key components involved.

Following the instructions, we provided 14 examples to GPT-4, spanning the categories under consideration. For each example, we start the rationale with the phrase “Let’s think step by step” to trigger chain-of-thought reasoning by GPT-4, a known technique to help the agent arrive at a prediction that follows an exemplified reasoning scheme. Each rationale follows: “Based on these considerations, I would categorize this sentence as: <final prediction>.” The 14 examples were chosen to illustrate a diverse and nonredundant reasoning scheme and were chosen iteratively while analyzing performance on a small set of unseen examples ( N = 100). We show one such example in Figure 3 .

Illustration of example sentence and rationale provided to GPT-4, with key components indicated.

GPT-4 produced a label (i.e., Direct-PVE, Contextual-PVE, or No-PVE) and a rationale for its decision. Our agreement with GPT’s labels went up to 95% in a conservative estimation in a separate evaluation set of 1,000 randomly sampled sentences (from the subset of 10,000 sentences).

Out of the 1,000 sentences in the validation set, we found only one case where the three human annotators disagreed with GPT-4. This sentence mentioned privacy concerns, which GPT-4 classified as No-PVE, when it should be a Contextual-PVE. GPT-4 wrongly suggested that privacy was not a broad societal concern. However, this was an isolated error, and other privacy sentences were correctly labeled by GPT-4.

We also found 42 cases where one annotator disagreed with GPT-4 and the other two agreed with it. We counted these as disagreements so as to be conservative in our evaluation. These cases involved nuances in the use of words such as “environment,” “smart cities,” “education,” and “government programs,” which could imply value or technical meanings, depending on the context. Although further refining the prompts for GPT-4 could reduce some disagreements, it could also increase others. Overall, we concluded that GPT-4 was generally accurate in labeling PVEs and that achieving small marginal improvements beyond this point would be difficult. As we demonstrate in the analysis section, not only did GPT-4 produce sensitive labels, but its rationales demonstrated a consistent understanding of our framework and provided logical reasons to support its classifications.

2.3. Assessment: Methods for Measuring the Faithfulness and Diversity of the AI Annotator’s Outputs

In contrast to discriminative models such as BERT and its variants, GLMs such as GPT-4 can be used for generating coherent long-form rationales along with the final predictions. These rationales serve three purposes. First, chain-of-thought prompting (i.e., encouraging the GLM to adopt a reasoning scheme by using phrases such as “Let’s think step by step” before predicting the final label) has demonstrated effectiveness in eliciting an accurate final prediction by the model ( Wei, Wang et al., 2022 ). Second, these rationales are material that allows human researchers to assess the labels, making the decision process of the GLM more transparent and subject to human review. Third, given the high quality of the rationales, these function as persuasive and standardization mechanisms—they help humans in their own labeling decisions and focus their justifications on a smaller set of standard and organized possibilities, thus enabling them to overcome cognitive limitations and biases.

To authenticate this line of argument, we use the rationales that GPT-4 generates while labeling the 10,000 examples to answer the following question: How diverse and faithful are the rationales (or reasoning mechanisms) and can the generative language model strike a balance between the two measures?

The BLEU score ranges from 0 to 1, where 1 denotes the maximum similarity between two sentences. However, as our goal is to measure the diversity of rationales, we interpret the scores as low values demonstrating higher diversity. We are interested in the following forms of diversity: (a) How diverse are the generated rationales with respect to the rationales provided to the model, and (b) how diverse are the generated rationales among each other. For (a), we quantify the category-wise average of maximum similarity that a given generated rationale has with the supplied rationales of the same category. For (b), we take the average pairwise similarity scores of all the within-category generated rationales. The same similarity scores can be used to comment on how “faithful” the generated rationales are with respect to the provided rationales. High faithfulness is desired to ensure that the model’s reasoning mechanism abides by the same set of reasoning principles as depicted by the supplied rationales.

In addition to using the BLEU similarity scores to assess the quality of the rationales generated by GPT-4, especially regarding striking a balance between diversity and rationales, we assess GPT-4’s ability to discover PV themes with topic modeling. To ensure cost-effectiveness, the number of illustrative examples that can be provided to GPT-4 is limited. Therefore, we carefully curate four examples that cover nonredundant themes demonstrating Direct-PVEs. However, as these four themes do not cover all possible themes related to PVs, we aim to evaluate whether GPT-4 can produce rationales belonging to different themes not included in the examples. We perform topic modeling on the generated rationales of the subset of sentences labeled “Direct-PVE” by GPT-4. More specifically, we use Latent Dirichlet Allocation (LDA) ( Blei, Ng, & Jordan, 2003 ) to discover 10 topics among all the aforementioned rationales. We chose the number 10 after qualitatively analyzing the coherence and overlap of produced topics. We then qualitatively inspect the 10 topics and assess whether they relate to the themes covered in the provided examples or have been revealed because of the reasoning capabilities of GPT-4.

2.4. Modeling: Training and Prediction Methods

The goal of obtaining labels for 10,000 sentences is to train open-source classifiers such as BERT ( Devlin, Chang et al., 2018 ), which could then be used for inferring the labels of millions of sentences. Given the remarkable performance of GPT-4, which was validated to align with human experts more than 95% of the time, a desirable setting would use GPT-4 to label all the millions of examples in the target corpus. However, using closed-source GLM for large-scale inference would incur infeasible costs for us (and many other academic research groups). Despite open-source counterparts of proprietary generative language models, such as LLaMA ( Touvron, Lavril et al., 2023 ), OPT ( Zhang, Roller et al., 2022 ), and Flan-T5 ( Chung, Hou et al., 2022 ), hosting them requires a large GPU infrastructure that may not be accessible to many scholars. Furthermore, current open-source language models are also limited in their maximum context length, which limits the information that can be provided as part of the instructions and examples.

To this end, we deploy a hybrid approach where we use the 10,000 examples labeled using GPT-4 to train open-source pretrained BERT-like models. BERT ( Devlin et al., 2018 ), is a language representation model that uses transformers and bidirectional training to understand the context of words within a sentence. It is pretrained using a combination of masked language modeling objective and next-sentence prediction on a large corpus of text. Such models can be hosted on a single GPU (NVIDIA A100 80GB, available via platforms such as Google Colab) and can be trained for the specific use case at hand with relative ease. For training the models, we use the 9,000 examples labeled using GPT-4 as the training corpus and the 1,000 GPT-4-identified, human-validated examples as the evaluation corpus. We use two formulations of the classification task: 3-class classification, under which the models are tasked to distinguish between Direct-PVE (D-PVE), Contextual-PVE (C-PVE), and No-PVE; and 2-class classification, under which the models are tasked to distinguish between PVE and No-PVE. By design, the former setting is more challenging, as it involves distinguishing between the subcategories of PVEs. To quantify the performance of the models, we use macro averages of class-wise F1 scores, precision, recall, and accuracy.

We evaluate a range of pretrained language models that are similar to the BERT model. We describe these models and the key differences between them. BERT-base-uncased is a version of BERT with 110 million parameters. It deals with lowercase English text, allowing it to be smaller and faster while maintaining robust performance on many tasks. BERT-large-uncased, an expanded version of BERT-base, contains 340 million parameters. Despite being computationally heavier, its larger size grants it improved language understanding capabilities. DistilBERT ( Sanh, Debut et al., 2019 ) is a distilled, lighter version of BERT, maintaining comparable performance while being smaller (66 million parameters), faster, and cheaper to run. It uses a teacher-student training paradigm, learning from a larger “teacher” BERT model. RoBERTa-large (354 million parameters), or Robustly Optimized BERT, is a variant of BERT that modifies BERT’s training process for improved performance, using larger batches of data, more data, and removing the next-sentence prediction task ( Liu, Ott et al., 2019 ). ALBERT-xxl-v2 (223 million parameters), or A Lite BERT, is a version of BERT that introduces parameter-reduction techniques to lower memory consumption and increase training speed. It achieves this by sharing parameters across layers and factorizing the embedding layer ( Lan, Chen et al., 2019 ). Finally, DeBERTa-xxl-v2 (1.5 billion parameters), or Decoding-enhanced BERT with disentangled attention, improves upon BERT and RoBERTa with a disentangled attention mechanism ( He, Liu et al., 2020 ). This allows the model to dynamically integrate contextual information, improving its understanding of complex language phenomena 3 .

To provide context for the performance of the classifiers obtained from training these pre-trained language models, we also include the performance that random classifiers would achieve on the evaluation set. These random classifiers, uniform and biased based on class ratio, serve as a baseline comparison for evaluating the effectiveness of our trained classifiers.

3.1. Evaluating GPT-4 Rationales

it checks for an explicit reference to a PV in the sentence;

it identifies possible PV themes that are related to the reference; and

it looks for evidence that connects the patented invention and the PV (e.g., “an objective of the present invention …”).

GPT-4 rationales for PVE labeling with three examples

To evaluate the quality of GPT-4’s rationales, we can use an additional method based on BLEU scores. Our goal is to ensure that these rationales are both faithful and diverse. This means that our framework should serve as a reliable guide for generating rationales for unseen examples. Additionally, the GLM should be able to deduce the label of any PV theme—even if it is not included in the examples—by leveraging the first principles outlined in the framework.

We analyze the diversity of the GPT-4 rationales with respect to the illustrative examples supplied to GPT-4. Recall that a lower BLEU score indicates more diversity. We use BLEU scores (see Table 2 ) to quantify the pairwise similarity among within-category rationales provided to GPT-4 (first column after categories), the average maximum similarity of generated rationales with within-category provided rationales (second column), and the average pairwise similarity among within-category generated rationales (third column).

Diversity and faithfulness of rationales

As indicated, we first looked at the diversity of the rationales that were provided, and the average of pairwise similarity among within-category rationales was calculated. These low values indicate that the provided rationales are very diverse with respect to each other, which is expected as the examples were chosen to be nonredundant. Next, we analyzed the diversity of the generated rationales with respect to the provided rationales. The average maximum similarity between each generated rationale and the provided rationales for the same label was calculated. The increase in BLEU scores indicates that the provided rationales are being used as guidelines for generating the rationales of unseen examples, a behavior often referred to as faithfulness. However, it is worth noting that the BLEU scores are not exceptionally high, which demonstrates diversity in the sampled data and the rationales required to categorize the examples. Finally, we looked at the diversity of the generated rationales with respect to each other. The average of pairwise similarity among within-category rationales was calculated. The observed results are expected, as the generated rationales are anchored in the provided rationales and show some similarity but not too much. The diversity among Direct-PVE is notably higher in comparison to Contextual-PVE and No-PVE. One possible explanation for why Direct-PVEs are more diverse than Contextual-PVEs or No-PVEs is that Direct-PVEs are more likely to reflect the specificity and novelty of the AI invention and how it addresses a societal challenge or benefit. In contrast, Contextual-PVEs or No-PVEs are more likely to reflect the generality and commonality of AI technologies and their potential applications.

Topic 1: Security and safety enhancement ( covered )

Topic 2: Mental health, well-being, patient, and medical monitoring ( covered )

Topic 3: Privacy, data security, and personal information protection ( covered )

Topic 4: Clean energy, sustainability, and renewable resources ( revealed )

Topic 5: Economic and financial development ( revealed )

Topic 6: Transparency in decision-making, machine bias, and participation consent ( revealed )

Topic 7: Developing organizational trust among people ( revealed )

Topics 8 and 9: Modeling and assessment of disease risk ( covered )

Topic 10: Applications for assessing human behavior ( revealed )

We note that GPT-4 rationales for categorizing content as Direct-PVE cover some of the new themes that are unrelated to themes covered in the examples (five out of 10), whereas the other half are related to the ones that have been included in the examples (five out of 10). These results further advance the argument that GPT-4 can strike a balance between faithfulness with the provided rationales while exhibiting reasoning based on “revealed” themes.

Based on the above two tests, using BLEU scores and topic modeling, the results indicate that GPT-4 has balanced diversity and faithfulness in justifying its PV classification decisions, and GPT-4 internalized our framework and used it as a deductive device to classify the various types of PVEs it encountered.

3.2. Training Open-Source Classifiers

In the final major step of our approach, we use the labels produced by GPT-4 to train open-source discriminative classifiers. We report the results for the two settings—three-way classification and two-way classification in Tables 3 and 4 . First, we note that the performance for both the classification settings is satisfactory—the best models achieve an F1 score of almost 0.85 for the 3-class classification task and an F1 score of above 0.90 for the 2-class classification task. This means that the models were able to learn with the labels obtained from GPT-4 and predict accurately on a held-out evaluation set. Second, we note that the fine-grained classification task is relatively more difficult for the BERT-based classifiers, as it involves distinguishing between Contextual-PVEs and Direct-PVEs. Finally, we observe a clear trend across the two tasks. The number of parameters in the pretrained language model positively affects the classification performance. We can note that larger models (DeBERTa-xxl-v2 with 1.5 billion parameters) perform better than models with fewer parameters. This pattern aligns with the scaling law, suggesting that substantial performance improvements are achieved by scaling up LLMs ( Bowman, 2023 ). We investigated the use of Latent Dirichlet Allocation (LDA) to model topics from the entire data set, but this did not prove to be viable. However, our framework was not designed to identify specific topics but rather to categorize sentences into three broad categories. The rationales presented well-defined and distinct topics, but these were only available for 10,000 sentences. The rest of the 5.4 million sentences did not have enough semantic content for the LDA approach to generate meaningful and distinct topics.

Classification performance of models trained on data labeled using GPT-4: 3-class classification (No-PVE, D-PVE, and C-PVE)

Classification performance of models trained on data labeled using GPT-4: 2-class classification (No-PVE and PVE)

The availability of labeled data is an important requirement for training language classifiers for subsequent large-scale machine-based text analysis. However, the manual labeling of text is a resource-intensive and time-consuming process, limiting the volume of data that can reasonably be coded by humans. Additionally, when dealing with complex and abstract concepts such as PVs, manual labeling also carries cognitive limitations and biases that can reduce the quality of the labels. To address this problem, we put forward an alternative semiautomated approach where a GLM serves as the main annotator, with human validators providing verification on smaller samples. This approach requires crafting a comprehensive and intelligible framework with guidelines on what to do (i.e., positive heuristics), what not to do (i.e., negative heuristics), precise definitions, examples, and rationales. Such a framework acts as a prompt with instructions for the AI annotator.

The quality of the framework’s output depends on both the design of the framework and prompt and the capabilities of the GLM. We refined our definitions, guidelines, and examples through multiple iterations to reduce the error rate of GPT-4’s results. We note that Although our framework design worked well with GPT-4, it had limitations with GPT-3.5. When we tested the same framework on both models, GPT-4 produced outstanding results, but GPT-3.5 showed only a slight improvement over human labeling. Other text-based experiments have found a similar contrast between these two versions of GPT ( Nori, King et al., 2023 ). GPT-4 appears to be already capable of addressing complex classification tasks, and future versions will likely require less framework design work and provide more accurate labels and sophisticated rationales for abstract concepts ( Bubeck, Chandrasekaran et al., 2023 ).

In our framework, we used GPT-4 to label a training set (at cost) which was then used to predict within a much larger set using open-source discriminative classifiers. We found that the labels generated using this approach were consistent and accurate, with convincing rationales that aided humans in organizing their ideas and differences around intricate topics. The GLM produced rationales that followed the instructed guidelines (i.e., it is faithful) while generalizing to label unseen topics (i.e., it is diverse). Our analysis of the rationales offers insights into the reasoning capabilities of GLMs, demonstrating that they can strike a balance between faithfulness and diversity when prompted with carefully crafted instructions and examples.

To be clear, the approach presented in this paper was not costless. The cost for labeling the 10,000 sentences in March 2023 via the Open-AI GPT-4 API was about US$ 1,200, with just under 40 million tokens used. However, the cost in terms of time and money was far less than the fully manual coding of 10,000 sentences. GLMs may become less expensive to use or increasingly available for large-scale language processing on an open-access basis. With the support of GLMs, the semiautomated framework that we put forward in this study is likely to be useful in many use cases, especially when analyzing complex concepts. In particular, the approach has the potential to be widely useful in the social sciences, including in science and innovation policy analysis.

A key element of our approach is human–machine interaction. The capabilities of the human side are important. We recognize that the rise of GLMs raises concerns about the risk of bias, even in classifying text. Complex and abstract concepts such as PVs require training in and understanding of public administration, policy, responsible innovation and ethics, and related fields, meaning that, unlike labeling examples where it is easy to discriminate between categories (e.g., an apple is a fruit and an iris is a plant), complex and abstract concepts cannot be left for outsourced labeling (e.g., Mechanical Turk). On the machine side, even with more capable GLMs, it would be a mistake to assume that human labor is no longer necessary for large-scale text analysis. On the contrary, it is more critical than ever. Human researchers are indispensable in designing frameworks, engineering prompts, and validating results. Crucially, the careful development of training instructions depends on a series of human-led steps to ensure the quality of the analytical lens being constructed. Further, human supervision of intermediary outputs is equally necessary. In their supervised ML approach to management analysis, Harrison, Josefy et al. (2022) , show how different phases of the process, from construct identification to data scaling, depend on human manual labor. This is in line with the findings of recent studies looking at the impact of digitalization on scientific work ( Ribeiro, Meckin et al., 2023 ). As this research demonstrates, these activities remain a time- and intellectually intensive task that requires the expertise and creativity of skilled professionals.

Although we deploy a framework for AI-assisted analysis of text, the depth of this part of the analysis remains largely descriptive, in content analysis terms ( Krippendorff, 2019 ). It is through human iterations with the text, with each other and, crucially, with AI through its labeling and justifications behind its choices, that the analysis becomes more interpretive and subjective. This movement between human and AI labeling resembles hybrid approaches adopted elsewhere through collaboration between computational and qualitative researchers ( Li, Dohan, & Abramson, 2021 ). We argue that using generative language and machine learning models can both enhance and restrict the subjectivity component of content analyses and qualitative research. Ultimately, the descriptive or interpretive tendencies of the analysis will depend on the context of each study, including its research questions, disciplinary orientations, the characteristics of the theoretical frameworks with which researchers are engaging, and the shortcomings of AI models in dealing with more subjective interpretations of the data.

The approach used in this study builds on the strengths of GLMs in classification and pattern identification. There are limitations that should be kept in mind when interpreting the results. Our empirical data is restricted to text found in US AI patent documents. The expression of PVs in that text depends on the motivations of the inventors, assignees, and agents involved in preparing these documents. Our interpretation of PVs is collated from a comprehensive review of PV discussions in available, English-language, peer-reviewed publications and other documents. Broader or narrower interpretations of what is a PV could be posited. The semiautomated approach to using GLMs has a cost and requires coding skills, as well as research methodology skills.

In work already under way, and aware of these limitations, we will use our approach to label, organize, and analyze trends and patterns of PVE expressions in AI patent documents. We will focus attention on the drivers behind the inclusion of PVEs and the potential societal and policy implications of the presence (and absence) of PVEs in emerging AI technologies. We anticipate that approaches similar to those described in this study to use GLMs for labeling and organizing complex and abstract concepts, as well as classifying text at large scales, will be taken up by other researchers. Although we found substantial improvements over manual coding, there is a need for further research to compare and validate the use of GLMs in classifying complex social science concepts in texts. However, our initial experience with embedding a semiautomated GLM approach is promising; the findings suggest significant potential to assist researchers—in science and innovation policy analysis as well as other domains—in text analysis.

Sergio Pelaez: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing—original draft, Writing—review & editing. Gaurav Verma: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing—original draft, Writing—review & editing. Barbara Ribeiro: Conceptualization, Data curation, Investigation, Methodology, Supervision, Validation, Writing—review & editing. Philip Shapira: Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Writing—review & editing.

The authors have no competing interests.

This work was supported in part by the Partnership for the Organization of Innovation and New Technologies, SSHRC [grant number 895-2018-1006] (SP, PS); and the Biotechnology and Biological Sciences Research Council [grant number BB/W013770/1] (PS). GV is partially supported by the Snap Research Fellowship.

For legal reasons, data from InnovationQ+® cannot be made openly available. The Liu et al. (2021) AI patent search approach that was applied to InnovationQ+® is openly available at https://doi.org/10.1371/journal.pone.0262050 . USPTO patent records and text are openly available at https://patentsview.org . The training guidelines used in the study are provided in the Supplementary material . Examples of code and instructions provided to GPT-4 via API are provided in Figures 3 and 4 in the Supplementary material . The code for obtaining GPT-4 predictions, output labels and returned rationales are available at https://github.com/pshapira/pve .

https://ip.com/ .

https://patentsview.org/ .

We also experimented with BERT for patents ( Lee & Hsiang, 2019 ), which was trained on 100 million patents and has demonstrated notable performance in tasks such as searching for prior art while drafting patents, autocomplete, and automatically generating classification codes. Therefore, one might expect BERT for patents to be among the models being used for this task. However, we observed that the performance of BERT for patents is lacking (F1 score of 0.78 for the 3-class classification task) in comparison to the other models mentioned here (F1 score > 0.80 for the 3-class classification task). We believe that this is because identification of public values is not a conventional patent-related task and requires the understanding of broader context, which domain-specific models such as BERT for patents may lose because of restrictions to the training corpus.

Author notes

Supplementary data, email alerts, affiliations.

• Online ISSN 2641-3337

A product of The MIT Press

Mit press direct.

• About MIT Press Direct

Information

• Accessibility
• For Authors
• For Customers
• For Librarians
• Direct to Open
• Open Access
• Media Inquiries
• Rights and Permissions
• For Advertisers
• About the MIT Press
• The MIT Press Reader
• MIT Press Blog
• Seasonal Catalogs
• MIT Press Home
• Give to the MIT Press
• Direct Service Desk
• Terms of Use
• Privacy Statement
• Crossref Member
• COUNTER Member  
• The MIT Press colophon is registered in the U.S. Patent and Trademark Office

This Feature Is Available To Subscribers Only

Sign In or Create an Account

Study links talc use to ovarian cancer — a potential boon for thousands suing J&J

New research published this week lends credence to the more than 50,000 lawsuits against Johnson & Johnson that allege its talc-based baby powder caused ovarian cancer.

The analysis , released Wednesday in the Journal of Clinical Oncology, found that applying talc powder to the genitals was associated with ovarian cancer — and that the association was greater for people who used the powder frequently or for long periods of time.

The researchers are from the National Institutes of Health, and their findings were based on data from the Sister Study, which enrolled more than 50,000 women in the U.S. from 2003 to 2009. The participants joined when they were between 35 and 74 years old, and each had a sister who’d been diagnosed with breast cancer, which might put them at increased risk for breast or ovarian cancer.

Lawsuits related to J&J’s talc-based baby powder date back to 1999, when a woman alleged that a lifetime of using it led to her mesothelioma, a rare cancer usually caused by exposure to asbestos — a known carcinogen. In 2009, another woman sued the company, alleging that its talc-based products caused her ovarian cancer. Since then, many thousands of others have filed claims over cases of ovarian cancer or mesothelioma that they say were caused by asbestos in J&J baby powder.

J&J has stood by the safety of its talc products and denies that they ever contained asbestos. The company has also argued that studies have not demonstrated a convincing link between ovarian cancer and talc-based products.

The new research could undermine that line of reasoning as the legal battles continue. Most of the lawsuits against J&J have been consolidated into a single federal case in New Jersey, with trial scheduled for December.

“This study is quite timely. We feel like it completely affirms and confirms the position taken by plaintiffs’ experts,” said Leigh O’Dell, a principal at Beasley Allen Law Firm. O’Dell is the co-lead counsel for the plaintiffs’ steering committee, a group of attorneys appointed to act on behalf of the many people with pending cases against J&J.

However, Erik Haas, J&J’s worldwide vice president of litigation, said the new analysis doesn’t establish causality or implicate a specific cancer-inducing agent.

“This study does not change the overwhelming evidence that talcum powder does not cause ovarian cancer,” he said.

Earlier this month, J&J proposed a payment of around $6.48 billion to resolve the lawsuits, but the deal would involve moving the cases to bankruptcy court and require 75% of claimants to vote in favor.

J&J has tried and failed twice to resolve talc lawsuits in bankruptcy court . The company created a subsidiary in 2021 that could assume liability for talc-related legal claims — a legal maneuver known as a Texas two-step. But thus far, courts have dismissed the bankruptcy filings on the grounds that the subsidiary is not in financial distress.

O’Dell said her team “would like to see these women offered a reasonable and fair resolution outside of bankruptcy.”

“Any effort to file another bankruptcy, we believe, is just yet another abuse of the bankruptcy system,” she said.

The potential harms of talc products

The new study asked women how often they used talc powder on their genitals from ages 10 to 13 and during the year before they enrolled in the study. NIH researchers followed up with surveys from 2017 to 2019 that asked women about their lifetime use of talc powder.

Based on the responses, the researchers estimated that up to 56% of the women used talc powder on their genitals at some point. These women were more likely to be Black, less educated and live in the South compared with people who didn’t use talc powder.

The analysis can’t prove that talc causes ovarian cancer, nor does it identify a brand or chemical driving the association. Dale Sandler, one of the study’s authors and the chief of the epidemiology branch at the National Institute of Environmental Health Sciences, said there probably isn’t a way to prove causality in human studies.

“You can’t do a clinical trial and randomize people to ‘powder’ and ‘no powder.’ So we’re going to need to look to other types of research,” she said.

At the very least, the findings should prompt women to rethink their use of talc products, said Katie O’Brien, the lead author of the analysis and an epidemiologist at the National Institute of Environmental Health Sciences.

“We’re not aware of any medically necessary reasons why someone would need to use talcum,” she said.

Current formulations of J&J baby powder use cornstarch, not talc. The company pulled the talc-based versions from the North American market in 2020, citing waning demand and “misinformation around the safety of the product,” and discontinued the product internationally last year.

Talc and asbestos are found in close proximity in nature, so some raw talc collected via mining may be contaminated with asbestos , according to the Food and Drug Administration.

A 2018 Reuters investigation suggested that J&J knew some of its baby powder was contaminated with small amounts of asbestos as early as the 1970s. But J&J denies asbestos was ever present in its products.

O’Brien said asbestos might not be the only reason for an association between talc and cancer. Some talc products may also contain phthalates — chemicals that disrupt hormones in the body and have been linked to ovarian cancer . Plus, talc itself can be abrasive, she added, so it may cause inflammation in the areas where it’s applied. Inflammation is independently associated with the development of cancer.

A debate over the science

Debates over the research linking talc and ovarian cancer will almost certainly be a focus of upcoming litigation in the J&J case.

The New Jersey federal court ruled in March that the company can contest findings that link ovarian cancer to talc.

To support its position, J&J has pointed to research that O’Brien and Sandler published in 2020 , which did not find a statistically significant association between ovarian cancer and the use of talc powder.

But O’Brien said that older study may not have been set up to detect small changes in risk because it did not ask women about their lifetime use or factor in the chance that people might misremember their past habits. Sandler said the new study accounts for those two variables.

“This newer analysis sort of tips the balance by accounting for all these possible ways that reporting could have been incomplete in the prior literature,” she said.

How talc may have played into body shame

J&J started selling talc-based baby powder in 1894.

Although many women have used it to keep their genitals dry, there’s no need to use powder to get rid of moisture in that area, said Alexandra Scranton, director of science and research at Women’s Voices for the Earth, a nonprofit that aims to eliminate chemicals that negatively affect women’s health.

“Moisture in this part of the body is a very healthy thing,” Scranton said. “This part of the body is covered in mucous membranes. It’s supposed to be moist.”

According to O’Brien’s research, some women in the 2000s — often those in their 20s and 30s — also used talc powder on their genitals to feel clean and reduce odor. That application isn’t advised by health experts, either, since the vagina is self-cleaning and good bacteria inside of it naturally produce a slight odor.

Companies like J&J were “basically creating and promoting this myth that this part of your body — your genitals, your vagina — are inherently dirty and that they’re inherently odorous, and therefore inherently shameful,” Scranton said.

J&J said it disagrees with that characterization.

Some women continue to use baby powder on their genitals or have adopted new products like vaginal washes or scented body deodorants.

“It’s so ingrained and so part of the way they take care of their bodies that they can’t imagine not doing it,” Scranton said. “They’ve got their mom’s voice in their head: ‘This is what you do to be a respectable woman.’”

Aria Bendix is the breaking health reporter for NBC News Digital.