case study selection methodology

• Search Menu

Sign in through your institution

• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical Numismatics
• Classical Literature
• Classical Reception
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Archaeology
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Papyrology
• Late Antiquity
• Religion in the Ancient World
• Social History
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Agriculture
• History of Education
• History of Emotions
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Variation
• Language Families
• Language Acquisition
• Language Evolution
• Language Reference
• Lexicography
• Linguistic Theories
• Linguistic Typology
• Linguistic Anthropology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Modernism)
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Culture
• Music and Religion
• Music and Media
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Science
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Society
• Law and Politics
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Legal System - Costs and Funding
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Restitution
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Oncology
• Medical Toxicology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Clinical Neuroscience
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Medical Ethics
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Games
• Computer Security
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Neuroscience
• Cognitive Psychology
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business History
• Business Strategy
• Business Ethics
• Business and Government
• Business and Technology
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Social Issues in Business and Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic Methodology
• Economic Systems
• Economic History
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Management of Land and Natural Resources (Social Science)
• Natural Disasters (Environment)
• Pollution and Threats to the Environment (Social Science)
• Social Impact of Environmental Issues (Social Science)
• Sustainability
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• Ethnic Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Theory
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Behaviour
• Political Economy
• Political Institutions
• Politics and Law
• Politics of Development
• Public Administration
• Public Policy
• Qualitative Political Methodology
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Disability Studies
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

28 Case Selection for Case‐Study Analysis: Qualitative and Quantitative Techniques

John Gerring is Professor of Political Science, Boston University.

• Published: 02 September 2009
• Cite Icon Cite
• Permissions Icon Permissions

This article presents some guidance by cataloging nine different techniques for case selection: typical, diverse, extreme, deviant, influential, crucial, pathway, most similar, and most different. It also indicates that if the researcher is starting from a quantitative database, then methods for finding influential outliers can be used. In particular, the article clarifies the general principles that might guide the process of case selection in case-study research. Cases are more or less representative of some broader phenomenon and, on that score, may be considered better or worse subjects for intensive analysis. The article then draws attention to two ambiguities in case-selection strategies in case-study research. The first concerns the admixture of several case-selection strategies. The second concerns the changing status of a case as a study proceeds. Some case studies follow only one strategy of case selection.

Personal account

• Sign in with email/username & password
• Get email alerts
• Save searches
• Purchase content
• Activate your purchase/trial code
• Add your ORCID iD

Institutional access

Sign in with a library card.

• Sign in with username/password
• Recommend to your librarian
• Institutional account management
• Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.

• Click Sign in through your institution.
• Select your institution from the list provided, which will take you to your institution's website to sign in.
• When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
• Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:

• Click Sign in through society site.
• When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

• View your signed in personal account and access account management features.
• View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Our books are available by subscription or purchase to libraries and institutions.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Rights and permissions
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• Open access
• Published: 27 June 2011

The case study approach

• Sarah Crowe 1 ,
• Kathrin Cresswell 2 ,
• Ann Robertson 2 ,
• Guro Huby 3 ,
• Anthony Avery 1 &
• Aziz Sheikh 2  

BMC Medical Research Methodology volume  11 , Article number:  100 ( 2011 ) Cite this article

795k Accesses

1104 Citations

42 Altmetric

Metrics details

The case study approach allows in-depth, multi-faceted explorations of complex issues in their real-life settings. The value of the case study approach is well recognised in the fields of business, law and policy, but somewhat less so in health services research. Based on our experiences of conducting several health-related case studies, we reflect on the different types of case study design, the specific research questions this approach can help answer, the data sources that tend to be used, and the particular advantages and disadvantages of employing this methodological approach. The paper concludes with key pointers to aid those designing and appraising proposals for conducting case study research, and a checklist to help readers assess the quality of case study reports.

Peer Review reports

Introduction

The case study approach is particularly useful to employ when there is a need to obtain an in-depth appreciation of an issue, event or phenomenon of interest, in its natural real-life context. Our aim in writing this piece is to provide insights into when to consider employing this approach and an overview of key methodological considerations in relation to the design, planning, analysis, interpretation and reporting of case studies.

The illustrative 'grand round', 'case report' and 'case series' have a long tradition in clinical practice and research. Presenting detailed critiques, typically of one or more patients, aims to provide insights into aspects of the clinical case and, in doing so, illustrate broader lessons that may be learnt. In research, the conceptually-related case study approach can be used, for example, to describe in detail a patient's episode of care, explore professional attitudes to and experiences of a new policy initiative or service development or more generally to 'investigate contemporary phenomena within its real-life context' [ 1 ]. Based on our experiences of conducting a range of case studies, we reflect on when to consider using this approach, discuss the key steps involved and illustrate, with examples, some of the practical challenges of attaining an in-depth understanding of a 'case' as an integrated whole. In keeping with previously published work, we acknowledge the importance of theory to underpin the design, selection, conduct and interpretation of case studies[ 2 ]. In so doing, we make passing reference to the different epistemological approaches used in case study research by key theoreticians and methodologists in this field of enquiry.

This paper is structured around the following main questions: What is a case study? What are case studies used for? How are case studies conducted? What are the potential pitfalls and how can these be avoided? We draw in particular on four of our own recently published examples of case studies (see Tables 1 , 2 , 3 and 4 ) and those of others to illustrate our discussion[ 3 – 7 ].

What is a case study?

A case study is a research approach that is used to generate an in-depth, multi-faceted understanding of a complex issue in its real-life context. It is an established research design that is used extensively in a wide variety of disciplines, particularly in the social sciences. A case study can be defined in a variety of ways (Table 5 ), the central tenet being the need to explore an event or phenomenon in depth and in its natural context. It is for this reason sometimes referred to as a "naturalistic" design; this is in contrast to an "experimental" design (such as a randomised controlled trial) in which the investigator seeks to exert control over and manipulate the variable(s) of interest.

Stake's work has been particularly influential in defining the case study approach to scientific enquiry. He has helpfully characterised three main types of case study: intrinsic , instrumental and collective [ 8 ]. An intrinsic case study is typically undertaken to learn about a unique phenomenon. The researcher should define the uniqueness of the phenomenon, which distinguishes it from all others. In contrast, the instrumental case study uses a particular case (some of which may be better than others) to gain a broader appreciation of an issue or phenomenon. The collective case study involves studying multiple cases simultaneously or sequentially in an attempt to generate a still broader appreciation of a particular issue.

These are however not necessarily mutually exclusive categories. In the first of our examples (Table 1 ), we undertook an intrinsic case study to investigate the issue of recruitment of minority ethnic people into the specific context of asthma research studies, but it developed into a instrumental case study through seeking to understand the issue of recruitment of these marginalised populations more generally, generating a number of the findings that are potentially transferable to other disease contexts[ 3 ]. In contrast, the other three examples (see Tables 2 , 3 and 4 ) employed collective case study designs to study the introduction of workforce reconfiguration in primary care, the implementation of electronic health records into hospitals, and to understand the ways in which healthcare students learn about patient safety considerations[ 4 – 6 ]. Although our study focusing on the introduction of General Practitioners with Specialist Interests (Table 2 ) was explicitly collective in design (four contrasting primary care organisations were studied), is was also instrumental in that this particular professional group was studied as an exemplar of the more general phenomenon of workforce redesign[ 4 ].

What are case studies used for?

According to Yin, case studies can be used to explain, describe or explore events or phenomena in the everyday contexts in which they occur[ 1 ]. These can, for example, help to understand and explain causal links and pathways resulting from a new policy initiative or service development (see Tables 2 and 3 , for example)[ 1 ]. In contrast to experimental designs, which seek to test a specific hypothesis through deliberately manipulating the environment (like, for example, in a randomised controlled trial giving a new drug to randomly selected individuals and then comparing outcomes with controls),[ 9 ] the case study approach lends itself well to capturing information on more explanatory ' how ', 'what' and ' why ' questions, such as ' how is the intervention being implemented and received on the ground?'. The case study approach can offer additional insights into what gaps exist in its delivery or why one implementation strategy might be chosen over another. This in turn can help develop or refine theory, as shown in our study of the teaching of patient safety in undergraduate curricula (Table 4 )[ 6 , 10 ]. Key questions to consider when selecting the most appropriate study design are whether it is desirable or indeed possible to undertake a formal experimental investigation in which individuals and/or organisations are allocated to an intervention or control arm? Or whether the wish is to obtain a more naturalistic understanding of an issue? The former is ideally studied using a controlled experimental design, whereas the latter is more appropriately studied using a case study design.

Case studies may be approached in different ways depending on the epistemological standpoint of the researcher, that is, whether they take a critical (questioning one's own and others' assumptions), interpretivist (trying to understand individual and shared social meanings) or positivist approach (orientating towards the criteria of natural sciences, such as focusing on generalisability considerations) (Table 6 ). Whilst such a schema can be conceptually helpful, it may be appropriate to draw on more than one approach in any case study, particularly in the context of conducting health services research. Doolin has, for example, noted that in the context of undertaking interpretative case studies, researchers can usefully draw on a critical, reflective perspective which seeks to take into account the wider social and political environment that has shaped the case[ 11 ].

How are case studies conducted?

Here, we focus on the main stages of research activity when planning and undertaking a case study; the crucial stages are: defining the case; selecting the case(s); collecting and analysing the data; interpreting data; and reporting the findings.

Defining the case

Carefully formulated research question(s), informed by the existing literature and a prior appreciation of the theoretical issues and setting(s), are all important in appropriately and succinctly defining the case[ 8 , 12 ]. Crucially, each case should have a pre-defined boundary which clarifies the nature and time period covered by the case study (i.e. its scope, beginning and end), the relevant social group, organisation or geographical area of interest to the investigator, the types of evidence to be collected, and the priorities for data collection and analysis (see Table 7 )[ 1 ]. A theory driven approach to defining the case may help generate knowledge that is potentially transferable to a range of clinical contexts and behaviours; using theory is also likely to result in a more informed appreciation of, for example, how and why interventions have succeeded or failed[ 13 ].

For example, in our evaluation of the introduction of electronic health records in English hospitals (Table 3 ), we defined our cases as the NHS Trusts that were receiving the new technology[ 5 ]. Our focus was on how the technology was being implemented. However, if the primary research interest had been on the social and organisational dimensions of implementation, we might have defined our case differently as a grouping of healthcare professionals (e.g. doctors and/or nurses). The precise beginning and end of the case may however prove difficult to define. Pursuing this same example, when does the process of implementation and adoption of an electronic health record system really begin or end? Such judgements will inevitably be influenced by a range of factors, including the research question, theory of interest, the scope and richness of the gathered data and the resources available to the research team.

Selecting the case(s)

The decision on how to select the case(s) to study is a very important one that merits some reflection. In an intrinsic case study, the case is selected on its own merits[ 8 ]. The case is selected not because it is representative of other cases, but because of its uniqueness, which is of genuine interest to the researchers. This was, for example, the case in our study of the recruitment of minority ethnic participants into asthma research (Table 1 ) as our earlier work had demonstrated the marginalisation of minority ethnic people with asthma, despite evidence of disproportionate asthma morbidity[ 14 , 15 ]. In another example of an intrinsic case study, Hellstrom et al.[ 16 ] studied an elderly married couple living with dementia to explore how dementia had impacted on their understanding of home, their everyday life and their relationships.

For an instrumental case study, selecting a "typical" case can work well[ 8 ]. In contrast to the intrinsic case study, the particular case which is chosen is of less importance than selecting a case that allows the researcher to investigate an issue or phenomenon. For example, in order to gain an understanding of doctors' responses to health policy initiatives, Som undertook an instrumental case study interviewing clinicians who had a range of responsibilities for clinical governance in one NHS acute hospital trust[ 17 ]. Sampling a "deviant" or "atypical" case may however prove even more informative, potentially enabling the researcher to identify causal processes, generate hypotheses and develop theory.

In collective or multiple case studies, a number of cases are carefully selected. This offers the advantage of allowing comparisons to be made across several cases and/or replication. Choosing a "typical" case may enable the findings to be generalised to theory (i.e. analytical generalisation) or to test theory by replicating the findings in a second or even a third case (i.e. replication logic)[ 1 ]. Yin suggests two or three literal replications (i.e. predicting similar results) if the theory is straightforward and five or more if the theory is more subtle. However, critics might argue that selecting 'cases' in this way is insufficiently reflexive and ill-suited to the complexities of contemporary healthcare organisations.

The selected case study site(s) should allow the research team access to the group of individuals, the organisation, the processes or whatever else constitutes the chosen unit of analysis for the study. Access is therefore a central consideration; the researcher needs to come to know the case study site(s) well and to work cooperatively with them. Selected cases need to be not only interesting but also hospitable to the inquiry [ 8 ] if they are to be informative and answer the research question(s). Case study sites may also be pre-selected for the researcher, with decisions being influenced by key stakeholders. For example, our selection of case study sites in the evaluation of the implementation and adoption of electronic health record systems (see Table 3 ) was heavily influenced by NHS Connecting for Health, the government agency that was responsible for overseeing the National Programme for Information Technology (NPfIT)[ 5 ]. This prominent stakeholder had already selected the NHS sites (through a competitive bidding process) to be early adopters of the electronic health record systems and had negotiated contracts that detailed the deployment timelines.

It is also important to consider in advance the likely burden and risks associated with participation for those who (or the site(s) which) comprise the case study. Of particular importance is the obligation for the researcher to think through the ethical implications of the study (e.g. the risk of inadvertently breaching anonymity or confidentiality) and to ensure that potential participants/participating sites are provided with sufficient information to make an informed choice about joining the study. The outcome of providing this information might be that the emotive burden associated with participation, or the organisational disruption associated with supporting the fieldwork, is considered so high that the individuals or sites decide against participation.

In our example of evaluating implementations of electronic health record systems, given the restricted number of early adopter sites available to us, we sought purposively to select a diverse range of implementation cases among those that were available[ 5 ]. We chose a mixture of teaching, non-teaching and Foundation Trust hospitals, and examples of each of the three electronic health record systems procured centrally by the NPfIT. At one recruited site, it quickly became apparent that access was problematic because of competing demands on that organisation. Recognising the importance of full access and co-operative working for generating rich data, the research team decided not to pursue work at that site and instead to focus on other recruited sites.

Collecting the data

In order to develop a thorough understanding of the case, the case study approach usually involves the collection of multiple sources of evidence, using a range of quantitative (e.g. questionnaires, audits and analysis of routinely collected healthcare data) and more commonly qualitative techniques (e.g. interviews, focus groups and observations). The use of multiple sources of data (data triangulation) has been advocated as a way of increasing the internal validity of a study (i.e. the extent to which the method is appropriate to answer the research question)[ 8 , 18 – 21 ]. An underlying assumption is that data collected in different ways should lead to similar conclusions, and approaching the same issue from different angles can help develop a holistic picture of the phenomenon (Table 2 )[ 4 ].

Brazier and colleagues used a mixed-methods case study approach to investigate the impact of a cancer care programme[ 22 ]. Here, quantitative measures were collected with questionnaires before, and five months after, the start of the intervention which did not yield any statistically significant results. Qualitative interviews with patients however helped provide an insight into potentially beneficial process-related aspects of the programme, such as greater, perceived patient involvement in care. The authors reported how this case study approach provided a number of contextual factors likely to influence the effectiveness of the intervention and which were not likely to have been obtained from quantitative methods alone.

In collective or multiple case studies, data collection needs to be flexible enough to allow a detailed description of each individual case to be developed (e.g. the nature of different cancer care programmes), before considering the emerging similarities and differences in cross-case comparisons (e.g. to explore why one programme is more effective than another). It is important that data sources from different cases are, where possible, broadly comparable for this purpose even though they may vary in nature and depth.

Analysing, interpreting and reporting case studies

Making sense and offering a coherent interpretation of the typically disparate sources of data (whether qualitative alone or together with quantitative) is far from straightforward. Repeated reviewing and sorting of the voluminous and detail-rich data are integral to the process of analysis. In collective case studies, it is helpful to analyse data relating to the individual component cases first, before making comparisons across cases. Attention needs to be paid to variations within each case and, where relevant, the relationship between different causes, effects and outcomes[ 23 ]. Data will need to be organised and coded to allow the key issues, both derived from the literature and emerging from the dataset, to be easily retrieved at a later stage. An initial coding frame can help capture these issues and can be applied systematically to the whole dataset with the aid of a qualitative data analysis software package.

The Framework approach is a practical approach, comprising of five stages (familiarisation; identifying a thematic framework; indexing; charting; mapping and interpretation) , to managing and analysing large datasets particularly if time is limited, as was the case in our study of recruitment of South Asians into asthma research (Table 1 )[ 3 , 24 ]. Theoretical frameworks may also play an important role in integrating different sources of data and examining emerging themes. For example, we drew on a socio-technical framework to help explain the connections between different elements - technology; people; and the organisational settings within which they worked - in our study of the introduction of electronic health record systems (Table 3 )[ 5 ]. Our study of patient safety in undergraduate curricula drew on an evaluation-based approach to design and analysis, which emphasised the importance of the academic, organisational and practice contexts through which students learn (Table 4 )[ 6 ].

Case study findings can have implications both for theory development and theory testing. They may establish, strengthen or weaken historical explanations of a case and, in certain circumstances, allow theoretical (as opposed to statistical) generalisation beyond the particular cases studied[ 12 ]. These theoretical lenses should not, however, constitute a strait-jacket and the cases should not be "forced to fit" the particular theoretical framework that is being employed.

When reporting findings, it is important to provide the reader with enough contextual information to understand the processes that were followed and how the conclusions were reached. In a collective case study, researchers may choose to present the findings from individual cases separately before amalgamating across cases. Care must be taken to ensure the anonymity of both case sites and individual participants (if agreed in advance) by allocating appropriate codes or withholding descriptors. In the example given in Table 3 , we decided against providing detailed information on the NHS sites and individual participants in order to avoid the risk of inadvertent disclosure of identities[ 5 , 25 ].

What are the potential pitfalls and how can these be avoided?

The case study approach is, as with all research, not without its limitations. When investigating the formal and informal ways undergraduate students learn about patient safety (Table 4 ), for example, we rapidly accumulated a large quantity of data. The volume of data, together with the time restrictions in place, impacted on the depth of analysis that was possible within the available resources. This highlights a more general point of the importance of avoiding the temptation to collect as much data as possible; adequate time also needs to be set aside for data analysis and interpretation of what are often highly complex datasets.

Case study research has sometimes been criticised for lacking scientific rigour and providing little basis for generalisation (i.e. producing findings that may be transferable to other settings)[ 1 ]. There are several ways to address these concerns, including: the use of theoretical sampling (i.e. drawing on a particular conceptual framework); respondent validation (i.e. participants checking emerging findings and the researcher's interpretation, and providing an opinion as to whether they feel these are accurate); and transparency throughout the research process (see Table 8 )[ 8 , 18 – 21 , 23 , 26 ]. Transparency can be achieved by describing in detail the steps involved in case selection, data collection, the reasons for the particular methods chosen, and the researcher's background and level of involvement (i.e. being explicit about how the researcher has influenced data collection and interpretation). Seeking potential, alternative explanations, and being explicit about how interpretations and conclusions were reached, help readers to judge the trustworthiness of the case study report. Stake provides a critique checklist for a case study report (Table 9 )[ 8 ].

Conclusions

The case study approach allows, amongst other things, critical events, interventions, policy developments and programme-based service reforms to be studied in detail in a real-life context. It should therefore be considered when an experimental design is either inappropriate to answer the research questions posed or impossible to undertake. Considering the frequency with which implementations of innovations are now taking place in healthcare settings and how well the case study approach lends itself to in-depth, complex health service research, we believe this approach should be more widely considered by researchers. Though inherently challenging, the research case study can, if carefully conceptualised and thoughtfully undertaken and reported, yield powerful insights into many important aspects of health and healthcare delivery.

Yin RK: Case study research, design and method. 2009, London: Sage Publications Ltd., 4

Google Scholar  

Keen J, Packwood T: Qualitative research; case study evaluation. BMJ. 1995, 311: 444-446.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Sheikh A, Halani L, Bhopal R, Netuveli G, Partridge M, Car J, et al: Facilitating the Recruitment of Minority Ethnic People into Research: Qualitative Case Study of South Asians and Asthma. PLoS Med. 2009, 6 (10): 1-11.

Article   Google Scholar  

Pinnock H, Huby G, Powell A, Kielmann T, Price D, Williams S, et al: The process of planning, development and implementation of a General Practitioner with a Special Interest service in Primary Care Organisations in England and Wales: a comparative prospective case study. Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO). 2008, [ http://www.sdo.nihr.ac.uk/files/project/99-final-report.pdf ]

Robertson A, Cresswell K, Takian A, Petrakaki D, Crowe S, Cornford T, et al: Prospective evaluation of the implementation and adoption of NHS Connecting for Health's national electronic health record in secondary care in England: interim findings. BMJ. 2010, 41: c4564-

Pearson P, Steven A, Howe A, Sheikh A, Ashcroft D, Smith P, the Patient Safety Education Study Group: Learning about patient safety: organisational context and culture in the education of healthcare professionals. J Health Serv Res Policy. 2010, 15: 4-10. 10.1258/jhsrp.2009.009052.

Article   PubMed   Google Scholar  

van Harten WH, Casparie TF, Fisscher OA: The evaluation of the introduction of a quality management system: a process-oriented case study in a large rehabilitation hospital. Health Policy. 2002, 60 (1): 17-37. 10.1016/S0168-8510(01)00187-7.

Stake RE: The art of case study research. 1995, London: Sage Publications Ltd.

Sheikh A, Smeeth L, Ashcroft R: Randomised controlled trials in primary care: scope and application. Br J Gen Pract. 2002, 52 (482): 746-51.

PubMed   PubMed Central   Google Scholar  

King G, Keohane R, Verba S: Designing Social Inquiry. 1996, Princeton: Princeton University Press

Doolin B: Information technology as disciplinary technology: being critical in interpretative research on information systems. Journal of Information Technology. 1998, 13: 301-311. 10.1057/jit.1998.8.

George AL, Bennett A: Case studies and theory development in the social sciences. 2005, Cambridge, MA: MIT Press

Eccles M, the Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG): Designing theoretically-informed implementation interventions. Implementation Science. 2006, 1: 1-8. 10.1186/1748-5908-1-1.

Article   PubMed Central   Google Scholar  

Netuveli G, Hurwitz B, Levy M, Fletcher M, Barnes G, Durham SR, Sheikh A: Ethnic variations in UK asthma frequency, morbidity, and health-service use: a systematic review and meta-analysis. Lancet. 2005, 365 (9456): 312-7.

Sheikh A, Panesar SS, Lasserson T, Netuveli G: Recruitment of ethnic minorities to asthma studies. Thorax. 2004, 59 (7): 634-

CAS   PubMed   PubMed Central   Google Scholar  

Hellström I, Nolan M, Lundh U: 'We do things together': A case study of 'couplehood' in dementia. Dementia. 2005, 4: 7-22. 10.1177/1471301205049188.

Som CV: Nothing seems to have changed, nothing seems to be changing and perhaps nothing will change in the NHS: doctors' response to clinical governance. International Journal of Public Sector Management. 2005, 18: 463-477. 10.1108/09513550510608903.

Lincoln Y, Guba E: Naturalistic inquiry. 1985, Newbury Park: Sage Publications

Barbour RS: Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?. BMJ. 2001, 322: 1115-1117. 10.1136/bmj.322.7294.1115.

Mays N, Pope C: Qualitative research in health care: Assessing quality in qualitative research. BMJ. 2000, 320: 50-52. 10.1136/bmj.320.7226.50.

Mason J: Qualitative researching. 2002, London: Sage

Brazier A, Cooke K, Moravan V: Using Mixed Methods for Evaluating an Integrative Approach to Cancer Care: A Case Study. Integr Cancer Ther. 2008, 7: 5-17. 10.1177/1534735407313395.

Miles MB, Huberman M: Qualitative data analysis: an expanded sourcebook. 1994, CA: Sage Publications Inc., 2

Pope C, Ziebland S, Mays N: Analysing qualitative data. Qualitative research in health care. BMJ. 2000, 320: 114-116. 10.1136/bmj.320.7227.114.

Cresswell KM, Worth A, Sheikh A: Actor-Network Theory and its role in understanding the implementation of information technology developments in healthcare. BMC Med Inform Decis Mak. 2010, 10 (1): 67-10.1186/1472-6947-10-67.

Article   PubMed   PubMed Central   Google Scholar  

Malterud K: Qualitative research: standards, challenges, and guidelines. Lancet. 2001, 358: 483-488. 10.1016/S0140-6736(01)05627-6.

Article   CAS   PubMed   Google Scholar  

Yin R: Case study research: design and methods. 1994, Thousand Oaks, CA: Sage Publishing, 2

Yin R: Enhancing the quality of case studies in health services research. Health Serv Res. 1999, 34: 1209-1224.

Green J, Thorogood N: Qualitative methods for health research. 2009, Los Angeles: Sage, 2

Howcroft D, Trauth E: Handbook of Critical Information Systems Research, Theory and Application. 2005, Cheltenham, UK: Northampton, MA, USA: Edward Elgar

Book   Google Scholar  

Blakie N: Approaches to Social Enquiry. 1993, Cambridge: Polity Press

Doolin B: Power and resistance in the implementation of a medical management information system. Info Systems J. 2004, 14: 343-362. 10.1111/j.1365-2575.2004.00176.x.

Bloomfield BP, Best A: Management consultants: systems development, power and the translation of problems. Sociological Review. 1992, 40: 533-560.

Shanks G, Parr A: Positivist, single case study research in information systems: A critical analysis. Proceedings of the European Conference on Information Systems. 2003, Naples

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/11/100/prepub

Download references

Acknowledgements

We are grateful to the participants and colleagues who contributed to the individual case studies that we have drawn on. This work received no direct funding, but it has been informed by projects funded by Asthma UK, the NHS Service Delivery Organisation, NHS Connecting for Health Evaluation Programme, and Patient Safety Research Portfolio. We would also like to thank the expert reviewers for their insightful and constructive feedback. Our thanks are also due to Dr. Allison Worth who commented on an earlier draft of this manuscript.

Author information

Authors and affiliations.

Division of Primary Care, The University of Nottingham, Nottingham, UK

Sarah Crowe & Anthony Avery

Centre for Population Health Sciences, The University of Edinburgh, Edinburgh, UK

Kathrin Cresswell, Ann Robertson & Aziz Sheikh

School of Health in Social Science, The University of Edinburgh, Edinburgh, UK

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sarah Crowe .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors' contributions

AS conceived this article. SC, KC and AR wrote this paper with GH, AA and AS all commenting on various drafts. SC and AS are guarantors.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Crowe, S., Cresswell, K., Robertson, A. et al. The case study approach. BMC Med Res Methodol 11 , 100 (2011). https://doi.org/10.1186/1471-2288-11-100

Download citation

Received : 29 November 2010

Accepted : 27 June 2011

Published : 27 June 2011

DOI : https://doi.org/10.1186/1471-2288-11-100

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Case Study Approach
• Electronic Health Record System
• Case Study Design
• Case Study Site
• Case Study Report

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Qual Stud Health Well-being

Methodology or method? A critical review of qualitative case study reports

Despite on-going debate about credibility, and reported limitations in comparison to other approaches, case study is an increasingly popular approach among qualitative researchers. We critically analysed the methodological descriptions of published case studies. Three high-impact qualitative methods journals were searched to locate case studies published in the past 5 years; 34 were selected for analysis. Articles were categorized as health and health services ( n= 12), social sciences and anthropology ( n= 7), or methods ( n= 15) case studies. The articles were reviewed using an adapted version of established criteria to determine whether adequate methodological justification was present, and if study aims, methods, and reported findings were consistent with a qualitative case study approach. Findings were grouped into five themes outlining key methodological issues: case study methodology or method, case of something particular and case selection, contextually bound case study, researcher and case interactions and triangulation, and study design inconsistent with methodology reported. Improved reporting of case studies by qualitative researchers will advance the methodology for the benefit of researchers and practitioners.

Case study research is an increasingly popular approach among qualitative researchers (Thomas, 2011 ). Several prominent authors have contributed to methodological developments, which has increased the popularity of case study approaches across disciplines (Creswell, 2013b ; Denzin & Lincoln, 2011b ; Merriam, 2009 ; Ragin & Becker, 1992 ; Stake, 1995 ; Yin, 2009 ). Current qualitative case study approaches are shaped by paradigm, study design, and selection of methods, and, as a result, case studies in the published literature vary. Differences between published case studies can make it difficult for researchers to define and understand case study as a methodology.

Experienced qualitative researchers have identified case study research as a stand-alone qualitative approach (Denzin & Lincoln, 2011b ). Case study research has a level of flexibility that is not readily offered by other qualitative approaches such as grounded theory or phenomenology. Case studies are designed to suit the case and research question and published case studies demonstrate wide diversity in study design. There are two popular case study approaches in qualitative research. The first, proposed by Stake ( 1995 ) and Merriam ( 2009 ), is situated in a social constructivist paradigm, whereas the second, by Yin ( 2012 ), Flyvbjerg ( 2011 ), and Eisenhardt ( 1989 ), approaches case study from a post-positivist viewpoint. Scholarship from both schools of inquiry has contributed to the popularity of case study and development of theoretical frameworks and principles that characterize the methodology.

The diversity of case studies reported in the published literature, and on-going debates about credibility and the use of case study in qualitative research practice, suggests that differences in perspectives on case study methodology may prevent researchers from developing a mutual understanding of practice and rigour. In addition, discussion about case study limitations has led some authors to query whether case study is indeed a methodology (Luck, Jackson, & Usher, 2006 ; Meyer, 2001 ; Thomas, 2010 ; Tight, 2010 ). Methodological discussion of qualitative case study research is timely, and a review is required to analyse and understand how this methodology is applied in the qualitative research literature. The aims of this study were to review methodological descriptions of published qualitative case studies, to review how the case study methodological approach was applied, and to identify issues that need to be addressed by researchers, editors, and reviewers. An outline of the current definitions of case study and an overview of the issues proposed in the qualitative methodological literature are provided to set the scene for the review.

Definitions of qualitative case study research

Case study research is an investigation and analysis of a single or collective case, intended to capture the complexity of the object of study (Stake, 1995 ). Qualitative case study research, as described by Stake ( 1995 ), draws together “naturalistic, holistic, ethnographic, phenomenological, and biographic research methods” in a bricoleur design, or in his words, “a palette of methods” (Stake, 1995 , pp. xi–xii). Case study methodology maintains deep connections to core values and intentions and is “particularistic, descriptive and heuristic” (Merriam, 2009 , p. 46).

As a study design, case study is defined by interest in individual cases rather than the methods of inquiry used. The selection of methods is informed by researcher and case intuition and makes use of naturally occurring sources of knowledge, such as people or observations of interactions that occur in the physical space (Stake, 1998 ). Thomas ( 2011 ) suggested that “analytical eclecticism” is a defining factor (p. 512). Multiple data collection and analysis methods are adopted to further develop and understand the case, shaped by context and emergent data (Stake, 1995 ). This qualitative approach “explores a real-life, contemporary bounded system (a case ) or multiple bounded systems (cases) over time, through detailed, in-depth data collection involving multiple sources of information … and reports a case description and case themes ” (Creswell, 2013b , p. 97). Case study research has been defined by the unit of analysis, the process of study, and the outcome or end product, all essentially the case (Merriam, 2009 ).

The case is an object to be studied for an identified reason that is peculiar or particular. Classification of the case and case selection procedures informs development of the study design and clarifies the research question. Stake ( 1995 ) proposed three types of cases and study design frameworks. These include the intrinsic case, the instrumental case, and the collective instrumental case. The intrinsic case is used to understand the particulars of a single case, rather than what it represents. An instrumental case study provides insight on an issue or is used to refine theory. The case is selected to advance understanding of the object of interest. A collective refers to an instrumental case which is studied as multiple, nested cases, observed in unison, parallel, or sequential order. More than one case can be simultaneously studied; however, each case study is a concentrated, single inquiry, studied holistically in its own entirety (Stake, 1995 , 1998 ).

Researchers who use case study are urged to seek out what is common and what is particular about the case. This involves careful and in-depth consideration of the nature of the case, historical background, physical setting, and other institutional and political contextual factors (Stake, 1998 ). An interpretive or social constructivist approach to qualitative case study research supports a transactional method of inquiry, where the researcher has a personal interaction with the case. The case is developed in a relationship between the researcher and informants, and presented to engage the reader, inviting them to join in this interaction and in case discovery (Stake, 1995 ). A postpositivist approach to case study involves developing a clear case study protocol with careful consideration of validity and potential bias, which might involve an exploratory or pilot phase, and ensures that all elements of the case are measured and adequately described (Yin, 2009 , 2012 ).

Current methodological issues in qualitative case study research

The future of qualitative research will be influenced and constructed by the way research is conducted, and by what is reviewed and published in academic journals (Morse, 2011 ). If case study research is to further develop as a principal qualitative methodological approach, and make a valued contribution to the field of qualitative inquiry, issues related to methodological credibility must be considered. Researchers are required to demonstrate rigour through adequate descriptions of methodological foundations. Case studies published without sufficient detail for the reader to understand the study design, and without rationale for key methodological decisions, may lead to research being interpreted as lacking in quality or credibility (Hallberg, 2013 ; Morse, 2011 ).

There is a level of artistic license that is embraced by qualitative researchers and distinguishes practice, which nurtures creativity, innovation, and reflexivity (Denzin & Lincoln, 2011b ; Morse, 2009 ). Qualitative research is “inherently multimethod” (Denzin & Lincoln, 2011a , p. 5); however, with this creative freedom, it is important for researchers to provide adequate description for methodological justification (Meyer, 2001 ). This includes paradigm and theoretical perspectives that have influenced study design. Without adequate description, study design might not be understood by the reader, and can appear to be dishonest or inaccurate. Reviewers and readers might be confused by the inconsistent or inappropriate terms used to describe case study research approach and methods, and be distracted from important study findings (Sandelowski, 2000 ). This issue extends beyond case study research, and others have noted inconsistencies in reporting of methodology and method by qualitative researchers. Sandelowski ( 2000 , 2010 ) argued for accurate identification of qualitative description as a research approach. She recommended that the selected methodology should be harmonious with the study design, and be reflected in methods and analysis techniques. Similarly, Webb and Kevern ( 2000 ) uncovered inconsistencies in qualitative nursing research with focus group methods, recommending that methodological procedures must cite seminal authors and be applied with respect to the selected theoretical framework. Incorrect labelling using case study might stem from the flexibility in case study design and non-directional character relative to other approaches (Rosenberg & Yates, 2007 ). Methodological integrity is required in design of qualitative studies, including case study, to ensure study rigour and to enhance credibility of the field (Morse, 2011 ).

Case study has been unnecessarily devalued by comparisons with statistical methods (Eisenhardt, 1989 ; Flyvbjerg, 2006 , 2011 ; Jensen & Rodgers, 2001 ; Piekkari, Welch, & Paavilainen, 2009 ; Tight, 2010 ; Yin, 1999 ). It is reputed to be the “the weak sibling” in comparison to other, more rigorous, approaches (Yin, 2009 , p. xiii). Case study is not an inherently comparative approach to research. The objective is not statistical research, and the aim is not to produce outcomes that are generalizable to all populations (Thomas, 2011 ). Comparisons between case study and statistical research do little to advance this qualitative approach, and fail to recognize its inherent value, which can be better understood from the interpretive or social constructionist viewpoint of other authors (Merriam, 2009 ; Stake, 1995 ). Building on discussions relating to “fuzzy” (Bassey, 2001 ), or naturalistic generalizations (Stake, 1978 ), or transference of concepts and theories (Ayres, Kavanaugh, & Knafl, 2003 ; Morse et al., 2011 ) would have more relevance.

Case study research has been used as a catch-all design to justify or add weight to fundamental qualitative descriptive studies that do not fit with other traditional frameworks (Merriam, 2009 ). A case study has been a “convenient label for our research—when we ‘can't think of anything ‘better”—in an attempt to give it [qualitative methodology] some added respectability” (Tight, 2010 , p. 337). Qualitative case study research is a pliable approach (Merriam, 2009 ; Meyer, 2001 ; Stake, 1995 ), and has been likened to a “curious methodological limbo” (Gerring, 2004 , p. 341) or “paradigmatic bridge” (Luck et al., 2006 , p. 104), that is on the borderline between postpositivist and constructionist interpretations. This has resulted in inconsistency in application, which indicates that flexibility comes with limitations (Meyer, 2001 ), and the open nature of case study research might be off-putting to novice researchers (Thomas, 2011 ). The development of a well-(in)formed theoretical framework to guide a case study should improve consistency, rigour, and trust in studies published in qualitative research journals (Meyer, 2001 ).

Assessment of rigour

The purpose of this study was to analyse the methodological descriptions of case studies published in qualitative methods journals. To do this we needed to develop a suitable framework, which used existing, established criteria for appraising qualitative case study research rigour (Creswell, 2013b ; Merriam, 2009 ; Stake, 1995 ). A number of qualitative authors have developed concepts and criteria that are used to determine whether a study is rigorous (Denzin & Lincoln, 2011b ; Lincoln, 1995 ; Sandelowski & Barroso, 2002 ). The criteria proposed by Stake ( 1995 ) provide a framework for readers and reviewers to make judgements regarding case study quality, and identify key characteristics essential for good methodological rigour. Although each of the factors listed in Stake's criteria could enhance the quality of a qualitative research report, in Table I we present an adapted criteria used in this study, which integrates more recent work by Merriam ( 2009 ) and Creswell ( 2013b ). Stake's ( 1995 ) original criteria were separated into two categories. The first list of general criteria is “relevant for all qualitative research.” The second list, “high relevance to qualitative case study research,” was the criteria that we decided had higher relevance to case study research. This second list was the main criteria used to assess the methodological descriptions of the case studies reviewed. The complete table has been preserved so that the reader can determine how the original criteria were adapted.

Framework for assessing quality in qualitative case study research.

Adapted from Stake ( 1995 , p. 131).

Study design

The critical review method described by Grant and Booth ( 2009 ) was used, which is appropriate for the assessment of research quality, and is used for literature analysis to inform research and practice. This type of review goes beyond the mapping and description of scoping or rapid reviews, to include “analysis and conceptual innovation” (Grant & Booth, 2009 , p. 93). A critical review is used to develop existing, or produce new, hypotheses or models. This is different to systematic reviews that answer clinical questions. It is used to evaluate existing research and competing ideas, to provide a “launch pad” for conceptual development and “subsequent testing” (Grant & Booth, 2009 , p. 93).

Qualitative methods journals were located by a search of the 2011 ISI Journal Citation Reports in Social Science, via the database Web of Knowledge (see m.webofknowledge.com). No “qualitative research methods” category existed in the citation reports; therefore, a search of all categories was performed using the term “qualitative.” In Table II , we present the qualitative methods journals located, ranked by impact factor. The highest ranked journals were selected for searching. We acknowledge that the impact factor ranking system might not be the best measure of journal quality (Cheek, Garnham, & Quan, 2006 ); however, this was the most appropriate and accessible method available.

International Journal of Qualitative Studies on Health and Well-being.

Search strategy

In March 2013, searches of the journals, Qualitative Health Research , Qualitative Research , and Qualitative Inquiry were completed to retrieve studies with “case study” in the abstract field. The search was limited to the past 5 years (1 January 2008 to 1 March 2013). The objective was to locate published qualitative case studies suitable for assessment using the adapted criterion. Viewpoints, commentaries, and other article types were excluded from review. Title and abstracts of the 45 retrieved articles were read by the first author, who identified 34 empirical case studies for review. All authors reviewed the 34 studies to confirm selection and categorization. In Table III , we present the 34 case studies grouped by journal, and categorized by research topic, including health sciences, social sciences and anthropology, and methods research. There was a discrepancy in categorization of one article on pedagogy and a new teaching method published in Qualitative Inquiry (Jorrín-Abellán, Rubia-Avi, Anguita-Martínez, Gómez-Sánchez, & Martínez-Mones, 2008 ). Consensus was to allocate to the methods category.

Outcomes of search of qualitative methods journals.

In Table III , the number of studies located, and final numbers selected for review have been reported. Qualitative Health Research published the most empirical case studies ( n= 16). In the health category, there were 12 case studies of health conditions, health services, and health policy issues, all published in Qualitative Health Research . Seven case studies were categorized as social sciences and anthropology research, which combined case study with biography and ethnography methodologies. All three journals published case studies on methods research to illustrate a data collection or analysis technique, methodological procedure, or related issue.

The methodological descriptions of 34 case studies were critically reviewed using the adapted criteria. All articles reviewed contained a description of study methods; however, the length, amount of detail, and position of the description in the article varied. Few studies provided an accurate description and rationale for using a qualitative case study approach. In the 34 case studies reviewed, three described a theoretical framework informed by Stake ( 1995 ), two by Yin ( 2009 ), and three provided a mixed framework informed by various authors, which might have included both Yin and Stake. Few studies described their case study design, or included a rationale that explained why they excluded or added further procedures, and whether this was to enhance the study design, or to better suit the research question. In 26 of the studies no reference was provided to principal case study authors. From reviewing the description of methods, few authors provided a description or justification of case study methodology that demonstrated how their study was informed by the methodological literature that exists on this approach.

The methodological descriptions of each study were reviewed using the adapted criteria, and the following issues were identified: case study methodology or method; case of something particular and case selection; contextually bound case study; researcher and case interactions and triangulation; and, study design inconsistent with methodology. An outline of how the issues were developed from the critical review is provided, followed by a discussion of how these relate to the current methodological literature.

Case study methodology or method

A third of the case studies reviewed appeared to use a case report method, not case study methodology as described by principal authors (Creswell, 2013b ; Merriam, 2009 ; Stake, 1995 ; Yin, 2009 ). Case studies were identified as a case report because of missing methodological detail and by review of the study aims and purpose. These reports presented data for small samples of no more than three people, places or phenomenon. Four studies, or “case reports” were single cases selected retrospectively from larger studies (Bronken, Kirkevold, Martinsen, & Kvigne, 2012 ; Coltart & Henwood, 2012 ; Hooghe, Neimeyer, & Rober, 2012 ; Roscigno et al., 2012 ). Case reports were not a case of something, instead were a case demonstration or an example presented in a report. These reports presented outcomes, and reported on how the case could be generalized. Descriptions focussed on the phenomena, rather than the case itself, and did not appear to study the case in its entirety.

Case reports had minimal in-text references to case study methodology, and were informed by other qualitative traditions or secondary sources (Adamson & Holloway, 2012 ; Buzzanell & D'Enbeau, 2009 ; Nagar-Ron & Motzafi-Haller, 2011 ). This does not suggest that case study methodology cannot be multimethod, however, methodology should be consistent in design, be clearly described (Meyer, 2001 ; Stake, 1995 ), and maintain focus on the case (Creswell, 2013b ).

To demonstrate how case reports were identified, three examples are provided. The first, Yeh ( 2013 ) described their study as, “the examination of the emergence of vegetarianism in Victorian England serves as a case study to reveal the relationships between boundaries and entities” (p. 306). The findings were a historical case report, which resulted from an ethnographic study of vegetarianism. Cunsolo Willox, Harper, Edge, ‘My Word’: Storytelling and Digital Media Lab, and Rigolet Inuit Community Government (2013) used “a case study that illustrates the usage of digital storytelling within an Inuit community” (p. 130). This case study reported how digital storytelling can be used with indigenous communities as a participatory method to illuminate the benefits of this method for other studies. This “case study was conducted in the Inuit community” but did not include the Inuit community in case analysis (Cunsolo Willox et al., 2013 , p. 130). Bronken et al. ( 2012 ) provided a single case report to demonstrate issues observed in a larger clinical study of aphasia and stroke, without adequate case description or analysis.

Case study of something particular and case selection

Case selection is a precursor to case analysis, which needs to be presented as a convincing argument (Merriam, 2009 ). Descriptions of the case were often not adequate to ascertain why the case was selected, or whether it was a particular exemplar or outlier (Thomas, 2011 ). In a number of case studies in the health and social science categories, it was not explicit whether the case was of something particular, or peculiar to their discipline or field (Adamson & Holloway, 2012 ; Bronken et al., 2012 ; Colón-Emeric et al., 2010 ; Jackson, Botelho, Welch, Joseph, & Tennstedt, 2012 ; Mawn et al., 2010 ; Snyder-Young, 2011 ). There were exceptions in the methods category ( Table III ), where cases were selected by researchers to report on a new or innovative method. The cases emerged through heuristic study, and were reported to be particular, relative to the existing methods literature (Ajodhia-Andrews & Berman, 2009 ; Buckley & Waring, 2013 ; Cunsolo Willox et al., 2013 ; De Haene, Grietens, & Verschueren, 2010 ; Gratton & O'Donnell, 2011 ; Sumsion, 2013 ; Wimpenny & Savin-Baden, 2012 ).

Case selection processes were sometimes insufficient to understand why the case was selected from the global population of cases, or what study of this case would contribute to knowledge as compared with other possible cases (Adamson & Holloway, 2012 ; Bronken et al., 2012 ; Colón-Emeric et al., 2010 ; Jackson et al., 2012 ; Mawn et al., 2010 ). In two studies, local cases were selected (Barone, 2010 ; Fourie & Theron, 2012 ) because the researcher was familiar with and had access to the case. Possible limitations of a convenience sample were not acknowledged. Purposeful sampling was used to recruit participants within the case of one study, but not of the case itself (Gallagher et al., 2013 ). Random sampling was completed for case selection in two studies (Colón-Emeric et al., 2010 ; Jackson et al., 2012 ), which has limited meaning in interpretive qualitative research.

To demonstrate how researchers provided a good justification for the selection of case study approaches, four examples are provided. The first, cases of residential care homes, were selected because of reported occurrences of mistreatment, which included residents being locked in rooms at night (Rytterström, Unosson, & Arman, 2013 ). Roscigno et al. ( 2012 ) selected cases of parents who were admitted for early hospitalization in neonatal intensive care with a threatened preterm delivery before 26 weeks. Hooghe et al. ( 2012 ) used random sampling to select 20 couples that had experienced the death of a child; however, the case study was of one couple and a particular metaphor described only by them. The final example, Coltart and Henwood ( 2012 ), provided a detailed account of how they selected two cases from a sample of 46 fathers based on personal characteristics and beliefs. They described how the analysis of the two cases would contribute to their larger study on first time fathers and parenting.

Contextually bound case study

The limits or boundaries of the case are a defining factor of case study methodology (Merriam, 2009 ; Ragin & Becker, 1992 ; Stake, 1995 ; Yin, 2009 ). Adequate contextual description is required to understand the setting or context in which the case is revealed. In the health category, case studies were used to illustrate a clinical phenomenon or issue such as compliance and health behaviour (Colón-Emeric et al., 2010 ; D'Enbeau, Buzzanell, & Duckworth, 2010 ; Gallagher et al., 2013 ; Hooghe et al., 2012 ; Jackson et al., 2012 ; Roscigno et al., 2012 ). In these case studies, contextual boundaries, such as physical and institutional descriptions, were not sufficient to understand the case as a holistic system, for example, the general practitioner (GP) clinic in Gallagher et al. ( 2013 ), or the nursing home in Colón-Emeric et al. ( 2010 ). Similarly, in the social science and methods categories, attention was paid to some components of the case context, but not others, missing important information required to understand the case as a holistic system (Alexander, Moreira, & Kumar, 2012 ; Buzzanell & D'Enbeau, 2009 ; Nairn & Panelli, 2009 ; Wimpenny & Savin-Baden, 2012 ).

In two studies, vicarious experience or vignettes (Nairn & Panelli, 2009 ) and images (Jorrín-Abellán et al., 2008 ) were effective to support description of context, and might have been a useful addition for other case studies. Missing contextual boundaries suggests that the case might not be adequately defined. Additional information, such as the physical, institutional, political, and community context, would improve understanding of the case (Stake, 1998 ). In Boxes 1 and 2 , we present brief synopses of two studies that were reviewed, which demonstrated a well bounded case. In Box 1 , Ledderer ( 2011 ) used a qualitative case study design informed by Stake's tradition. In Box 2 , Gillard, Witt, and Watts ( 2011 ) were informed by Yin's tradition. By providing a brief outline of the case studies in Boxes 1 and 2 , we demonstrate how effective case boundaries can be constructed and reported, which may be of particular interest to prospective case study researchers.

Article synopsis of case study research using Stake's tradition

Ledderer ( 2011 ) used a qualitative case study research design, informed by modern ethnography. The study is bounded to 10 general practice clinics in Denmark, who had received federal funding to implement preventative care services based on a Motivational Interviewing intervention. The researcher question focussed on “why is it so difficult to create change in medical practice?” (Ledderer, 2011 , p. 27). The study context was adequately described, providing detail on the general practitioner (GP) clinics and relevant political and economic influences. Methodological decisions are described in first person narrative, providing insight on researcher perspectives and interaction with the case. Forty-four interviews were conducted, which focussed on how GPs conducted consultations, and the form, nature and content, rather than asking their opinion or experience (Ledderer, 2011 , p. 30). The duration and intensity of researcher immersion in the case enhanced depth of description and trustworthiness of study findings. Analysis was consistent with Stake's tradition, and the researcher provided examples of inquiry techniques used to challenge assumptions about emerging themes. Several other seminal qualitative works were cited. The themes and typology constructed are rich in narrative data and storytelling by clinic staff, demonstrating individual clinic experiences as well as shared meanings and understandings about changing from a biomedical to psychological approach to preventative health intervention. Conclusions make note of social and cultural meanings and lessons learned, which might not have been uncovered using a different methodology.

Article synopsis of case study research using Yin's tradition

Gillard et al. ( 2011 ) study of camps for adolescents living with HIV/AIDs provided a good example of Yin's interpretive case study approach. The context of the case is bounded by the three summer camps of which the researchers had prior professional involvement. A case study protocol was developed that used multiple methods to gather information at three data collection points coinciding with three youth camps (Teen Forum, Discover Camp, and Camp Strong). Gillard and colleagues followed Yin's ( 2009 ) principles, using a consistent data protocol that enhanced cross-case analysis. Data described the young people, the camp physical environment, camp schedule, objectives and outcomes, and the staff of three youth camps. The findings provided a detailed description of the context, with less detail of individual participants, including insight into researcher's interpretations and methodological decisions throughout the data collection and analysis process. Findings provided the reader with a sense of “being there,” and are discovered through constant comparison of the case with the research issues; the case is the unit of analysis. There is evidence of researcher immersion in the case, and Gillard reports spending significant time in the field in a naturalistic and integrated youth mentor role.

This case study is not intended to have a significant impact on broader health policy, although does have implications for health professionals working with adolescents. Study conclusions will inform future camps for young people with chronic disease, and practitioners are able to compare similarities between this case and their own practice (for knowledge translation). No limitations of this article were reported. Limitations related to publication of this case study were that it was 20 pages long and used three tables to provide sufficient description of the camp and program components, and relationships with the research issue.

Researcher and case interactions and triangulation

Researcher and case interactions and transactions are a defining feature of case study methodology (Stake, 1995 ). Narrative stories, vignettes, and thick description are used to provoke vicarious experience and a sense of being there with the researcher in their interaction with the case. Few of the case studies reviewed provided details of the researcher's relationship with the case, researcher–case interactions, and how these influenced the development of the case study (Buzzanell & D'Enbeau, 2009 ; D'Enbeau et al., 2010 ; Gallagher et al., 2013 ; Gillard et al., 2011 ; Ledderer, 2011 ; Nagar-Ron & Motzafi-Haller, 2011 ). The role and position of the researcher needed to be self-examined and understood by readers, to understand how this influenced interactions with participants, and to determine what triangulation is needed (Merriam, 2009 ; Stake, 1995 ).

Gillard et al. ( 2011 ) provided a good example of triangulation, comparing data sources in a table (p. 1513). Triangulation of sources was used to reveal as much depth as possible in the study by Nagar-Ron and Motzafi-Haller ( 2011 ), while also enhancing confirmation validity. There were several case studies that would have benefited from improved range and use of data sources, and descriptions of researcher–case interactions (Ajodhia-Andrews & Berman, 2009 ; Bronken et al., 2012 ; Fincham, Scourfield, & Langer, 2008 ; Fourie & Theron, 2012 ; Hooghe et al., 2012 ; Snyder-Young, 2011 ; Yeh, 2013 ).

Study design inconsistent with methodology

Good, rigorous case studies require a strong methodological justification (Meyer, 2001 ) and a logical and coherent argument that defines paradigm, methodological position, and selection of study methods (Denzin & Lincoln, 2011b ). Methodological justification was insufficient in several of the studies reviewed (Barone, 2010 ; Bronken et al., 2012 ; Hooghe et al., 2012 ; Mawn et al., 2010 ; Roscigno et al., 2012 ; Yeh, 2013 ). This was judged by the absence, or inadequate or inconsistent reference to case study methodology in-text.

In six studies, the methodological justification provided did not relate to case study. There were common issues identified. Secondary sources were used as primary methodological references indicating that study design might not have been theoretically sound (Colón-Emeric et al., 2010 ; Coltart & Henwood, 2012 ; Roscigno et al., 2012 ; Snyder-Young, 2011 ). Authors and sources cited in methodological descriptions were inconsistent with the actual study design and practices used (Fourie & Theron, 2012 ; Hooghe et al., 2012 ; Jorrín-Abellán et al., 2008 ; Mawn et al., 2010 ; Rytterström et al., 2013 ; Wimpenny & Savin-Baden, 2012 ). This occurred when researchers cited Stake or Yin, or both (Mawn et al., 2010 ; Rytterström et al., 2013 ), although did not follow their paradigmatic or methodological approach. In 26 studies there were no citations for a case study methodological approach.

The findings of this study have highlighted a number of issues for researchers. A considerable number of case studies reviewed were missing key elements that define qualitative case study methodology and the tradition cited. A significant number of studies did not provide a clear methodological description or justification relevant to case study. Case studies in health and social sciences did not provide sufficient information for the reader to understand case selection, and why this case was chosen above others. The context of the cases were not described in adequate detail to understand all relevant elements of the case context, which indicated that cases may have not been contextually bounded. There were inconsistencies between reported methodology, study design, and paradigmatic approach in case studies reviewed, which made it difficult to understand the study methodology and theoretical foundations. These issues have implications for methodological integrity and honesty when reporting study design, which are values of the qualitative research tradition and are ethical requirements (Wager & Kleinert, 2010a ). Poorly described methodological descriptions may lead the reader to misinterpret or discredit study findings, which limits the impact of the study, and, as a collective, hinders advancements in the broader qualitative research field.

The issues highlighted in our review build on current debates in the case study literature, and queries about the value of this methodology. Case study research can be situated within different paradigms or designed with an array of methods. In order to maintain the creativity and flexibility that is valued in this methodology, clearer descriptions of paradigm and theoretical position and methods should be provided so that study findings are not undervalued or discredited. Case study research is an interdisciplinary practice, which means that clear methodological descriptions might be more important for this approach than other methodologies that are predominantly driven by fewer disciplines (Creswell, 2013b ).

Authors frequently omit elements of methodologies and include others to strengthen study design, and we do not propose a rigid or purist ideology in this paper. On the contrary, we encourage new ideas about using case study, together with adequate reporting, which will advance the value and practice of case study. The implications of unclear methodological descriptions in the studies reviewed were that study design appeared to be inconsistent with reported methodology, and key elements required for making judgements of rigour were missing. It was not clear whether the deviations from methodological tradition were made by researchers to strengthen the study design, or because of misinterpretations. Morse ( 2011 ) recommended that innovations and deviations from practice are best made by experienced researchers, and that a novice might be unaware of the issues involved with making these changes. To perpetuate the tradition of case study research, applications in the published literature should have consistencies with traditional methodological constructions, and deviations should be described with a rationale that is inherent in study conduct and findings. Providing methodological descriptions that demonstrate a strong theoretical foundation and coherent study design will add credibility to the study, while ensuring the intrinsic meaning of case study is maintained.

The value of this review is that it contributes to discussion of whether case study is a methodology or method. We propose possible reasons why researchers might make this misinterpretation. Researchers may interchange the terms methods and methodology, and conduct research without adequate attention to epistemology and historical tradition (Carter & Little, 2007 ; Sandelowski, 2010 ). If the rich meaning that naming a qualitative methodology brings to the study is not recognized, a case study might appear to be inconsistent with the traditional approaches described by principal authors (Creswell, 2013a ; Merriam, 2009 ; Stake, 1995 ; Yin, 2009 ). If case studies are not methodologically and theoretically situated, then they might appear to be a case report.

Case reports are promoted by university and medical journals as a method of reporting on medical or scientific cases; guidelines for case reports are publicly available on websites ( http://www.hopkinsmedicine.org/institutional_review_board/guidelines_policies/guidelines/case_report.html ). The various case report guidelines provide a general criteria for case reports, which describes that this form of report does not meet the criteria of research, is used for retrospective analysis of up to three clinical cases, and is primarily illustrative and for educational purposes. Case reports can be published in academic journals, but do not require approval from a human research ethics committee. Traditionally, case reports describe a single case, to explain how and what occurred in a selected setting, for example, to illustrate a new phenomenon that has emerged from a larger study. A case report is not necessarily particular or the study of a case in its entirety, and the larger study would usually be guided by a different research methodology.

This description of a case report is similar to what was provided in some studies reviewed. This form of report lacks methodological grounding and qualities of research rigour. The case report has publication value in demonstrating an example and for dissemination of knowledge (Flanagan, 1999 ). However, case reports have different meaning and purpose to case study, which needs to be distinguished. Findings of our review suggest that the medical understanding of a case report has been confused with qualitative case study approaches.

In this review, a number of case studies did not have methodological descriptions that included key characteristics of case study listed in the adapted criteria, and several issues have been discussed. There have been calls for improvements in publication quality of qualitative research (Morse, 2011 ), and for improvements in peer review of submitted manuscripts (Carter & Little, 2007 ; Jasper, Vaismoradi, Bondas, & Turunen, 2013 ). The challenging nature of editor and reviewers responsibilities are acknowledged in the literature (Hames, 2013 ; Wager & Kleinert, 2010b ); however, review of case study methodology should be prioritized because of disputes on methodological value.

Authors using case study approaches are recommended to describe their theoretical framework and methods clearly, and to seek and follow specialist methodological advice when needed (Wager & Kleinert, 2010a ). Adequate page space for case study description would contribute to better publications (Gillard et al., 2011 ). Capitalizing on the ability to publish complementary resources should be considered.

Limitations of the review

There is a level of subjectivity involved in this type of review and this should be considered when interpreting study findings. Qualitative methods journals were selected because the aims and scope of these journals are to publish studies that contribute to methodological discussion and development of qualitative research. Generalist health and social science journals were excluded that might have contained good quality case studies. Journals in business or education were also excluded, although a review of case studies in international business journals has been published elsewhere (Piekkari et al., 2009 ).

The criteria used to assess the quality of the case studies were a set of qualitative indicators. A numerical or ranking system might have resulted in different results. Stake's ( 1995 ) criteria have been referenced elsewhere, and was deemed the best available (Creswell, 2013b ; Crowe et al., 2011 ). Not all qualitative studies are reported in a consistent way and some authors choose to report findings in a narrative form in comparison to a typical biomedical report style (Sandelowski & Barroso, 2002 ), if misinterpretations were made this may have affected the review.

Case study research is an increasingly popular approach among qualitative researchers, which provides methodological flexibility through the incorporation of different paradigmatic positions, study designs, and methods. However, whereas flexibility can be an advantage, a myriad of different interpretations has resulted in critics questioning the use of case study as a methodology. Using an adaptation of established criteria, we aimed to identify and assess the methodological descriptions of case studies in high impact, qualitative methods journals. Few articles were identified that applied qualitative case study approaches as described by experts in case study design. There were inconsistencies in methodology and study design, which indicated that researchers were confused whether case study was a methodology or a method. Commonly, there appeared to be confusion between case studies and case reports. Without clear understanding and application of the principles and key elements of case study methodology, there is a risk that the flexibility of the approach will result in haphazard reporting, and will limit its global application as a valuable, theoretically supported methodology that can be rigorously applied across disciplines and fields.

Conflict of interest and funding

The authors have not received any funding or benefits from industry or elsewhere to conduct this study.

• Adamson S, Holloway M. Negotiating sensitivities and grappling with intangibles: Experiences from a study of spirituality and funerals. Qualitative Research. 2012; 12 (6):735–752. doi: 10.1177/1468794112439008. [ CrossRef ] [ Google Scholar ]
• Ajodhia-Andrews A, Berman R. Exploring school life from the lens of a child who does not use speech to communicate. Qualitative Inquiry. 2009; 15 (5):931–951. doi: 10.1177/1077800408322789. [ CrossRef ] [ Google Scholar ]
• Alexander B. K, Moreira C, Kumar H. S. Resisting (resistance) stories: A tri-autoethnographic exploration of father narratives across shades of difference. Qualitative Inquiry. 2012; 18 (2):121–133. doi: 10.1177/1077800411429087. [ CrossRef ] [ Google Scholar ]
• Austin W, Park C, Goble E. From interdisciplinary to transdisciplinary research: A case study. Qualitative Health Research. 2008; 18 (4):557–564. doi: 10.1177/1049732307308514. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ayres L, Kavanaugh K, Knafl K. A. Within-case and across-case approaches to qualitative data analysis. Qualitative Health Research. 2003; 13 (6):871–883. doi: 10.1177/1049732303013006008. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Barone T. L. Culturally sensitive care 1969–2000: The Indian Chicano Health Center. Qualitative Health Research. 2010; 20 (4):453–464. doi: 10.1177/1049732310361893. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bassey M. A solution to the problem of generalisation in educational research: Fuzzy prediction. Oxford Review of Education. 2001; 27 (1):5–22. doi: 10.1080/03054980123773. [ CrossRef ] [ Google Scholar ]
• Bronken B. A, Kirkevold M, Martinsen R, Kvigne K. The aphasic storyteller: Coconstructing stories to promote psychosocial well-being after stroke. Qualitative Health Research. 2012; 22 (10):1303–1316. doi: 10.1177/1049732312450366. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Broyles L. M, Rodriguez K. L, Price P. A, Bayliss N. K, Sevick M. A. Overcoming barriers to the recruitment of nurses as participants in health care research. Qualitative Health Research. 2011; 21 (12):1705–1718. doi: 10.1177/1049732311417727. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Buckley C. A, Waring M. J. Using diagrams to support the research process: Examples from grounded theory. Qualitative Research. 2013; 13 (2):148–172. doi: 10.1177/1468794112472280. [ CrossRef ] [ Google Scholar ]
• Buzzanell P. M, D'Enbeau S. Stories of caregiving: Intersections of academic research and women's everyday experiences. Qualitative Inquiry. 2009; 15 (7):1199–1224. doi: 10.1177/1077800409338025. [ CrossRef ] [ Google Scholar ]
• Carter S. M, Little M. Justifying knowledge, justifying method, taking action: Epistemologies, methodologies, and methods in qualitative research. Qualitative Health Research. 2007; 17 (10):1316–1328. doi: 10.1177/1049732307306927. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cheek J, Garnham B, Quan J. What's in a number? Issues in providing evidence of impact and quality of research(ers) Qualitative Health Research. 2006; 16 (3):423–435. doi: 10.1177/1049732305285701. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Colón-Emeric C. S, Plowman D, Bailey D, Corazzini K, Utley-Smith Q, Ammarell N, et al. Regulation and mindful resident care in nursing homes. Qualitative Health Research. 2010; 20 (9):1283–1294. doi: 10.1177/1049732310369337. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Coltart C, Henwood K. On paternal subjectivity: A qualitative longitudinal and psychosocial case analysis of men's classed positions and transitions to first-time fatherhood. Qualitative Research. 2012; 12 (1):35–52. doi: 10.1177/1468794111426224. [ CrossRef ] [ Google Scholar ]
• Creswell J. W. Five qualitative approaches to inquiry. In: Creswell J. W, editor. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage; 2013a. pp. 53–84. [ Google Scholar ]
• Creswell J. W. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage; 2013b. [ Google Scholar ]
• Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Medical Research Methodology. 2011; 11 (1):1–9. doi: 10.1186/1471-2288-11-100. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cunsolo Willox A, Harper S. L, Edge V. L, ‘My Word’: Storytelling and Digital Media Lab, & Rigolet Inuit Community Government Storytelling in a digital age: Digital storytelling as an emerging narrative method for preserving and promoting indigenous oral wisdom. Qualitative Research. 2013; 13 (2):127–147. doi: 10.1177/1468794112446105. [ CrossRef ] [ Google Scholar ]
• De Haene L, Grietens H, Verschueren K. Holding harm: Narrative methods in mental health research on refugee trauma. Qualitative Health Research. 2010; 20 (12):1664–1676. doi: 10.1177/1049732310376521. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• D'Enbeau S, Buzzanell P. M, Duckworth J. Problematizing classed identities in fatherhood: Development of integrative case studies for analysis and praxis. Qualitative Inquiry. 2010; 16 (9):709–720. doi: 10.1177/1077800410374183. [ CrossRef ] [ Google Scholar ]
• Denzin N. K, Lincoln Y. S. Introduction: Disciplining the practice of qualitative research. In: Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011a. pp. 1–6. [ Google Scholar ]
• Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011b. [ Google Scholar ]
• Edwards R, Weller S. Shifting analytic ontology: Using I-poems in qualitative longitudinal research. Qualitative Research. 2012; 12 (2):202–217. doi: 10.1177/1468794111422040. [ CrossRef ] [ Google Scholar ]
• Eisenhardt K. M. Building theories from case study research. The Academy of Management Review. 1989; 14 (4):532–550. doi: 10.2307/258557. [ CrossRef ] [ Google Scholar ]
• Fincham B, Scourfield J, Langer S. The impact of working with disturbing secondary data: Reading suicide files in a coroner's office. Qualitative Health Research. 2008; 18 (6):853–862. doi: 10.1177/1049732307308945. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flanagan J. Public participation in the design of educational programmes for cancer nurses: A case report. European Journal of Cancer Care. 1999; 8 (2):107–112. doi: 10.1046/j.1365-2354.1999.00141.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flyvbjerg B. Five misunderstandings about case-study research. Qualitative Inquiry. 2006; 12 (2):219–245. doi: 10.1177/1077800405284.363. [ CrossRef ] [ Google Scholar ]
• Flyvbjerg B. Case study. In: Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011. pp. 301–316. [ Google Scholar ]
• Fourie C. L, Theron L. C. Resilience in the face of fragile X syndrome. Qualitative Health Research. 2012; 22 (10):1355–1368. doi: 10.1177/1049732312451871. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gallagher N, MacFarlane A, Murphy A. W, Freeman G. K, Glynn L. G, Bradley C. P. Service users’ and caregivers’ perspectives on continuity of care in out-of-hours primary care. Qualitative Health Research. 2013; 23 (3):407–421. doi: 10.1177/1049732312470521. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gerring J. What is a case study and what is it good for? American Political Science Review. 2004; 98 (2):341–354. doi: 10.1017/S0003055404001182. [ CrossRef ] [ Google Scholar ]
• Gillard A, Witt P. A, Watts C. E. Outcomes and processes at a camp for youth with HIV/AIDS. Qualitative Health Research. 2011; 21 (11):1508–1526. doi: 10.1177/1049732311413907. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Grant M, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009; 26 :91–108. doi: 10.1111/j.1471-1842.2009.00848.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gratton M.-F, O'Donnell S. Communication technologies for focus groups with remote communities: A case study of research with First Nations in Canada. Qualitative Research. 2011; 11 (2):159–175. doi: 10.1177/1468794110394068. [ CrossRef ] [ Google Scholar ]
• Hallberg L. Quality criteria and generalization of results from qualitative studies. International Journal of Qualitative Studies on Health and Wellbeing. 2013; 8 :1. doi: 10.3402/qhw.v8i0.20647. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hames I. Committee on Publication Ethics, 1. 2013, March. COPE Ethical guidelines for peer reviewers. Retrieved April 7, 2013, from http://publicationethics.org/resources/guidelines . [ Google Scholar ]
• Hooghe A, Neimeyer R. A, Rober P. “Cycling around an emotional core of sadness”: Emotion regulation in a couple after the loss of a child. Qualitative Health Research. 2012; 22 (9):1220–1231. doi: 10.1177/1049732312449209. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jackson C. B, Botelho E. M, Welch L. C, Joseph J, Tennstedt S. L. Talking with others about stigmatized health conditions: Implications for managing symptoms. Qualitative Health Research. 2012; 22 (11):1468–1475. doi: 10.1177/1049732312450323. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jasper M, Vaismoradi M, Bondas T, Turunen H. Validity and reliability of the scientific review process in nursing journals—time for a rethink? Nursing Inquiry. 2013 doi: 10.1111/nin.12030. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jensen J. L, Rodgers R. Cumulating the intellectual gold of case study research. Public Administration Review. 2001; 61 (2):235–246. doi: 10.1111/0033-3352.00025. [ CrossRef ] [ Google Scholar ]
• Jorrín-Abellán I. M, Rubia-Avi B, Anguita-Martínez R, Gómez-Sánchez E, Martínez-Mones A. Bouncing between the dark and bright sides: Can technology help qualitative research? Qualitative Inquiry. 2008; 14 (7):1187–1204. doi: 10.1177/1077800408318435. [ CrossRef ] [ Google Scholar ]
• Ledderer L. Understanding change in medical practice: The role of shared meaning in preventive treatment. Qualitative Health Research. 2011; 21 (1):27–40. doi: 10.1177/1049732310377451. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lincoln Y. S. Emerging criteria for quality in qualitative and interpretive research. Qualitative Inquiry. 1995; 1 (3):275–289. doi: 10.1177/107780049500100301. [ CrossRef ] [ Google Scholar ]
• Luck L, Jackson D, Usher K. Case study: A bridge across the paradigms. Nursing Inquiry. 2006; 13 (2):103–109. doi: 10.1111/j.1440-1800.2006.00309.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mawn B, Siqueira E, Koren A, Slatin C, Devereaux Melillo K, Pearce C, et al. Health disparities among health care workers. Qualitative Health Research. 2010; 20 (1):68–80. doi: 10.1177/1049732309355590. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Merriam S. B. Qualitative research: A guide to design and implementation. 3rd ed. San Francisco, CA: Jossey-Bass; 2009. [ Google Scholar ]
• Meyer C. B. A case in case study methodology. Field Methods. 2001; 13 (4):329–352. doi: 10.1177/1525822x0101300402. [ CrossRef ] [ Google Scholar ]
• Morse J. M. Mixing qualitative methods. Qualitative Health Research. 2009; 19 (11):1523–1524. doi: 10.1177/1049732309349360. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Morse J. M. Molding qualitative health research. Qualitative Health Research. 2011; 21 (8):1019–1021. doi: 10.1177/1049732311404706. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Morse J. M, Dimitroff L. J, Harper R, Koontz A, Kumra S, Matthew-Maich N, et al. Considering the qualitative–quantitative language divide. Qualitative Health Research. 2011; 21 (9):1302–1303. doi: 10.1177/1049732310392386. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nagar-Ron S, Motzafi-Haller P. “My life? There is not much to tell”: On voice, silence and agency in interviews with first-generation Mizrahi Jewish women immigrants to Israel. Qualitative Inquiry. 2011; 17 (7):653–663. doi: 10.1177/1077800411414007. [ CrossRef ] [ Google Scholar ]
• Nairn K, Panelli R. Using fiction to make meaning in research with young people in rural New Zealand. Qualitative Inquiry. 2009; 15 (1):96–112. doi: 10.1177/1077800408318314. [ CrossRef ] [ Google Scholar ]
• Nespor J. The afterlife of “teachers’ beliefs”: Qualitative methodology and the textline. Qualitative Inquiry. 2012; 18 (5):449–460. doi: 10.1177/1077800412439530. [ CrossRef ] [ Google Scholar ]
• Piekkari R, Welch C, Paavilainen E. The case study as disciplinary convention: Evidence from international business journals. Organizational Research Methods. 2009; 12 (3):567–589. doi: 10.1177/1094428108319905. [ CrossRef ] [ Google Scholar ]
• Ragin C. C, Becker H. S. What is a case?: Exploring the foundations of social inquiry. Cambridge: Cambridge University Press; 1992. [ Google Scholar ]
• Roscigno C. I, Savage T. A, Kavanaugh K, Moro T. T, Kilpatrick S. J, Strassner H. T, et al. Divergent views of hope influencing communications between parents and hospital providers. Qualitative Health Research. 2012; 22 (9):1232–1246. doi: 10.1177/1049732312449210. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rosenberg J. P, Yates P. M. Schematic representation of case study research designs. Journal of Advanced Nursing. 2007; 60 (4):447–452. doi: 10.1111/j.1365-2648.2007.04385.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rytterström P, Unosson M, Arman M. Care culture as a meaning- making process: A study of a mistreatment investigation. Qualitative Health Research. 2013; 23 :1179–1187. doi: 10.1177/1049732312470760. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M. Whatever happened to qualitative description? Research in Nursing & Health. 2000; 23 (4):334–340. doi: 10.1002/1098-240X. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M. What's in a name? Qualitative description revisited. Research in Nursing & Health. 2010; 33 (1):77–84. doi: 10.1002/nur.20362. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M, Barroso J. Reading qualitative studies. International Journal of Qualitative Methods. 2002; 1 (1):74–108. [ Google Scholar ]
• Snyder-Young D. “Here to tell her story”: Analyzing the autoethnographic performances of others. Qualitative Inquiry. 2011; 17 (10):943–951. doi: 10.1177/1077800411425149. [ CrossRef ] [ Google Scholar ]
• Stake R. E. The case study method in social inquiry. Educational Researcher. 1978; 7 (2):5–8. [ Google Scholar ]
• Stake R. E. The art of case study research. Thousand Oaks, CA: Sage; 1995. [ Google Scholar ]
• Stake R. E. Case studies. In: Denzin N. K, Lincoln Y. S, editors. Strategies of qualitative inquiry. Thousand Oaks, CA: Sage; 1998. pp. 86–109. [ Google Scholar ]
• Sumsion J. Opening up possibilities through team research: Investigating infants’ experiences of early childhood education and care. Qualitative Research. 2013; 14 (2):149–165. doi: 10.1177/1468794112468471.. [ CrossRef ] [ Google Scholar ]
• Thomas G. Doing case study: Abduction not induction, phronesis not theory. Qualitative Inquiry. 2010; 16 (7):575–582. doi: 10.1177/1077800410372601. [ CrossRef ] [ Google Scholar ]
• Thomas G. A typology for the case study in social science following a review of definition, discourse, and structure. Qualitative Inquiry. 2011; 17 (6):511–521. doi: 10.1177/1077800411409884. [ CrossRef ] [ Google Scholar ]
• Tight M. The curious case of case study: A viewpoint. International Journal of Social Research Methodology. 2010; 13 (4):329–339. doi: 10.1080/13645570903187181. [ CrossRef ] [ Google Scholar ]
• Wager E, Kleinert S. Responsible research publication: International standards for authors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. In: Mayer T, Steneck N, editors. Promoting research integrity in a global environment. Singapore: Imperial College Press/World Scientific; 2010a. pp. 309–316. [ Google Scholar ]
• Wager E, Kleinert S. Responsible research publication: International standards for editors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. In: Mayer T, Steneck N, editors. Promoting research integrity in a global environment. Singapore: Imperial College Press/World Scientific; 2010b. pp. 317–328. [ Google Scholar ]
• Webb C, Kevern J. Focus groups as a research method: A critique of some aspects of their use in nursing research. Journal of Advanced Nursing. 2000; 33 (6):798–805. doi: 10.1046/j.1365-2648.2001.01720.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wimpenny K, Savin-Baden M. Exploring and implementing participatory action synthesis. Qualitative Inquiry. 2012; 18 (8):689–698. doi: 10.1177/1077800412452854. [ CrossRef ] [ Google Scholar ]
• Yeh H.-Y. Boundaries, entities, and modern vegetarianism: Examining the emergence of the first vegetarian organization. Qualitative Inquiry. 2013; 19 (4):298–309. doi: 10.1177/1077800412471516. [ CrossRef ] [ Google Scholar ]
• Yin R. K. Enhancing the quality of case studies in health services research. Health Services Research. 1999; 34 (5 Pt 2):1209–1224. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Yin R. K. Case study research: Design and methods. 4th ed. Thousand Oaks, CA: Sage; 2009. [ Google Scholar ]
• Yin R. K. Applications of case study research. 3rd ed. Thousand Oaks, CA: Sage; 2012. [ Google Scholar ]

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Case Study? | Definition, Examples & Methods

What Is a Case Study? | Definition, Examples & Methods

Published on May 8, 2019 by Shona McCombes . Revised on November 20, 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organization, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyze the case, other interesting articles.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

TipIf your research is more practical in nature and aims to simultaneously investigate an issue as you solve it, consider conducting action research instead.

Unlike quantitative or experimental research , a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

Example of an outlying case studyIn the 1960s the town of Roseto, Pennsylvania was discovered to have extremely low rates of heart disease compared to the US average. It became an important case study for understanding previously neglected causes of heart disease.

However, you can also choose a more common or representative case to exemplify a particular category, experience or phenomenon.

Example of a representative case studyIn the 1920s, two sociologists used Muncie, Indiana as a case study of a typical American city that supposedly exemplified the changing culture of the US at the time.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews , observations , and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data.

Example of a mixed methods case studyFor a case study of a wind farm development in a rural area, you could collect quantitative data on employment rates and business revenue, collect qualitative data on local people’s perceptions and experiences, and analyze local and national media coverage of the development.

The aim is to gain as thorough an understanding as possible of the case and its context.

Prevent plagiarism. Run a free check.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis , with separate sections or chapters for the methods , results and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyze its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Case Study? | Definition, Examples & Methods. Scribbr. Retrieved August 25, 2024, from https://www.scribbr.com/methodology/case-study/

Is this article helpful?

Shona McCombes

Other students also liked, primary vs. secondary sources | difference & examples, what is a theoretical framework | guide to organizing, what is action research | definition & examples, what is your plagiarism score.

• DOI: 10.1177/1065912907313077
• Corpus ID: 35757617

Case Selection Techniques in Case Study Research

• Jason Seawright , J. Gerring
• Published 9 February 2008
• Sociology, Education
• Political Research Quarterly

Figures and Tables from this paper

2,919 Citations

Selecting cases for intensive analysis, a careful look at modern case selection methods, the case for selecting cases that are deviant or extreme on the independent variable, participant selection and access in case study research, a bayesian perspective on case selection, three pathways to case selection in ib: a twenty–year review, analysis & synthesis, is there a (viable) crucial-case method.

• Highly Influenced

Case-to-Condition Ratios in Qualitative Comparative Analysis: Adding Cases Instead of Removing Conditions

Three pathways to case selection in international business: a twenty–year review, analysis and synthesis, paired comparison and theory development: considerations for case selection, 105 references, single-outcome studies, what is a case study and what is it good for, case study research: principles and practices, the possibility principle: choosing negative cases in comparative research, making the most of a case study: theories of the welfare state and the american experience, nested analysis as a mixed-method strategy for comparative research, evidence and inference in the comparative case studyevidence and inference in the comparative case study, insights and pitfalls: selection bias in qualitative research, an experimental template for case study research, causal inference and comparative methods, related papers.

Showing 1 through 3 of 0 Related Papers

Case Selection Module

Choosing cases.

Once we have a hypothesis to explore or test, and once we have settled on a general research design, then we need to choose specific cases to analyze. Case selection is important whether we are conducting a case study of a single civil war, an experiment involving a few dozen college students, or a statistical comparison of hundreds of elections. All social scientists should learn how to select cases with care.

Done well, case selection can enhance the external validity of our research, making us more confident that our results would hold true beyond our particular study. Case selection can also help the internal validity of our research, making us more confident that our conclusions hold true within the confines of our study. Done poorly, case selection can compromise our research or even render it useless.

The purpose of this module is to help you think systematically and intelligently about case selection.

View Module As PDF

1. population vs. sample.

The first question to ask is whether you plan to study the entire population of cases (often referred to as N) or a smaller sample (n) taken from that population.

Studying the entire population is appealing because it essentially guarantees the external validity of our research. We don’t need to make inferences about what happened or why; we have analyzed every relevant case. Over time, teams of scholars have developed datasets with information about every recorded vote in the history of Congress, and about every single interstate war over the last two centuries. Some research projects analyze entire populations like these.

Nevertheless, we usually lack the time, money, or skills needed to analyze the entire population of relevant cases. We might not even have a good way to identify the entire population; there is no master list, for example, of Kurdish rebels or newspaper stories about Senate elections. As a result of these various constraints, we typically pick a smaller sample. This is why polling firms interview 1500 people instead of 250 million. This is why undergraduates write their research papers about, say, democratization in India during the 20th century, and not about democratization in every country of the world over the last three centuries.

However, the choice between population and sample also depends on how we define the larger population. If we believe that “modern world wars” are conceptually distinct from “interstate wars” or “militarized conflicts,” then the population of modern world wars might consist of just two cases – World War I and World War II. Someone who wanted to study the origins of these world wars might actually have the resources needed to examine the entire population of cases. One can imagine other examples, such as “Communist nations in the 21st century” or “U.S. presidential elections decided by the Supreme Court” where the total population (N) is pretty small. In each of these examples, the author would have to justify the boundaries of their population. Are Communist nations in the 21st century really all that different from those that existed in the 20th century?

Suppose that someone wanted to study the relationship between motorcycle helmet laws and motorcycle fatalities in the American states. One could gather data for the most recent year available in all 50 states, which certainly sounds like the entire population of cases. If our aim is to generalize across a wider time period, though, then we would be dealing with a one-year sample. And if we are trying to generalize to some larger population of traffic laws, such as speed limits and seatbelts, and to a larger set of traffic fatalities, then our motorcycle helmet cases would also qualify as a sample.

Frankly, it seems unlikely that all you would want to accomplish in this example is to figure out what happened with one specific kind of law in one year. Doing so would really limit the larger significance of your work. Thus, the choice of sample versus population connects back to the larger aims of the study. When choosing cases we always need to ask ourselves, “What puzzle am I hoping to solve? To what scholarly literature or policy debate am I trying to contribute? What, then, is the population of relevant cases?”

2. Sampling: Random vs. Deliberate

The vast majority of the time, for practical or conceptual reasons, we are dealing with samples. At the most general level, we need to decide whether to choose a sample of cases randomly or deliberately. One might think that random selection would always be preferred because the sample would more likely resemble the entire population, thus giving our study added external validity. This intuition is correct – as long as the number of cases is pretty large. If the number is small, then one might randomly select an atypical sample, which would actually hurt external validity.

You can take a real course about probability and statistics to understand why, or you can accept the following example as a rough proof. Let’s imagine that a polling firm wanted to know what American adults think about a controversial issue like immigration. If the firm randomly selected just two people – let’s call them Border Wall Bob and No Amnesty Nancy – it might conclude that all Americans have strongly negative views toward immigrants. And those conclusions would be wrong. If that same firm chose 1000 or 1500 Americans at random, it would be much more likely to identify the full range of attitudes, as well as the correct distribution. (The sample would rarely look exactly like the population, but it would probably be close if the firm sampled correctly.) With so many cases, a few extreme values in any direction will not distort the entire sample.

Thus, if the research design is based on a statistical comparison of many cases, scholars will probably choose their cases randomly. (The large number of cases will have the added benefit of helping us to establish the internal validity of our research: we can become more confident in concluding whether our measures are correlated, and whether any apparent relationships could be spurious.) If the research design is a detailed case study, however, the cases will almost always be chosen deliberately.

With experimental designs, the cases could be chosen deliberately or randomly. A lab experiment will probably not rely on a random sample of individuals; researchers will usually have to take whoever is willing or required to participate in the experiment. A survey or field experiment, on the other hand, might select at random a large number of individuals, voting precincts, villages, development projects, or some other unit of analysis. Such random selection of cases will help the external validity of the study, while experimental controls and random assignment of cases will generate internal validity.

Whether we choose cases randomly or deliberately, we are concerned about generating a biased sample. Some types of bias originate with the researcher. Suppose you wanted to sample opinions from the entire college campus, but you only distributed surveys to three freshmen dorms. That sample would not reflect the full range of students on campus, and could bias the results if freshmen held different opinions from upperclassmen. Other types of sample bias are beyond the researchers’ control — sometimes just bad luck. We might distribute surveys to a variety of dorms on campus, yet the main people who filled them out and returned them might be freshmen. Therefore, after taking a sample, it often makes sense to compare it to whatever is known about the larger population.

3. Generating a Random Sample

To learn different ways of choosing cases randomly, you can consult standard research methods textbooks, which often do a good job of teaching this skill. See, for example, chapter 7 in Johnson and Reynolds, Political Science Research Methods 7th edition, or chapter 6 in Kellstedt and Whitten, The Fundamentals of Political Science Research 2nd edition. There you will encounter simple, systematic, stratified, and cluster random samples. You can also find helpful videos on-line, such as these two:

[Note: both of these videos discuss “convenience sampling,” which they don’t exactly endorse. Convenience samples and snowball samples are both nonprobability samples in which each element or group within the population does not have an equal chance of being selected. The external validity of such samples is thus highly suspect. Nonprobability samples are used occasionally in social scientific research, but not often.]

It is certainly possible to combine strategies as well. A survey research firm conducting an exit poll on Election Day could start with a simple random sample of congressional districts, then a systematic random sample of voting precincts within those districts, and finish with a stratified random sample of individuals who showed up to vote at those precincts. Someone analyzing trends in media coverage of terrorism might analyze only those years ending in 0, 2, 5, and 8, and then collect a simple random sample of stories for each year.

4. Picking Cases Deliberately

Standard methods textbooks are pretty useless if you intend to choose cases deliberately. That’s too bad, for it means that students planning to conduct case studies receive practically no guidance about a crucial step in the research process. One reason for this gap, I suspect, is that many social scientists view deliberate case selection with suspicion. A crafty researcher could pick one or two cases to prove, well, just about anything. What is supposed to be reputable social science could easily degenerate into intellectual sleight-of-hand or trickery. For a playful analogy, watch how master magician Ricky Jay manages to reveal just the right cards from a full deck:

While we might be delighted to watch someone manipulate cards so effortlessly, we could be outraged to discover a political scientist doing something similar when he or she picked cases to study. We would seriously doubt the study’s internal validity, external validity, or both. Carefully selecting a few vivid examples to “prove” a general point is common among policy advocates and strong partisans, but it is not good practice for social scientists.

To learn more about deliberate case selection, especially for case studies, I would recommend reading chapter 6 in Lipson, How to Write a BA Thesis; pages 77-88 of Van Evera, Guide to Methods for Students of Political Science; and, if you’re feeling ambitious, chapters 3 and 5 in Gerring, Case Study Research.

Because snappy YouTube videos about deliberate case selection are so rare, I will highlight some of their advice in the table below. One general strategy is to look for ways of maximizing the number of observations within each case; a related strategy is to find cases with analytically-useful variation (i.e., variation linked to the hypotheses we wish to explore or test). In both instances, we are trying to approximate the analytic leverage that comes with large-n statistical comparisons. But another strategy is simply to emphasize a distinctive strength of the case study design – identifying causal links and mechanisms through careful process tracing – and to choose a single case or a few cases that will enable the researcher to study a piece of the political world in real depth.

Enter the URL below into your favorite RSS reader.

A case study of the assistive technology network in Sierra Leone before and after a targeted systems-level investment

• Citation (BibTeX)

Sorry, something went wrong. Please try again.

If this problem reoccurs, please contact Scholastica Support

Error message:

View more stats

Many people with disabilities in low-income settings, such as Sierra Leone, do not have access to the assistive technology (AT) they need, yet research to measure and address this issue remains limited. This paper presents a case study of the Assistive Technology 2030 (AT2030) funded Country Investment project in Sierra Leone. The research explored the nature and strength of the AT stakeholder network in Sierra Leone over the course of one year, presenting a snapshot of the network before and after a targeted systems level investment.

Mixed-method surveys were distributed via the Qualtrics software twice, in December 2021 and September 2022 to n=20 and n=16 participants (respectively). Qualitative data was analyzed thematically, while quantitative data was analyzed with the NodeXL software and MS Excel to generate descriptive statistics, visualizations, and specific metrics related to indegree, betweenness and closeness centrality of organizations grouped by type.

Findings suggest the one-year intervention did stimulate change within the AT network in Sierra Leone, increasing the number of connections within the AT network and strengthening existing relationships within the network. Findings are also consistent with existing data suggesting cost is a key barrier to AT access for both organizations providing AT and people with disabilities to obtain AT.

While this paper is the first to demonstrate that a targeted investment in AT systems and policies at the national level can have a resulting impact on the nature and strength of the AT, it only measures outcomes at one-year after investment. Further longitudinal impact evaluation would be desirable. Nonetheless, the results support the potential for systemic investments which leverage inter-organizational relationships and prioritize financial accessibility of AT, as one means of contributing towards increased access to AT for all, particularly in low-income settings.

Assistive technology (AT) is an umbrella term which broadly encompasses assistive products (AP) and the related services which improves function and enhances the user’s participation in all areas of life. 1 Assistive products are “any external products (including devices, equipment, instruments and software) […] with the primary purpose to maintain or improve an individual’s functioning and independence and/or well-being, or to prevent impairments and secondary health conditions”. 2

Recently, awareness for the urgent need to improve access to Assistive Technology has expanded, as 2022 global population statistics highlights one in three people, or 2.5 billion people, requires at least one assistive product. 1 The demand for AT is projected to increase to 3.5 billion people by 2050, yet 90% of them lack access to the products and services they need. 1 , 3 A systemic approach which adequately measures outcomes and impact is urgently required to stimulate evidence-based policies and systems which support universal access to AT. 1 , 4 , 5 However, a systemic approach first necessitates baseline understandings of the existing system, inclusive of sociopolitical context and the key stakeholders working within that context.

Assistive technology is necessary for people with disabilities to engage in activities of daily living, such as personal care or employment, and social engagement. 6 Moreover, people with disabilities also require AT to enact their basic human rights, as outlined in the United Nations Convention on the Rights of Persons with Disabilities (UNCRPD). 7 Unfortunately, many people do not have access to the AT they require, an inequity which is perpetuated within low-income settings. 8 Despite this growing disparity and a well-documented association between poverty and disability, 9 research gaps remain related to AT within low-income settings in the global South. 10

In Sierra Leone, the national prevalence of disability is estimated to be 1.3%, according to the most recent population and housing census data. 11 , 12 This is unusually low, as compared to the 16% global prevalence (World Health Organization, 2022). National stakeholders within the AT network argue this statistic does not adequately represent the true scope of disability in Sierra Leone. 10 Their stance is supported by survey data from the Rapid Assistive Technology Assessment (rATA) across a subset of the population in Freetown, which indicated a dramatically different picture: a 24.9% prevalence of self-reported disability on the basis of the Washington Group Questions (20.6% reported as having “some difficulty”, while 4.3% rated “a lot of difficulty” or above), predominantly mobility and vision related disabilities. 13 The rATA also highlighted 62.5% of older people surveyed indicated having a disability, while the incidence of disability among females was nearly 2% higher than in males. 13

Despite the 2011 Sierra Leone Disability Act being implemented, access to AT in Sierra Leone remains poor. 13 The rATA suggests only 14.9% of those with disabilities in Freetown have the assistive products they require, an alarming rate which also fails to consider people with disabilities not surveyed in rural Sierra Leone where access to such services is likely lower. 13 Meanwhile, it is estimated over half of the population of Sierra Leone lives in poverty, with 13% in extreme poverty. 14 As affordability ranks as the top barrier for AT access, poverty further perpetuates the challenges of people with disabilities within this subset of the population to access necessary AT. 13 Within the context of low-resource settings it is therefore imperative that those resources which are allocated to provide assistive products are used in the most optimal manner, and that different stakeholders work together to co-construct a systemic approach which can identify and prioritise those most in need.

This paper presents a dataset collected in tandem with an Assistive Technology 2030 (AT2030) funded Country Investment project in Sierra Leone in collaboration with Clinton Health Access Initiative (CHAI). The study aimed to explore the nature and strength of the assistive technology stakeholder network in Sierra Leone over the course of one year through a mixed methods survey methodology. We provide a systemic snapshot of the AT network in Sierra Leone, highlighting what assistive products are available, who provides and receives them, and how. We also present a relational analysis of the existing AT network, inclusive of the organizations working within areas of AT and their degrees of connectivity and collaboration amongst one another. We hope that such data can strengthen the provision of AT in Sierra Leone through identifying assistive product availability, procurement, and provision, as well as the nature of the relationships between (the relationality ) of the AT network. We also sought to provide an overview of any possible changes to the network over the course of a one-year investment by AT2030.

This study used a mixed methods survey approach, facilitated by Qualtrics online survey software. Surveys were collaboratively developed and distributed at the two time periods in December 2021 and September 2022 (herein respectively described as Baseline=T1 and Follow Up= T2).

Intervention

This paper presents the Sierra Leone country project built within a larger, targeted investment in assistive technology systems development in four African countries,by AT2030, a project led by the Global Disability Innovation Hub and funded by UK Aid. The four in-country projects were administered by Clinton Health Access Initiative (CHAI) in partnership with local government ministries and agencies. As part of this investment, CHAI and its partners convened a Technical Working Group which brought together key stakeholders in the assistive technology field. Over the course of one year, the Technical Working Group had an overarching goal to develop and strengthen key assistive technology related policies in each of the four countries. The data in this study on the AT network in Sierra Leone was collected at the outset and following completion of the AT2030 investment, by researchers who were not part of the investment process, thus allowing for third-party evaluation. To maintain objectivity, neither CHAI nor the funder were responsible for the design, data collection, analysis or reporting of results, but this paper has benefited from a programmatic perspective provided by CHIA.

Participants

Participants included members of relevant ministries involved in assistive technology leadership and/or delivery, and staff representing relevant non-profit organizations (both international and local), service providers and organizations for persons with disabilities. Participants were asked to respond on behalf of their organization. All prospective participants were identified by the researchers and local project partners, including those coordinating the investment identified above, and added to a distribution list on Qualtrics, which only contained pertinent identifying information such as name, organization, and email. Over the course of the study, n=20 (T1) and n=16 (T2) participants consented to and completed surveys. While the relatively small sample size may inherently restrict the generalizability of this study, the sample size is reflective of the size of the assistive technology network in Sierra Leone, which we aimed to explore.

Data collection

The survey was emailed to the distribution list at two time points: December 2021 (T1) and September 2022 (T2). Two reminder emails were sent out via Qualtrics at two-week intervals following each time point, to participants who had not yet completed the surveys as a means to stimulate participant retention. The T1 and T2 surveys were identical, however the T2 survey utilized display logic functionalities such as conditional skipping to prevent retained respondents from completing redundant questions such as demographic information. If a participant completed the survey for the first time during the T2 period, they received the survey in its entirety without conditional skipping.

Survey content

Survey questions aimed to capture what AT is available, how it is being provided, who is receiving it and how. Questions also consisted of demographic information and qualitative prompts to identify participants’ roles within the AT network and critical challenges experienced in enacting their roles, as well as the nature and strength of relationships between stakeholders. Additional data was collected on participatory engagement in policy development, knowledge of assistive technology, and capacity for leadership which will be published separately.

Using the methodology reported by Smith and colleagues, 15 the WHO priority assistive products list was provided for respondents to select the products and associated services their organization provides. Additionally, the survey requested respondents to select from a list of organizations, which ones they were aware of as working within AT areas in Sierra Leone, followed by a subsequent 5-point Likert scale (1-5, 1= no relationship, 5= collaboration) to indicate which organizations they had working relationships with and to what extent. In attempts to maximize response rates and maintain participant retention, two reminder emails were sent to participants for T1 and T2; however, challenges encountered were participant drop-out from T1 to T2.

Data analysis

Data was reviewed across the two time periods and descriptive statistics (counts and means) were calculated for all variables using MS Excel software. Qualitative data employed content analysis of the text responses from each open-ended survey question, with a particular emphasis on themes which represented commonalities or a lack of representation across all stakeholders. Network data was analyzed using the NodeXL software and MS Excel to generate visualizations, and specific metrics related to indegree, betweenness and closeness centrality of organizations grouped by organization type. Indegree represents the total number of incoming connections per organization, while weighted indegree represents the sum of weights (strength) of each connection. Closeness centrality represents the relationship of the organization to the centre of the network (lower scores indicate greater centrality). To accommodate for different response rates at baseline and follow up, indegree was calculated as a proportion of incoming connections out of the total respondents (n) for that time point. Weighted indegree was calculated as a proportion of the sum of weights of incoming connections divided by the total possible weighting for the respondents for that time point (i.e. n*5). Statistical comparisons for overall network metrics across T1 and T2 were calculated using a paired t-test in SPSS v.28. While means are also reported by organization type as a subsample of the overall data, no statistical tests were carried out due to small subsample sizes.

The study received ethical approval from Maynooth University and the Sierra Leone Ethics and Scientific Review Committee. Each survey contained a mandatory informed consent section which required completion prior to respondents accessing the survey questions. Respondents were not required to answer any specific questions and were not coerced to participate. All respondents received a unique identification code to preserve anonymity, and any identifying information was removed prior to data analysis.

A total of 27 participants from 24 organizations participated in the surveys across both baseline and follow-up time points (T1 n=20 and T2 n= 16). Nine individuals and 11 organizations were retained across both T1 and T2 surveys. The majority of participants represented International non-governmental organizations (n=9), followed by Organizations of Persons with Disabilities (n=8), Government Ministry (n=4), Service Delivery organisations (n=4) and Academic Institutions (n=2).

Additionally, the respondents were requested to identify multiple areas of AT that their organizations were aligned with. Advocacy ranked as the top selection (24.5%), followed by direct service provision (14.9%), human resources and capacity building (14.9%), policy or systems development (13.8%), product selection and/or procurement (13.8%), data and information systems (11.7%), and financing (6.4%).

Assistive Products in Sierra Leone

Participants were asked to select from the APL which products and/or product services they provide. Manual wheelchairs, crutches, canes, lower limb prosthetics and orthopaedic footwear were the most selected across both time points. Table 1 lists summarises the types of assistive products and services provided in Sierra Leone, and the number of organisations providing each product and/or service across all 50 APL products.

*Other assistive product offered but not on the Assistive Product List **Assistive product not provided, only service related to the prescription, servicing and maintenance, and customization of that Assistive product

Respondents indicated that the products they provide were most commonly procured by their organizations through purchase (38.7%), followed by donation (29%), building products themselves (22.6%) or other (9.7%), which was explicated as recycling used products.

Providers of Assistive Products in Sierra Leone

Participants were asked to indicate whether their organization provided assistive products and/or related services. The findings highlighted 38.3% of stakeholders directly provided AT and 40.4% directly provided AT related services to beneficiaries, while only 21.3% indicated they do not provide AT or AT related services at all.

More specifically, respondents who did indicate providing products and/or services indicated they provided the following services: provision of locally made assistive products, repairs and maintenance of assistive products, education and training of users on the utility of assistive products, referrals of people with disabilities to service providers, prosthetic and orthotics, accessibility assessments, and rehabilitation service provision. Participants whom do not directly provide AT or AT related services indicated their work falls within AT advocacy, fundraising, procurement, policy, and research.

When asked about the challenges they experienced procuring and distributing these products to beneficiaries, qualitative data indicated difficulty sourcing materials, challenges obtaining products due to poor infrastructure, poor quality standards and/or customizability of products, and low technical and managerial support as common barriers. High product and material costs and inadequate funds from both the organizations and beneficiaries was the most commonly cited challenge.

Beneficiaries of Assistive Products in Sierra Leone

When probed on the number of clients they served each month, respondents indicated the range of beneficiaries spanned from as little as 10 per month to upwards of 1000, while one respondent noted there was no fixed number as they serve at the national level. Respondents noted that their beneficiaries were predominantly people with mobility related disabilities or functional limitations (21.4%), closely followed by people with vision disabilities (17.9%), communication disabilities (15.4%) and hearing disabilities (13.1%).

Participants emphasized children and adolescents were the highest populations served, with an equal representation among the ages of 5-12 (23.7%) and 13-18 (23.7%). Adults aged 20-50 years (21%) closely followed, while children under 4 (15.8%) and adults over 50 years of age (15.8%) are equally less represented as beneficiaries of assistive products and services in Sierra Leone.

Respondents whose organizations provide assistive products indicated that their beneficiaries most commonly received APs free of cost (63.2%), followed by client payment (26.3%) and a fixed cost structure (10.5%).

Network Analysis

Respondents were asked to indicate which organizations in the AT network they were aware of, and subsequently to rate the strength of their relationship with the organizations they indicated an awareness of. The degree of relationality among these stakeholders involved in the assistive technology network was then analyzed across the two time points and organizational relationships were visualized using the NodeXL software, presented in Figure 1 and Figure 2 . The colored nodes in the figures depict the various sub-types of organizations, while the lines between the nodes represent their relationships, with thicker lines indicating stronger relationships. The red nodes represent government ministries and agencies, the green represent service delivery organizations, blue represents organization of persons with disabilities, black represents NGOs and yellow represents academic institutions.

Overall, this representation depicts a relatively centralized network with a higher degree of connections between organizations. Furthermore, ministries and government agencies appear towards the centre of the network, indicating a relatively greater role in connecting organizations to one another, however it is noteworthy that these are not the most central organizations in the network.

Table 2 provides quantitative data which demonstrates the overall number and strength of interconnections among the organizations within the assistive technology network in Sierra Leone. Indegree is the number of identified inward connections, or the number of other organizations who identified a connection with that organization. Indegree data are presented as a mean value per organization type to preserve anonymity. The data visualized in Table 2 significantly increased over one year from baseline to follow up, while the relative centrality of organizations did not change, at least over the one-year time period of this study.

SD – standard deviation, NGO – non-governmental organization *Differs significantly from baseline at p<0.01 (two-tailed)

Overall, there was a statistically significant increase in indegree scores between the two timepoints suggesting a higher level of connection among AT organizations in Sierra Leone following the 1-year investment. This suggests those organizations built more relationships and expanded their reach within the AT network. As relationship strength was measured on a 5-point scale (no awareness, awareness, communication, cooperation, collaboration), we can interpret increases in weighted indegree to suggest greater inter-organizational working between members of the network (please refer to Table 2 ).

These findings suggest the one-year intervention did indeed stimulate change within the AT network in Sierra Leone, increasing the number connections within the AT network, and strengthening existing relationships within the network.

The most common assistive products available in Sierra Leone were indicated to be manual wheelchairs, crutches, canes, lower limb prosthetics and orthopaedic footwear. This aligns with our participants ranking mobility related disabilities or functional limitations as the most common reason for beneficiary referral, as well as the rATA data 13 ). The global report on AT notes “the type, complexity, magnitude and duration of a humanitarian crisis impacts the need for and supply of assistive technology”. 1 When we factor in the sociopolitical context of Sierra Leone and its history of civil war, and the population requiring these products due to political violence, such as lower limb amputations, it is also not surprising that mobility related products are so widely available due to population need. Moreover, as many low-income settings procure their products through donations, often from abroad, these items are probable to be in high circulation in relation to the high global prevalence of mobility related disabilities, likely shaping what products donors perceive as being most relevant. 1

Interestingly, data from the rATA shows the people with disabilities who did have AP, most often obtained their product(s) through purchase, despite cost being the most significant barrier to access. 13 As such, these APs were often purchased through informal and unregulated providers who offer lower costs, such as market vendors. 16 In comparison, our findings demonstrated AT stakeholders providing AP did so predominantly at no-cost. This discrepancy could suggest those who need AT most are not aware of the regulated providers who offer free AP and/or AP services in Sierra Leone, or they simply cannot access them due to infrastructural barriers, or not having AT needed to navigate their environment in the first place. For example, our data highlighted only two organizations offering spectacles, yet the rATA indicated spectacles as being the most common AP obtained by people with disabilities sampled in Sierra Leone. This further supports our interpretation that access to free APs is limited if only a small subset of regulated providers are offering them, leading to an increased reliance on people with disabilities procuring APs from informal and unregulated providers in Sierra Leone. An interconnected and coherent national AT network could offer a way forward, should collaborative relationships among AT stakeholders continue to forge and their collective resources, contacts and beneficiaries were to be cross-pollinated for the advancement of beneficiary access.

As technology and what constitutes as AT continues to advance, juxtaposed with the prevalence of disability increasing, there is a risk that the gap in access to AT will continue to rise. 17 It is therefore paramount that the goal of improving AT related outcomes, such as improved access to AT for all, is first warranted by the measurement of such outcomes. 4 This paper has attempted to provide a systemic snapshot of the AT network in Sierra Leone, highlighting key information such as what assistive products are presently available, who provides them, who receives them (and how), and the relational cohesion of the network itself.

This paper is the first to demonstrate that a targeted investment in assistive technology systems and policies at the national level can have a resulting impact on the nature and strength of the assistive technology ecosystem relationships. It is therefore recommended as an intervention to engage stakeholders within the assistive technology space, in particular policy makers who have power to formulate AP related policy and access. However, this work is limited in scope as it only provides a reassessment of outcomes following the one-year investment, and does provide a more longitudinal evaluation of the impact of that investment in the longer term.

Future research is recommended to replicate the work done to date to evaluate whether there is an improvement in access to assistive technologies over a longer period of time as a result of targeted policy and systems change, as well as larger impacts on policy formulation for AP access. For example, attention to data collection of which types and categories of AP are being manufactured locally can inform policy formation to encourage continuity of local manufacturing, while improving access to AP. Moreover, further studies to investigate factors influencing limited uptake of free AP by persons with disabilities, as explicated above and discovered in this study, are recommended.

CONCLUSIONS

Cohesive AT networks are particularly important in low-income settings such as Sierra Leone, where the intersection of poverty and disability disproportionately reduces people with disabilities’ access to the AT they need. Power and colleagues 18 have proposed the Assistive Technology Embedded Systems Thinking (ATEST) Model as a way of conceptualising the embedded relationships between individual-community- system-country-world influences on assistive technology provision. This paper suggests that even where resources are scarce and systemic relationships are uneven, an internationally-funded investment, which embraces the participation of country-level stakeholders and service providing organisations can result in enhanced inter-organisational working, which in turn has the potential to use existing resources more optimally, allowing greater access to services for individuals most in need.

The findings of this paper demonstrate an increase in organizational collaboration can strengthen assistive technology networks, however key barriers to access remain cost for both organizations providing AT and people with disabilities to obtain AT. Future work should use systemic approaches to leverage organizational relationality and prioritize financial accessibility of AT within systemic approaches to AT policy and practice, to leverage existing resources (particularly no-cost AT) and advance towards the ultimate goal of increased access to AT for all.

Ethics Statement

Ethical approval for the study was granted by Maynooth University and the Sierra Leone Ethics and Scientific Review Committee. The study involved human participants but was not a clinical trial. All participants provided informed consent freely and were aware they could withdraw from the study at any time.

Data Availability

All data generated or analysed during this study are included in this article.

This work was funded by the Assistive Technology 2030 project, funded by the United Kingdom Foreign Commonwealth Development Office (FCDO; UK Aid) and administered by the Global Disability Innovation Hub.

Authorship Contributions

Stephanie Huff led the manuscript preparation and contributed to data analysis. Emma M. Smith led the research design, data collection, analysis and contributed to manuscript preparation. Finally, Malcolm MacLachlan contributed to research design, analysis, manuscript review, and supervision. All authors read and approved the final manuscript.

Disclosure of interest

The authors completed the ICMJE Disclosure of Interest Form (available upon request from the corresponding author) and disclose no relevant interests.

Correspondence to:

Emma M. Smith Maynooth University Maynooth, Co. Kildare Ireland [email protected]

Submitted : February 27, 2024 BST

Accepted : June 26, 2024 BST

• Open access
• Published: 19 August 2024

Machine learning model predicts airway stenosis requiring clinical intervention in patients after lung transplantation: a retrospective case-controlled study

• Dong Tian 1 , 2   na1 ,
• Yu-Jie Zuo 1 , 3   na1 ,
• Hao-Ji Yan 4   na1 ,
• Heng Huang 1 ,
• Ming-Zhao Liu 2 ,
• Hang Yang 2 ,
• Jin Zhao 2 ,
• Ling-Zhi Shi 2 &
• Jing-Yu Chen 2  

BMC Medical Informatics and Decision Making volume  24 , Article number:  229 ( 2024 ) Cite this article

165 Accesses

Metrics details

Patients with airway stenosis (AS) are associated with considerable morbidity and mortality after lung transplantation (LTx). This study aims to develop and validate machine learning (ML) models to predict AS requiring clinical intervention in patients after LTx.

Patients who underwent LTx between January 2017 and December 2019 were reviewed. The conventional logistic regression (LR) model was fitted by the independent risk factors which were determined by multivariate LR. The optimal ML model was determined based on 7 feature selection methods and 8 ML algorithms. Model performance was assessed by the area under the curve (AUC) and brier score, which were internally validated by the bootstrap method.

A total of 381 LTx patients were included, and 40 (10.5%) patients developed AS. Multivariate analysis indicated that male, pulmonary arterial hypertension, and postoperative 6-min walking test were significantly associated with AS (all P  < 0.001). The conventional LR model showed performance with an AUC of 0.689 and brier score of 0.091. In total, 56 ML models were developed and the optimal ML model was the model fitted using a random forest algorithm with a determination coefficient feature selection method. The optimal model exhibited the highest AUC and brier score values of 0.760 (95% confidence interval [CI], 0.666–0.864) and 0.085 (95% CI, 0.058–0.117) among all ML models, which was superior to the conventional LR model.

Conclusions

The optimal ML model, which was developed by clinical characteristics, allows for the satisfactory prediction of AS in patients after LTx.

Peer Review reports

Introduction

Lung transplantation (LTx) has been considered the only effective therapeutic option for end-stage lung diseases. The number of lung transplants has been increasing over the last two decades, with approximately 70,000 adult lung transplants performed worldwide thus far [ 1 ]. Since the first clinical LTx in 1963, airway complications (AC) have resulted in substantial mortality and clinical LTx failure [ 2 ]. In recent years, the occurrence of AC has tended to decrease with improvements in surgical techniques, immunosuppression, and patient allocation [ 3 ]. Nevertheless, large studies have reported that the prevalence of AC remains high.

Airway stenosis (AS) refers to a fixed reduction in the caliber of the airway and is the most common AC after LTx with a reported occurrence rate ranging from 1.6%–32.0% in previous studies [ 4 , 5 , 6 , 7 , 8 , 9 ]. The onset of AS usually occurs between 2 and 9 months after LTx [ 10 , 11 ]. A reduction in the cross-sectional area > 50% is confirmation of severe AS, which reduces the quality of life and increases the morbidity and mortality of patients [ 12 ]. Severe AS requires timely clinical intervention to prevent further progression of AS [ 13 ]. Early detection of AS and treatment by balloon dilation can achieve good efficacy [ 14 ]. However, the early stages of AS are difficult to detect since they often present without specific clinical symptoms. Bronchoscopy is the gold standard for diagnosis, but it is usually used in patients who present with clinical symptoms [ 15 ]. Therefore, early and accurate detection of AS requiring clinical intervention is crucial to guide clinical decision-making about subsequent treatment.

Although the published 2018 International Society for Heart and Lung Transplantation (ISHLT) consensus statement reported risk factors for AC, the risk factors for AS remain unclear [ 4 ]. The risk factors for AS are still controversial due to the inconsistency of risk factors among different institutions [ 16 , 17 ]. In addition, the occurrence of AS is difficult to accurately predict by independent risk factors. Identification of AS status requiring clinical intervention using an accurate prediction model could be valuable to conduct optimal treatment and improve outcomes for LTx patients. However, there has been no satisfactory tool to accurately predict AS requiring clinical intervention. Machine learning (ML) algorithms, a branch of artificial intelligence, can integrate clinical characteristics to achieve accurate predictive outcomes [ 18 ]. Our prior research underscored the efficacy of ML algorithms in predicting survival outcomes in LTx patients. Building on this foundation, we endeavored to extend the application of ML models to address the prediction of AS requiring clinical intervention after LTx [ 19 ]. No published research has reported using ML algorithms to predict AS requiring clinical intervention. In this study, we assessed the clinical characteristics of patients and developed ML models to predict AS requiring clinical intervention. Moreover, the conventional logistic regression (LR) model was fitted by independent risk factors and compared in performance to the optimal ML model.

Patients who underwent LTx in Wuxi People’s Hospital affiliated with Nanjing Medical University between January 2017 and December 2019 were included. The study excluded 3 retransplant patients, 3 pediatric lung transplant patients, 2 patients who were lost to follow-up, and 2 patients with incomplete clinical records. Figure  1 shows a flow chart of the included and excluded patients. All the research procedures were consistent with the ISHLT Ethics statement. The Institutional Review Board of Wuxi People’s Hospital affiliated with Nanjing Medical University approved this study (No. 2020 [374]). Patient consent was waived due to the retrospective nature of the study.

Flow diagram for selection of lung transplant recipients. A total of 391 lung transplant recipients were assessed for eligibility. Of this cohort, patients with re-transplant, pediatric lung transplant, lost follow-up, and incomplete clinical records were excluded from the study leaving 381 patients available for the analysis

Parameter measurements

The following variables were extracted from the database: age, body mass index (BMI), sex, diagnosis, surgical type, extracorporeal membrane oxygenation (ECMO) type, ECMO support, preoperative hormone use, grade 3 primary graft dysfunction at 72 h (72 h PGD 3), operation time, postoperative ventilator time, intensive care unit (ICU) stay, postoperative 6-minute walking test (6MWT), cold-ischemia time, and arterial oxygen tension/inspired oxygen fraction (PaO 2 /FiO 2 ). Diagnoses included interstitial lung disease (ILD), chronic obstructive pulmonary disease (COPD), pulmonary arterial hypertension (PAH), and others. By definition, 72 h PGD 3 refers to the syndrome of acute lung injury over the first 72 h after LTx and is clinically manifested by diffuse alveolar infiltration on chest radiographs with PaO 2 /FiO 2  < 200 mmHg (10 mmHg = 1.33 kPa) [ 20 ]. Cold-ischemia time in single lung transplantation (SLTx) was defined as the interval between the beginning of cold perfusion of the donor lung and blood reperfusion during LTx surgery. For double lung transplantation (DLTx), the cold-ischemia time was determined at the end of reperfusion of the second lung.

Surgery and perioperative management

Since January 1, 2015, China has stopped using organs from executed prisoners, and voluntary organ donation has become the only legal source. Each bronchial anastomosis was performed in an “end-to-end” technique avoiding telescoping during LTx surgery. All recipients were treated with regular triple immunosuppressive therapy. Patients underwent routine bronchoscopy after LTx, prior to extubation and prior to discharge to assess the condition of the bronchial anastomoses, and the examination frequency was adjusted according to the actual situation. If patients have obvious airflow limitations such as respiratory distress and wheezing, relevant clinical intervention will be activated. An experienced physician (MZL) evaluated the classification of AS based on all definitions and grading systems of AS in the 2018 ISHLT consensus statement [ 4 ].

Development of the LR model and ML model

Univariate LR was used to select factors associated with AS based on our cohort. Multivariate LR included only factors with a P  < 0.05 in univariate LR. A conventional LR model of AS was developed by LR using independent risk factors. For feature selection, three types of methods were used: filtering, wrapping and embedding, which aim to reduce dimension and avoid overfitting of ML models. Within these three categories of feature selection methods, seven methods were utilized , including LR, determination coefficient (DC), Relief, recursive feature elimination (RFE), Boruta, random forest (RF), and least absolute shrinkage and selection operator (LASSO). Finally, 7 groups of features were determined for the subsequent modeling. For the development of ML model , we applied eight ML algorithms, LR, decision tree (DT), k-nearest neighbors (KNN), naïve bayes (NB), support vector machine (SVM), generalized boosted regression modeling (GBRM), random forest (RF), and extreme gradient boosting (XGB). A total of 56 ML models were developed based on the 8 ML algorithms with 7 feature selection methods for predicting AS requiring clinical intervention. The model with the highest the area under the curve (AUC) was identified as the optimal ML model.

Predictive performance of the LR model and ML model

We compared the predictive performance of the conventional LR model with the optimal ML model for AS requiring clinical intervention. The performance of all models was evaluated in terms of discrimination and calibration. The AUC of the receiver operating characteristic (ROC) curve was used to evaluate the discrimination of the model. The brier score was used to assess the calibration of the model. The brier score ranges from 0 to 1; a score that is close to 0 indicates excellent calibration. Moreover, the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were also evaluated. All statistics were internally validated by the bootstrap method with 1000 resamples.

Patients were stratified into high- and low-risk groups in the optimal ML model based on the threshold determined by ROC. Mean decrease accuracy measures the extent to which each feature’s contribution to the model affects the accuracy of the prediction. It was used to identify features that contributed most significantly to the optimal ML model performance. In addition, the relative importance scores of each predictor in the optimal RF model were assessed using two metrics: Percentage Increase in MSE (percentage increase in mean square error) and Increase in Node Purity. Percentage Increase in MSE measures the impact of the variable on the prediction performance, while Increase in Node Purity measures the contribution of the variable to the purity of the decision tree nodes.

Statistical analysis

Statistical analysis was performed using IBM SPSS Statistics (version 22.0 Inc., Chicago, IL, USA), R programming language (version 4.2.1, Vienna, Austria) and GraphPad Prism (version 10.1.2, CA, USA). Patient demographics and clinical parameters were summarized as the means ± standard deviations for continuous variables and numbers with percentages for categorical variables. The odds ratio (OR) and 95% confidence interval (CI) were calculated. A value of P  < 0.05 was considered statistically significant in all analyses.

Clinical characteristics

The clinical characteristics of the LTx patients are summarized in Table  1 . A total of 381 patients with 244 males and 137 females were enrolled, and the median age of patients was 57 (range, 19–82) years. In the cohort, most of the indications for LTx were ILD (N = 214) and COPD (N = 67). Regarding surgical type, the numbers of patients with SLTx and DLTx were 201 (52.8%) and 180 (47.2%), respectively. The ECMO type was venoarterial (VA) in 120 cases (31.5%) and venovenous (VV) in 150 cases (39.4%); there were 111 cases (29.1%) that did not involve ECMO. In addition, the operation time, postoperative ventilator time, ICU stay, postoperative 6MWT, cold-Ischemia time and PaO 2 /FiO 2 were 327.76 ± 98.39 min, 5.76 ± 12.42 days, 7.78 ± 10.20 days, 460.84 ± 80.58 m, 7.31 ± 2.05 h and 443.55 ± 66.40, respectively. In this study, forty (10.5%) patients encountered AS requiring clinical intervention during the follow-up period.

Univariate analysis indicated that male (OR = 3.535, 95% CI, 1.445–8.650, P  = 0.006), PAH (OR = 9.651, 95% CI, 2.828–32.930, P  < 0.001), VV-ECMO (OR = 0.267, 95% CI, 0.100–0.711, P  = 0.008), and postoperative 6MWT (OR = 0.995, 95% CI, 0.991–0.998, P  = 0.006) were significantly associated with AS requiring clinical intervention. The multivariate analysis further revealed that male (OR = 7.034, 95% CI, 2.232–22.170, P  < 0.001), PAH (OR = 11.249, 95% CI, 2.554–49.549, P  < 0.001), and postoperative 6MWT (OR = 0.993, 95% CI, 0.988–0.997, P  < 0.001) were independent risk factors for AS requiring clinical intervention (Table  2 ). Conventional LR models were established based on independent risk factors. For the ML model, a total of 5, 5, 7, 8, 7, and 7 features were selected for modeling in the DC, Relief, RF, RFE, Boruta, and LASSO methods, respectively (Table  3 ). The combination of 7 feature selection methods and 8 ML algorithms (56 ML models) is shown in a heatmap (Fig.  2 ). The heatmap shows the AUC for the 56 ML models with a median bootstrapped AUC of 0.679 (range 0.569–0.760). The ML model using an RF algorithm with the DC feature selection method exhibited the highest bootstrapped AUC of 0.760 among the models and was confirmed to be the optimal ML model.

Heatmaps of the ML models for predicting AS requiring clinical intervention after LTx. Heatmaps illustrated the performance of each ML algorithm (columns) with each feature selection method (rows), measured by AUC. LR, logistic regression; DT, decision tree; KNN, k-nearest neighbors; NB, naïve bayes; SVM, support vector machine; GBRM, generalized boosted regression modeling; RF, random forest; XGB, extreme gradient boosting; LASSO, least absolute shrinkage and selection operator; RFE, recursive feature elimination; DC, determination coefficient; ML, machine learning; AS, airway stenosis; LTx, lung transplantation; AUC, the area under the curve

The model performance for the prediction of AS requiring clinical intervention is summarized in Table  4 . The differences emerged in the predicted values of the conventional LR and optimal ML models. The bootstrapped AUC of the optimal ML model was 0.760 (95% CI, 0.666–0.864), which was superior to the conventional LR model of 0.689 (95% CI, 0.545–0.803). The brier score of the optimal ML models was 0.085 (95% CI, 0.058–0.117), outperforming the conventional LR models of 0.091 (95% CI, 0.064–0.125). Furthermore, the sensitivity of the optimal ML model versus the conventional LR model was 0.782 (95% CI, 0.526–1.000) versus 0.680 (95% CI, 0.350–1.000). The specificity of the optimal ML model versus the conventional LR model was 0.689 (95% CI, 0.424–0.917) versus 0.623 (95% CI, 0.305–0.956). The PPV of the optimal ML model versus the conventional LR model was 0.252 (95% CI, 0.133–0.429) versus 0.236 (95% CI, 0.105–0.500). The NPV of the optimal ML model versus the conventional LR model was 0.965 (95% CI, 0.927–1.000) versus 0.952 (95% CI, 0.905–1.000).

A histogram established by the optimal threshold of 0.163 indicates different distributions in the optimal ML model between patients in the high- and low-risk groups (Fig.  3 ). The majority of patients in the high-risk groups stratified by the optimal ML model presented with AS requiring clinical intervention, while the majority of patients in the low-risk group presented without AS requiring clinical intervention.

Histogram of the predicted values in patients with and without AS requiring clinical intervention after LTx. Patients were divided into high- and low-risk patients with a cut-off value of 0.163. Most of the high-risk patients presented with AS requiring clinical intervention, while most of the low-risk patients presented without AS requiring clinical intervention. AS, airway stenosis; LTx, lung transplantation

Figure  4 illustrates the ranking of features by importance in the optimal ML model for predicting AS requiring clinical intervention. Mean decrease accuracy was calculated over the optimal ML model for the features considered in the model. The five features of the DC feature selection method were postoperative 6MWT, diagnosis, sex, ECMO type, and preoperative hormone use, with postoperative 6MWT being the most significant. Figure  5 illustrates the relative importance scores of the predictor variables in the optimal RF model. Postoperative 6MWT showed the highest Percentage Increase in MSE with Increase in Node Purity, implying that it had the greatest impact on the predictive performance of the model and contributed the most to the purity of the decision tree nodes.

Variable importance in the optimal RF model. Mean decrease accuracy calculated over the optimal RF model for the features considered in the model. 6MWT: 6-minute walking test; ECMO, extracorporeal membrane oxygenation; RF, random forest

Relative importance score in the optimal RF model. In the optimal RF model, Percentage Increase in MSE measures the impact of the variable on the prediction performance, while Increase in Node Purity measures the contribution of the variable to the purity of the decision tree nodes. 6MWT, 6-minute walking test; ECMO, extracorporeal membrane oxygenation; Percentage Increase in MSE, percentage increase in mean square error; RF, random forest

Considering the significant value of predicting AS requiring clinical intervention in patients after LTx for treatment guidance, we sought to evaluate the clinical characteristics of the patients and further construct prediction models. The following major findings were revealed in this study: (a) Postoperative 6MWT, diagnosis, sex, ECMO type, and preoperative hormone use are five important features of the optimal ML model. (b) Compared with the conventional LR model, the optimal ML model showed better performance in the prediction of AS requiring clinical intervention. (c) The predictive values of the optimal ML model could obviously distinguish patients with AS requiring clinical intervention. Our study suggests that the optimal ML model may become an effective method for predicting AS requiring clinical intervention.

The 6MWT is used to quantify the functional exercise capacity of patients with moderate to severe lung disease [ 21 ]. The negative correlation between the postoperative 6MWT and AS has been described in previous literature [ 22 ]. In our study, postoperative 6MWT was the feature with the highest importance in the optimal ML model, indicating the importance of the postoperative 6MWT in predicting AS requiring clinical intervention. PAH is a progressive hemodynamic disease characterized by proliferation and remodeling of small pulmonary arteries [ 23 ]. We confirmed that PAH is significantly associated with AS requiring clinical intervention. Patients with PAH are prone to hemodynamic instability in the early postoperative period, which may exacerbate the ischemic condition after LTx by limiting collateral blood flow and lead to development of AS. Sex was usually regarded as a potential contributor to posttransplant complications in LTx patients. The present study found that males were related to an increased probability of AS. Castleberry et al. [ 24 ] also reported similar findings. However, Van De Wauwer et al. [ 25 ] concluded that males have no negative impact on AS since the sex of the donor and recipient generally overlap. In our opinion, males, with higher levels of PGD after LTx, can have an inadequate anastomotic blood flow supply, which may induce abnormal airway remodeling and increase the occurrence of AS [ 26 ]. Additionally, lower estrogen levels in males may lack the protective effect on the airway [ 27 ]. VA-ECMO is the bridging modality for patients with respiratory failure awaiting LTx [ 28 ]. However, patients on VA-ECMO inherently demonstrate a higher risk of AS episodes since VA-ECMO is more likely to result in bleeding and thrombotic complications compared to VV-ECMO [ 29 ]. Our study emphasized the necessity of appropriate use of VV-ECMO rather than VA-ECMO in the LTx perioperative period. The present study also found that preoperative hormone use (prednisone) increased the incidence of AS, which is consistent with the study by Park et al. [ 30 ]. Kim et al. [ 31 ] reported that the AC rate did not vary significantly with preoperative hormone use. Nevertheless, they found that the incidence of AC in the first postoperative year remains high after receiving high doses of preoperative prednisone. Hence, the effects of receiving high doses of prednisone preoperatively cannot be ignored. McAnally et al. [ 32 ] concluded that preoperative hormone use may induce related complications, such as poor bronchial anastomotic healing and severe infections, which may be the reason for the increased risk of AS episodes. Therefore, reducing the preoperative dose of prednisone or discontinuing prednisone may be a feasible way to reduce the risk of AS episodes.

ML algorithm is a scientific tool that focuses on how computers learn from data [ 33 ]. It can be applied to clinical characteristics to develop robust risk prediction models and predict patient outcomes [ 34 ]. In previous studies, Hindocha et al. utilized clinical features to develop, validate, and externally test ML model. They found that the ML model might allow satisfactory predictions of survival after treatment for non-small cell lung cancer [ 18 ]. In this study, we constructed 56 ML models by clinical characteristics, and an optimal ML model was developed based on the most appropriate RF algorithm and DC feature selection method. A conventional LR model was constructed based on three independent risk factors. The discrimination, calibration, sensitivity, and specificity of the models highlighted their performance. Finally, the bootstrap method was used to internally validate the two models. The bootstrapped AUCs of the optimal ML model were higher than 0.750, indicating that the optimal ML model had acceptable discrimination. A brier score of 0.085 proves the calibration of the optimal ML model. Both discrimination and calibration demonstrated that the optimal ML model had better performance in predicting AS requiring clinical intervention compared to the conventional LR model.

The optimal ML model has higher sensitivity and specificity than the conventional LR model, further proving that it is an effective prediction model. Our study is the first to assess the predictive value of the optimal ML model for AS requiring clinical intervention in patients after LTx. The important advantage of the optimal ML model is that it exhibits excellent performance and the application of this method does not require data to conform to statistical assumptions, such as the avoidance of independent variable multicollinearity. Although the optimal ML model exhibits the best performance, not all ML models outperform the conventional LR models. Only the ML model constructed with the most appropriate ML algorithm and feature selection method performed best. Additionally, the results of our study do not completely negate the performance of the conventional LR model since they are applicable to different scenarios respectively [ 35 ].

Historically, the conventional LR model is widely used to predict the effect of variables on disease [ 36 ]. Nevertheless, the conventional LR model assumes that the contribution of all clinical characteristics to the model is linear, which is not applicable to clinical practice. ML models can be better applied to deal with high-dimensional and nonlinear clinical characteristics. Therefore, it is more suitable for clinical practice to achieve good performance. Moreover, the histogram of predicted AS requiring clinical intervention showed that the predicted outcomes and actual outcomes of the optimal ML model were approximately equal, indicating excellent performance. The majority of high-risk patients presented with AS requiring clinical intervention, and the most intensive follow-up could be performed for high-risk populations. In future studies, developing ML model by using large sample size data is warranted. The ML model could be used in clinical trials to help clinicians screen out high-risk patients and improve patient prognosis.

The limitations of this current study are presented as follows. First, being retrospective, the study had some inevitable selection bias and the results are less convincing than prospective studies. However, strict inclusion and exclusion criteria were used to control for bias. Second, we performed this study in a single center with a relatively small sample size, which limited the application of the model. Therefore, investigations with a large sample size are warranted in the future. Third, microbial infection, an important risk factor, was not evaluated in this study. As patients present with an infectious condition, they are administered the appropriate clinical intervention to suppress the infectious response, which would have an impact on our study results. Fourth, the dataset was imbalanced, with only 10% of patients developing AS. This imbalance may affect the results and the generalization ability of the ML model. Fifth, the study was limited by the absence of certain clinical characteristics such as lung function, imaging, or pathological data, which could potentially enhance the accuracy of predictions. Last, the validation process was conducted by bootstrap resampling instead of application of an independent validation set. Considering that the patient cohort consisted of only 381 individuals, we needed to keep as many samples as possible for model training in order to enhance the model’s generalization. However, bootstrapping could not provide comprehensive validation for the model.

In this study, postoperative 6MWT, diagnosis, sex, ECMO type, and preoperative hormone use were identified as five important features of the optimal ML model. We constructed ML models that can effectively predict AS requiring clinical intervention for patients after LTx with good performance. The optimal ML model outperformed the conventional LR model in predicting AS requiring clinical intervention. Multicenter studies with large data samples are warranted to further validate the model. The obtained results may enable early and accurate prediction of AS requiring clinical intervention, guiding clinical decisions for subsequent treatment. Future multi-center studies with large data samples are anticipated to further validate the model. Moreover, the deep learning model could potentially be applied to the personalized treatment of LTx patients in the future.

Availability of data and materials

The data that support the findings of this study are available from the corresponding authors upon reasonable request.

Abbreviations

• Airway stenosis

The area under the curve

Body mass index

Chronic obstructive pulmonary disease

Double lung transplantation

Decision tree

Determination coefficient

Extracorporeal membrane oxygenation

Generalized boosted regression modeling

Interstitial lung disease

Intensive care unit

International Society for Heart and Lung Transplantation

K-nearest neighbors

• Lung transplantation

Least absolute shrinkage and selection operator

• Logistic regression
• Machine learning

Naïve bayes

Negative predictive value

Positive predictive value

Percentage increase in mean square error

Pulmonary arterial hypertension

Arterial oxygen tension/inspired oxygen fraction

Random forest

Recursive feature elimination

Receiver operating characteristic

Single lung transplantation

Support vector machine

Venoarterial

Extreme gradient boosting

Grade 3 primary graft dysfunction at 72 h

6- Minute walking test

Chambers DC, Perch M, Zuckermann A, Cherikh WS, Harhay MO, Hayes D Jr, et al. The international thoracic organ transplant registry of the international society for heart and lung transplantation: Thirty-eighth adult lung transplantation report - 2021; Focus on recipient characteristics. J Heart Lung Transplant. 2021;40(10):1060–72.

Article   PubMed   PubMed Central   Google Scholar  

Hardy JD, Webb WR, Dalton ML Jr, Walker GR Jr. Lung homotransplantation in man. Jama. 1963;186:1065–74.

PubMed   CAS   Google Scholar  

Date H, Trulock EP, Arcidi JM, Sundaresan S, Cooper JD, Patterson GA. Improved airway healing after lung transplantation. An analysis of 348 bronchial anastomoses. J Thorac Cardiovasc Surg. 1995;110(5):1424–32 discussion 32–3.

Article   PubMed   CAS   Google Scholar  

Crespo MM, McCarthy DP, Hopkins PM, Clark SC, Budev M, Bermudez CA, et al. ISHLT Consensus Statement on adult and pediatric airway complications after lung transplantation: Definitions, grading system, and therapeutics. J Heart Lung Transplant. 2018;37(5):548–63.

Article   PubMed   Google Scholar  

Santacruz JF, Mehta AC. Airway complications and management after lung transplantation: ischemia, dehiscence, and stenosis. Proc Am Thorac Soc. 2009;6(1):79–93.

Machuzak M, Santacruz JF, Gildea T, Murthy SC. Airway complications after lung transplantation. Thorac Surg Clin. 2015;25(1):55–75.

Moreno P, Alvarez A, Algar FJ, Cano JR, Espinosa D, Cerezo F, et al. Incidence, management and clinical outcomes of patients with airway complications following lung transplantation. Eur J Cardiothorac Surg. 2008;34(6):1198–205.

Thistlethwaite PA, Yung G, Kemp A, Osbourne S, Jamieson SW, Channick C, et al. Airway stenoses after lung transplantation: incidence, management, and outcome. J Thorac Cardiovasc Surg. 2008;136(6):1569–75.

Redmond J, Diamond J, Dunn J, Cohen GS, Soliman AM. Rigid bronchoscopic management of complications related to endobronchial stents after lung transplantation. Ann Otol Rhinol Laryngol. 2013;122(3):183–9.

Felton TW, Roberts SA, Isalska B, Brennan S, Philips A, Whiteside S, et al. Isolation of Aspergillus species from the airway of lung transplant recipients is associated with excess mortality. J Infect. 2012;65(4):350–6.

Puchalski J, Lee HJ, Sterman DH. Airway complications following lung transplantation. Clin Chest Med. 2011;32(2):357–66.

Bin Saeedan M, Rizk A, Yadav R, Ghosh S. Imaging Evaluation of Airway Complications After Lung Transplant. J Comput Assist Tomogr. 2020;44(3):314–27.

Varela A, Hoyos L, Romero A, Campo-Cañaveral JL, Crowley S. Management of Bronchial Complications After Lung Transplantation and Sequelae. Thorac Surg Clin. 2018;28(3):365–75.

Chhajed PN, Malouf MA, Tamm M, Spratt P, Glanville AR. Interventional bronchoscopy for the management of airway complications following lung transplantation. Chest. 2001;120(6):1894–9.

Luecke K, Trujillo C, Ford J, Decker S, Pelaez A, Hazelton TR, et al. Anastomotic airway complications after lung transplant: clinical, bronchoscopic and CT correlation. J Thorac Imaging. 2016;31(5):W62-71.

Yserbyt J, Dooms C, Vos R, Dupont LJ, Van Raemdonck DE, Verleden GM. Anastomotic airway complications after lung transplantation: risk factors, treatment modalities and outcome-a single-centre experience. Eur J Cardiothorac Surg. 2016;49(1):e1-8.

Shofer SL, Wahidi MM, Davis WA, Palmer SM, Hartwig MG, Lu Y, et al. Significance of and risk factors for the development of central airway stenosis after lung transplantation. Am J Transplant Off J Am Soc Transplant Am Soc Transplant Surg. 2013;13(2):383–9.

Article   CAS   Google Scholar  

Hindocha S, Charlton TG, Linton-Reid K, Hunter B, Chan C, Ahmed M, et al. A comparison of machine learning methods for predicting recurrence and death after curative-intent radiotherapy for non-small cell lung cancer: Development and validation of multivariable clinical prediction models. EBioMedicine. 2022;77: 103911.

Tian D, Yan HJ, Huang H, Zuo YJ, Liu MZ, Zhao J, et al. Machine Learning-Based Prognostic Model for Patients After Lung Transplantation. JAMA Netw Open. 2023;6(5): e2312022.

Snell GI, Yusen RD, Weill D, Strueber M, Garrity E, Reed A, et al. Report of the ISHLT working group on primary lung graft dysfunction, part I: definition and grading-A 2016 Consensus Group statement of the international society for heart and lung transplantation. J Heart Lung Transplant. 2017;36(10):1097–103.

Agarwala P, Salzman SH. Six-minute walk test: clinical role, technique, coding, and reimbursement. Chest. 2020;157(3):603–11.

Nęcki M, Latos M, Urlik M, Antończyk R, Gawęda M, Pandel A, et al. Number of Bronchoscopic Interventions in Lung Transplant Recipients Correlates with Respiratory Function Assessed by Pulmonary Function Tests. Ann Transplant. 2021;26: e927025.

Frost AE. The intersection of pulmonary hypertension and solid organ transplantation. Methodist Debakey Cardiovasc J. 2016;12(4 Suppl):10–3.

Castleberry AW, Worni M, Kuchibhatla M, Lin SS, Snyder LD, Shofer SL, et al. A comparative analysis of bronchial stricture after lung transplantation in recipients with and without early acute rejection. Ann Thorac Surg. 2013;96(3):1008–17 discussion 17–8.

Van De Wauwer C, Van Raemdonck D, Verleden GM, Dupont L, De Leyn P, Coosemans W, et al. Risk factors for airway complications within the first year after lung transplantation. Eur J Cardiothorac Surg. 2007;31(4):703–10.

Chacon-Alberty L, Ye S, Daoud D, Frankel WC, Virk H, Mase J, et al. Analysis of sex-based differences in clinical and molecular responses to ischemia reperfusion after lung transplantation. Respir Res. 2021;22(1):318.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Loor G, Brown R, Kelly RF, Rudser KD, Shumway SJ, Cich I, Holley CT, Quinlan C, Hertz MI. Gender differences in long-term survival posttransplant: A single-institution analysis in the lung allocation score era. Clin Transplant. 2017;31(3):10.1111/ctr.12889. https://doi.org/10.1111/ctr.12889 . Epub 2017 Feb 8.

Faccioli E, Terzi S, Pangoni A, Lomangino I, Rossi S, Lloret A, et al. Extracorporeal membrane oxygenation in lung transplantation: Indications, techniques and results. World J Transplant. 2021;11(7):290–302.

Guimbretière G, Anselmi A, Roisne A, Lelong B, Corbineau H, Langanay T, et al. Prognostic impact of blood product transfusion in VA and VV ECMO. Perfusion. 2019;34(3):246–53.

Park SJ, Nguyen DQ, Savik K, Hertz MI, Bolman RM 3rd. Pre-transplant corticosteroid use and outcome in lung transplantation. J Heart Lung Transplant. 2001;20(3):304–9.

Kim HE, Paik HC, Kim SY, Park MS, Lee JG. Preoperative Corticosteroid Use and Early Postoperative Bronchial Anastomotic Complications after Lung Transplantation. Korean J Thorac Cardiovasc Surg. 2018;51(6):384–9.

McAnally KJ, Valentine VG, LaPlace SG, McFadden PM, Seoane L, Taylor DE. Effect of pre-transplantation prednisone on survival after lung transplantation. J Heart Lung Transplant. 2006;25(1):67–74.

Deo RC. Machine Learning in Medicine. Circulation. 2015;132(20):1920–30.

Greener JG, Kandathil SM, Moffat L, Jones DT. A guide to machine learning for biologists. Nat Rev Mol Cell Biol. 2022;23(1):40–55.

Chen Z, Luo H, Xu L. Machine learning models of ischemia/hemorrhage in moyamoya disease and analysis of its risk factors. Clin Neurol Neurosurg. 2021;209: 106919.

Nick TG, Campbell KM. Logistic regression. Methods Mol Biol. 2007;404:273–301.

Download references

Acknowledgements

We would also like to thank American Journal Experts ( https://secure.aje.com/cn/researcher/ ) for editing the English text of a draft of this manuscript.

This study was supported by the National Natural Science Foundation of China (No. 82070059).

Author information

Dong Tian, Yu-Jie Zuo and Hao-Ji Yan contributed equally to this work.

Authors and Affiliations

Department of Thoracic Surgery, West China Hospital, Sichuan University, Chengdu, 610041, China

Dong Tian, Yu-Jie Zuo & Heng Huang

Wuxi Lung Transplant Center, Wuxi People’s Hospital affiliated to Nanjing Medical University, Wuxi, 214023, China

Dong Tian, Ming-Zhao Liu, Hang Yang, Jin Zhao, Ling-Zhi Shi & Jing-Yu Chen

Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, 510120, China

Department of General Thoracic Surgery, Juntendo University School of Medicine, Tokyo, 113-8431, Japan

You can also search for this author in PubMed   Google Scholar

Contributions

DT: Conceptualization, Methodology, Software, Data collection, Statistical analysis, Features extraction, Original draft. YJZ: Conceptualization, Methodology, Software, Data collection, Statistical analysis, Features extraction, Original draft. HJY: Conceptualization, Methodology, Software, Data collection, Statistical analysis, Features extraction, Original draft. HH: Methodology, Data collection, Statistical analysis, Features extraction, Manuscript editing. MZL: Software, Data collection, Statistical analysis, Manuscript editing. HY: Data collection, Features extraction, Manuscript editing. JZ: Data collection, Manuscript editing. LZS: Conceptualization, Methodology, Statistical analysis, Manuscript editing. JYC: Conceptualization, Methodology, Statistical analysis, Manuscript editing.

Corresponding authors

Correspondence to Dong Tian , Ling-Zhi Shi or Jing-Yu Chen .

Ethics declarations

Ethics approval and consent to participate.

The Institutional Review Board of Wuxi People’s Hospital affiliated with Nanjing Medical University approved this study (No. 2020 [374]). Patient does not need to provide informed consent to participate due to waiver by Institutional Review Board of Wuxi People’s Hospital affiliated with Nanjing Medical University.

Consent for publication

Not applicable.

Competing interests

The authors have no conflicts of interest to disclose.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Tian, D., Zuo, YJ., Yan, HJ. et al. Machine learning model predicts airway stenosis requiring clinical intervention in patients after lung transplantation: a retrospective case-controlled study. BMC Med Inform Decis Mak 24 , 229 (2024). https://doi.org/10.1186/s12911-024-02635-8

Download citation

Received : 09 April 2024

Accepted : 14 August 2024

Published : 19 August 2024

DOI : https://doi.org/10.1186/s12911-024-02635-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Prediction model

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 23 August 2024

NK model simulation study of intelligent manufacturing transformation path selection in pharmaceutical manufacturing enterprises

• Wei Zhu 1 , 2 ,
• Ping Ouyang 1 ,
• Shanshan Qiu 2 ,
• Shuqin Li 2 &
• Zhensong Jiang 2  

Scientific Reports volume  14 , Article number:  19646 ( 2024 ) Cite this article

Metrics details

• Information technology
• Sustainability

Under the wave of Industry 4.0, traditional pharmaceutical manufacturing enterprises are in urgent need of intelligent manufacturing transformation and upgrading, and exploring the optimal realization path of intelligent manufacturing transformation can help accelerate the transformation process of pharmaceutical manufacturing enterprises. This paper uses grounded theory research methods to conduct a multi-case study to summarize six influencing factors of the intelligent manufacturing transformation of Chinese pharmaceutical manufacturing enterprises; and then analyzes the correlation between the intelligent manufacturing influencing factors based on the DEMATEL method and the NK model, and identifies the key influencing factors according to this, and then draws the fitness landscape map of the intelligent manufacturing transformation, and finally arrives at the optimal path selection of the intelligent manufacturing transformation of pharmaceutical manufacturing enterprises. The study enriches and extends the research paradigm of intelligent manufacturing transformation, and provides lessons for pharmaceutical manufacturing enterprises to realize intelligent manufacturing transformation and upgrading.

Similar content being viewed by others

Upgrading models, evolutionary mechanisms and vertical cases of service-oriented manufacturing in SVC leading enterprises: Product-development and service-innovation for industry 4.0

Configurational paths to the green transformation of Chinese manufacturing enterprises: a TOE framework based on the fsQCA and NCA approaches

Environmental regulations, green innovation and intelligent upgrading of manufacturing enterprises: evidence from China

Introduction.

With the continuous evolution of technology and industry globally, traditional enterprises are seeking intelligent manufacturing transformation paths to improve their ability to cope with uncertainty, enhance their competitive advantages, and further acquire more resources 1 . Intelligent manufacturing transformation pathways clarify the process of creating new value with new logic and resource utilization by enterprises, which in essence help manufacturing enterprises choose a path that best suits the long-term development of the enterprise in the process of optimizing and restructuring resources 2 , 3 . In this selection process, enterprises must clarify their interconnections with intelligent manufacturing transformation and the key elements for achieving success in intelligent manufacturing transformation 4 , and then gain sustained competitive advantage by choosing the transformation path that best suits the long-term development of the enterprise 5 , 6 . Therefore, it is particularly important for traditional manufacturing enterprises to explore intelligent manufacturing paths and methods that are suitable for their own long-term development.

The intelligent manufacturing transformation of the pharmaceutical manufacturing industry, as an important industry concerning the national economy and people's livelihoods, is particularly urgent 7 . In particular, the outbreak of COVID-19 has drawn attention to the medical industry and caused the state and society to pay more attention to the pharmaceutical manufacturing industry 8 , 9 . At the same time, with the aging of the population and the improvement of health awareness, the market demand for pharmaceutical products continue to grow, while the traditional pharmaceutical production equipment and processes have been difficult to meet market demand, restricting the further development of the industry. Intelligent manufacturing transformation can bring significant advantages to pharmaceutical companies, such as increasing production efficiency, reducing production costs, improving product quality and safety, and helping companies achieve digital management and fine-tune operations 10 . For example, by introducing intelligent manufacturing technology, CR Jiangzhong Pharmaceutical has improved its productivity by 25% and successfully reduced its operating costs by 60%, while effectively reducing its comprehensive energy consumption and carbon emission intensity, and achieved a significant advantage in market competition. Therefore, pharmaceutical manufacturing enterprises must actively explore the road of intelligent manufacturing transformation to be invincible in the future competition.

The issue of intelligent manufacturing transformation path selection in pharmaceutical enterprises refers to the process of selecting the most suitable path for the long-term development of the enterprise in the transformation process, and this selection process describes optimal implementation steps for the enterprise to create new value with new logic. In short, choosing a suitable intelligent manufacturing transformation path can guide enterprises to gain profit and value in a more effective way 11 . With the advent of the artificial intelligence era, more and more pharmaceutical companies are trying to gain sustainable competitive advantage through intelligent manufacturing transformation 12 . Whether an enterprise can find an intelligent manufacturing transformation path suitable for its development is crucial for it to gain a competitive advantage in the fierce market competition 6 , 13 . As a result, how to adopt a feasible decision analysis method to solve the problem of intelligent manufacturing transformation path selection is a research problem worthy of attention in the academic and business communities.

In addition, a review of the literature reveals that while scholars have attempted to identify the factors influencing enterprise transformation 14 and explore the inherent relationship between information technology and intelligent manufacturing transformation and upgrading 15 , 16 , few have adopted a feasible decision analysis methodology to address the issue of intelligent manufacturing transformation path selection. Existing studies have not yet effectively integrated the internal and external influences of intelligent manufacturing in enterprises. They have also ignored the role of path creation evolution in complex systems 17 and failed to accurately reflect the actual results of enterprises’ use of technological tools to achieve intelligent manufacturing.

In view of this, this study selects Chinese pharmaceutical manufacturing enterprises as the research object. Based on the review and summarization of the research literature on the intelligent manufacturing transformation of prior enterprises, this study considers intelligent manufacturing transformation as a complex adaptive system. It explores the influencing factors of the intelligent manufacturing transformation of pharmaceutical manufacturing enterprises through grounded theory methods of multiple case studies. Then, drawing on the NK model theory proposed by Kauffman 18 , this study adopts a computer simulation method to give a selection method for the intelligent manufacturing transformation path for pharmaceutical manufacturing enterprises.

Path selection framework

• Intelligent manufacturing transformation

Intelligent manufacturing transformation is a hot area in the research of enterprise digital transformation. American scholars Wright and Bourne 19 were the first to put forward the concept of “intelligent manufacturing,” which is defined as the process of small-batch production without human intervention, driven by knowledge engineering, manufacturing software systems, and robot vision. Since then, scholars have defined intelligent manufacturing in terms of intelligent design 20 , intelligent production 21 , intelligent management, intelligent service, and integrated perspectives 22 . With the acceleration of the new technological revolution of digitalisation and intelligence, the definition of intelligent manufacturing has been given a new connotation: not only automated and unmanned production but also the ability to help enterprises transition from mass production to customised production through the mechanism of prosumption 2 , 23 , thereby improving production efficiency and optimising resource allocation.

With the wide application of AI technology, the manufacturing industry is transforming from traditional manufacturing to intelligent manufacturing, facilitating enterprises to climb up the global value chain 6 , 12 . Currently, research related to the intelligent manufacturing transformation of enterprises is in its infancy, and theoretical explorations remain rare 24 . Existing literature has explored issues such as the driving force and development mode of intelligent manufacturing and platform building in manufacturing from industrial fields such as home appliance manufacturing 4 , mechanical components 21 , and equipment manufacturing 25 , revealing the role of digitalisation technologies 26 , as well as in the areas of digitalisation strategies, innovation and entrepreneurship, and platforms 4 . However, less research has been conducted on traditional pharmaceutical manufacturing enterprises 10 , and there is a lack of understanding of the transformation process and its intrinsic mechanisms, which fails to address the difficulties faced by pharmaceutical enterprises in their intelligent manufacturing transformation.

In addition, although there have been studies that have provided valuable insights for intelligent manufacturing transformation 10 , 13 , the research on how to choose the intelligent manufacturing transformation path has produced few results. It should be noted that, in the few specific studies on intelligent manufacturing transformation, most of them emphasise the single-factor impact of intelligent manufacturing transformation 6 , 16 , lack systematic thinking about it, and there are fewer studies that use pharmaceutical manufacturing enterprises as specific research objects. Therefore, this study explores a simulation method that can effectively analyse the problem of intelligent manufacturing transformation path selection for pharmaceutical manufacturing enterprises from the perspective of complex systems and with reference to the NK model, to better guide traditional pharmaceutical manufacturing enterprises to achieve high-quality intelligent transformation.

The NK model is a structured simulation method for systems based on the theory of evolutionary biology, which can be used to study how modular systems reach an optimal state at a faster rate through adaptive search 27 . It is particularly suitable for exploring the effects of interactions between elements within a system on the overall adaptability 28 , and is now mainly used to study the evolutionary laws of a system by analyzing the interactions between elements within the system 29 , 30 .

In the NK model, N represents the number of elements of the system, and each element can have two states, 0 or 1, so there are 2^N possible states of the system, and each state corresponds to a fitness value. K represents the degree of interaction between the elements, and a mutation in the alleles of an element affects not only its own fitness value for the system, but also the fitness values determined by the K elements associated with it. Mapping these 2^N states and their corresponding fitness values into three-dimensional space constructs the fitness landscape, which is a three-dimensional graphic consisting of all possible states in the system, taking the form of peaks and valleys. The horizontal coordinates characterise the state combinations of a certain part of the elements in the system, while the vertical coordinates represent the state combinations of the remaining elements, and the height of each point reflects the fitness value of that state combination. The evolutionary process of a complex system can be viewed as a "climb" on the fitness landscape 31 , in which the elements change their states through mutation to find higher peaks, i.e., local optima. The NK model determines the number of local optima and the climbing route needed to reach the summit of the optimal fitness, thus helping to understand the evolutionary mechanism of adaptation in complex systems.

Since Levinthal 32 first used the NK model to explore the relationship between self-organizing behavior and natural selection, the NK model has been widely used in the field of organizational and strategic management research due to its characteristics and advantages 33 , 34 . For example, Hsu et al. 35 used the NK model to solve the integration problem between different departments within an enterprise in supply chain management; Joseph & Gaba 36 used the NK model to explore the interrelationships between organizational structure and performance in multinational corporations; Li et al. 37 applied the NK model to technological innovation; and Ma et al. 38 applied the NK model to the effect of member interactions on organizational performance.

However, this study is based on the NK model and the fitness landscape map, which analyzes the key elements and their interactions in the intelligent manufacturing transformation of an enterprise, and evaluates the fitness of the transformation decisions formed by the key elements to find out the intelligent manufacturing transformation with the highest fitness and to determine the optimal transformation paths to reach that state. The reasons for adopting the NK model in this paper are as follows: First, the computer simulation method can provide a more quantitative description of the influencing elements in the system by integrating massive amounts of data, especially for the intelligent manufacturing transformation, for which the conceptual definition and dimensional division of the dynamic system have not reached agreement; through data simulation, it can effectively deduce the order of the influencing elements of the system. Second, the NK model can simulate the complex relationships within enterprise organizational management, making it suitable for solving complex systems like intelligent manufacturing transformation and predicting its evolutionary patterns.

Although academics have not yet formed a unified view on the definition of intelligent manufacturing transformation 39 , most scholars believe that intelligent manufacturing is a whole composed of a number of interconnected, interacting elements 40 , which have interrelated relationships, and that the transformation of one of the constituent elements induces synergistic transformations in the other elements, or can indirectly change the combinatorial relationship between the constituent elements of intelligent manufacturing, giving uniqueness and complexity to enterprise intelligent manufacturing 41 . However, the complexity that constitutes intelligent manufacturing is not the diversity of elemental forms, but the stochasticity, diversity, and nonlinearity of the interaction relationship between combinations of intelligent manufacturing components, that is, structural complexity, which is very similar to the purpose of using the NK model theory, to judge the evolutionary laws of complex systems.

This study proposes an intelligent manufacturing transformation path selection method based on the NK model for the problem of intelligent manufacturing transformation path selection, whose core idea is to explore the constituent elements of intelligent manufacturing and the logical relationships between these constituent elements, so as to discover the direction and degree of change brought about by changes in the constituent elements of intelligent manufacturing to the complex system of intelligent manufacturing. To this end, this study gives a framework for intelligent manufacturing transformation path selection for pharmaceutical manufacturing enterprises based on the NK model, as shown in Fig.  1 below.

Intelligent manufacturing transformation path selection framework based on the NK model.

According to the intelligent manufacturing transformation path selection framework based on the NK model in Fig.  1 , the optimal path for the intelligent manufacturing transformation of pharmaceutical manufacturing enterprises is derived through simulation. The specific steps are as follows: firstly, the influencing factors of intelligent manufacturing transformation in pharmaceutical manufacturing enterprises are identified by collecting multi-case data and using grounded theory methods; secondly, based on the DEMATEL method to identify the key influencing factors in the intelligent manufacturing transformation (determine the parameter N), and analyze the correlation between the key influencing factors (determine the parameter K), and then combine the different selection results of the influencing factors, which can be obtained as a set of intelligent manufacturing transformation decision options; thirdly, the random distribution function is used to assign fitness values randomly to the selection results of each transformation decision, and the fitness landscape map of intelligent manufacturing transformation is drawn based on the final set of intelligent manufacturing transformation decision options and their corresponding fitness values; fourthly, on this basis, according to the corresponding “climbing way,” the use of computer simulation technology for intelligent manufacturing transformation path selection is employed to get the intelligent manufacturing transformation path selection results.

Analysis of influential factors for intelligent manufacturing transformation

Case selection and data sources, case selection.

In this study, the selection of case samples is based on the criteria of typicality, availability, and matching with the research topic. The object of this paper is pharmaceutical manufacturing enterprises. Firstly, the enterprises within the pharmaceutical manufacturing industry that have at least one high-tech enterprise certification are selected; secondly, non-innovative enterprises that have been established for more than 15 years and have more detailed information disclosure are prioritized; thirdly, to take into account the accessibility and authenticity of the research, this study takes listed companies as the main source of the samples, so that it can be easier to access the sample data and can improve the credibility of the study.

Finally, this study has selected 12 representative pharmaceutical manufacturing enterprises as research samples, which are all leading enterprises in their localities and ranked high in the list of China's Top 100 pharmaceutical industry enterprises. These pharmaceutical manufacturing enterprises have been at the forefront of production intelligence in the national round of concentrated technological transformation of pharmaceutical enterprises since the “12th Five-Year Plan.” Case selection details are provided in Table 1 . Drawing on Shaughnessy's 42 “1/3 principle,” the 12 enterprises were divided into two groups: 2/3 (8) enterprises were used as the modeling group to conduct grounded theory methods and construct theoretical models, and 1/3 (4) enterprises were used as the testing group to conduct theoretical saturation tests. To avoid the interference of subjective factors, 8 enterprises such as China Resources (CR) Jiangzhong were randomly selected from them as modeling samples.

Data sources

According to the triangulation method proposed by Madihally & Matthew 43 , data and case information collection were conducted through multiple ways:

Searching for information about the development history of the enterprise, main products, enterprise news dynamics, and other materials related to the transformation and upgrading of intelligent manufacturing through the official website of the enterprise.

Searching for expert comments, interview records of core enterprise personnel, feature stories, and other relevant information reported by news media on the transformation and upgrading of the enterprise's intelligent manufacturing.

Searching for information about enterprises through information disclosure websites such as Juchao.com, Shenzhen Stock Exchange (SZSE), and Shanghai Stock Exchange (SSE).

Collecting and organizing the data in the process of production and operation of enterprises through digital resources such as Cathay Pacific Database (CSMAR) and China Research Data Service Platform (CNRDS).

The original case information collected were coded. After coding each of the eight case enterprises, if there was a coded content that could not be agreed upon or was significantly different, the remaining four case enterprises were utilized for testing and supplementation; if the test samples were able to support the coded content, the corresponding additions and modifications were made, otherwise the coded content was abandoned.

Case study and model construction

Open coding.

Coding is the analytical process of transforming collected semi-structured interviews and secondary data from simple descriptions into quantitative data 44 . The core element of open coding is to obtain initial categories by summarizing all raw data multiple times through the processes of labeling, defining phenomena, conceptualization, and categorization to extract concepts with the same or similar meanings and categories. In this study, 62 initial categories are finally formed through labeling and refining. Due to space limitations, only CR Jiangzhong is used as a representative for illustration. See Table 2 .

Axial coding

Successive comparisons of data segments are made to discover related ideas or themes, which in turn enables small-scale categorization to study causal relationships between categories 45 . Through axial coding, this study categorized the categories based on the inter-category linkages, which were eventually grouped into 13 main categories, such as information technology use, as shown in Table 3 .

Selective coding

The grounded theory proposed by Glaser et al. 46 uses data collection as a starting point for constructing initial categories and continually explores these to discover a core category that explains the others to improve practical perspectives and develop a formal theory. Based on this, data is analyzed so that the intrinsic connections between the established categories can be explored in depth. In this paper, the main categories are summarized, and then the core categories are identified and saturated through further theoretical sampling and data collection. Finally, technological innovation, pharmaceutical policy, industry competition, organizational management, resource heterogeneity, and market demand are obtained as the core categories, and the results of selective coding are shown in Table 4 below.

In the process of intelligent manufacturing transformation, technological innovation is the development of new technologies by enterprises or the development of new products and services based on existing technologies; pharmaceutical policy is the macroeconomic policy of the government to guide the development of the industry, adjust the imbalance of the industrial structure, build healthy market competition, and provide organizations with the necessary financial subsidies, tax exemptions, technological support, and other resources for industry development; industry competition is the degree of industry competition, which can be used as an indicator to measure the level of competition among enterprises and is also an important factor in regulating the relationship between economic policy uncertainty and enterprise investment behavior; organizational management is the ability of an enterprise to effectively integrate internal and external resources to cope with external threats and to identify and take advantage of external opportunities. It reflects the competitive advantage in a dynamic environment and plays an important role in the organization's strategy and normative behavior; resource heterogeneity encompasses all the things used by the organization to create value, including assets, knowledge, and various capabilities; market demand refers to the sum of the quantities of products or services that a particular customer is willing to purchase in a particular region, time, marketing environment, and marketing plan. Market demand is the main manifestation of the market's regulation of supply and demand, and the new demand generated by consumers can continuously promote the innovative activities of enterprises.

To test the theoretical saturation of the above findings, the core categories included in the theoretical model were tested using an additional four case studies. No new categories were generated, and the six existing categories were saturated with the available case studies, thus confirming that the model was theoretically saturated.

NK model construction for intelligent manufacturing transformation path selection

Symbolic description of model construction.

To elucidate the construction method proposed in this study regarding the NK model, the following notations are relevant:

C = {C 1 , C 2 , …, C n }: the set of intelligent manufacturing transformation elements, in which: C i represents the ith constituent element, i  ∈  {1, 2, …, n}.

P = {P 1 , P 2 , …, P h }: the set of experts involved in the construction of the intelligent manufacturing transformation NK model, where: P o represents the oth expert, o  ∈  {1, 2, …, h}. Here the weight or importance of each expert is considered equal.

Z = {Z 1 , Z 2 , …, Z u }: A set of linguistic evaluation phrases for evaluating the strength of association between intelligent manufacturing transformation elements, where: Z r refers to the rth linguistic phrase in the linguistic phrase set Z, r  ∈  {1, 2, …, r}. Here the set of language evaluation phrases is set as Z = {Z 0  = NO (no association), Z 1  = VL (very low), Z 2  = L (low), Z 3  = H (high), Z 4  = VH (very high)}.

F 0  =  \({\left[{f}_{\text{ij}}^{0}\right]}_{\text{n}\times \text{n}}\) : the association evaluation matrix between the constituent elements given by the expert Po based on the set of linguistic evaluation phrases Z, where: denotes that the expert P o selects a linguistic phrase from the set of linguistic evaluation phrases Z as the evaluation value of the strength of the association effect between the element C i and the element C j , o  ∈  {1, 2, …, h}, i, j  ∈  {1, 2, …, n}. The correlation of the evaluation elements themselves is not considered here, so the main diagonal element of the matrix F p is denoted as “one” and is considered as 0 in the operation.

B = {B 1 , B 2 , …, B N }: the key elements of intelligent manufacturing transformation path selection extracted from the intelligent manufacturing transformation elements C i , where: B s denotes the sth key element, s  ∈  {1, 2, …, N}.

\({\varphi }^{s}=\{{\varphi }_{1}^{s}, {\varphi }_{2}^{s},\cdots ,{\varphi }_{q}^{s}\}\) : the set of alleles of the key elements B s of intelligent manufacturing transformation, where: \({\varphi }_{1}^{s}\) denotes the lth allele of the key elements B s , l  ∈  {1, 2, …, q}. Here, the number of alleles q = 2 can be considered based on the reality, that is, the set of alleles of key elements Bs is \({\varphi }^{s}=\left\{\begin{array}{c}{\varphi }_{1}^{s}=0\end{array}\right.,{\varphi }_{2}^{s}=1\}\) , Where: 0 and 1 denote no element transition and element transition.

M = {M 1 , M 2 , …, M a }: the set of allele combinations of key elements of intelligent manufacturing transformation, where: M d denotes the dth allele combination of key elements of intelligent manufacturing, which can be expressed as \(M_{d} = \varphi_{{l_{1} }}{\prime} \varphi_{{l_{2} }}^{2} \cdots \varphi_{{l_{N} }}^{N} , l_{1} ,l_{2} , \cdots , l_{N} \in \left\{ {\begin{array}{*{20}c} {1,2} \\ \end{array} } \right.,a = q^{N} ,{\text{d}} \in \left\{ {1,2, \cdots ,a} \right\}\) .

\(E={\left[{e}_{s}^{d}\right]}_{a\times N}\) : matrix of adaptability of key elements of intelligent manufacturing transformation, where: \({e}_{s}^{d}\) denotes the effect of an allele of key element B s in the allele combination M d of key elements of intelligent manufacturing transformation on the adaptability of the system, that is, the adaptability value.

The aim of this study is to construct the NK model of intelligent manufacturing transformation based on the key elements B s of intelligent manufacturing transformation, consider the participation of expert P o , generate the fitness landscape map of intelligent manufacturing transformation, and select the path of intelligent manufacturing transformation through computer simulation.

Identification of key elements (identification of parameter N)

Firstly, according to the correspondence between the linguistic evaluation phrase and its subscript value, the linguistic evaluation phrase Z r is transformed into its corresponding subscript value r, r  ∈  {1, 2, …, u \(\}\) , and the arithmetic average method is applied, and the direct correlation evaluation matrix between the constituent elements given by all the experts, \({F}_{0}={\left[{f}_{\text{i}j}^{0}\right]}_{n\times n}\) , is assembled into the constituent elements' direct correlation group evaluation matrix, \(G={\left[{g}_{ij}\right]}_{n\times n}\) , in which the formula for the calculation of the element \({g}_{ij}\) in the matrix is:

Secondly, using the DEMATEL method (Fontela and Gabus, 1976; Si et al., 2018), the constituent direct correlation group evaluation matrix \(G={\left[{g}_{ij}\right]}_{n\times n}\) is normalised to obtain the normalised direct correlation group evaluation matrix \(X={\left[{x}_{ij}\right]}_{n\times n}\) , where the element \({x}_{ij}\) of the matrix is is calculated by the formula:

Based on Markov absorptivity 47 , 48 , it is known that the DEMATEL method assumes that there is at least one i such that \(\sum_{3=1}^{n}{q}_{ij}<\underset{0\le \text{i}\le n}{\text{max}}\left\{\sum_{j=1}^{n}{g}_{ij}\right\}\) holds, and this assumption is reality is mostly satisfied. As a result, the matrix \(x={\left[{x}_{ij}\right]}_{n\times n}\) satisfies the properties (1) \( {lim}_{\tau \to \infty }{\left(\text{X}\right)}^{\tau }=OZ\) ; (2) \({lim}_{\tau \to \infty }\left[1+X+{\left(X\right)}^{2}+\cdots {\left(\text{X}\right)}^{\tau }\right]={\left(1-\text{X}\right)}^{-1}Z\) , where O and I are the zero and the unit matrices.

From the above properties, the indirect correlation evaluation matrix \(Y={\left[{Y}_{ij}\right]}_{n\times n}\) can be constructed for intelligent manufacturing transformation elements respectively, and its calculation formula is:

Further, a comprehensive association evaluation matrix \(T={\left[{\gamma }_{ij}\right]}_{n\times n}\) is constructed for the intelligent manufacturing transformation elements, where \({\gamma }_{ij}\) denotes the sum of the degree of direct association and indirect association, that is, the degree of comprehensive association, of the elements \({C}_{i}\) and \({C}_{j}\) , and \(\text{i},j\in \{\text{1,2},\cdots ,n\}\) . The formula for the element \({\gamma }_{ij}\) in the matrix T is:

Gathering the row and column elements in the matrix \(T={\left[{\gamma }_{ij}\right]}_{n\times n}\) respectively, the centrality \({\alpha }_{i}\) of the constituent elements can be obtained, which is calculated as:

In the formula, the centrality degree \({\alpha }_{i}\) denotes the size of the role played by the corresponding intelligent manufacturing transformation element in the set C of constituent elements. To extract the key elements of intelligent manufacturing transformation, that is, to determine the parameter N in the NK model, the centrality extraction threshold \(\xi \) can be pre-determined, which is calculated as:

In the above formula, χ indicates the maximum centrality percentage of intelligent manufacturing transformation elements, 0 <  \(\chi \) ≤ 1, the value of which is determined by the decision maker based on subjective willingness, experience or historical data, the larger the \(\chi \) , the higher the centrality of the extracted key elements, and accordingly the smaller the number of the extracted key elements (namely elements to be subjected to transformation decisions).

When \({\alpha }_{i}\ge \xi \) is satisfied, the corresponding intelligent manufacturing constituents will be extracted, and finally N constituents will be extracted as the set of key elements for intelligent manufacturing transformation, that is, B = {B 1 , B 2 , …, B N }, where B s denotes the sth key element, that is, B s  = { \(\left.{C}_{i}\right|{\alpha }_{i}\ge \xi ,\) \(\text{i}\in \{\text{1,2},\cdots ,n\}\) }, which determines the value of the parameter N. Clearly, B  ⊂  C.

Complexity identification (determination of parameter K)

On the basis of the direct correlation group evaluation matrix \(G={\left[{g}_{ij}\right]}_{n\times n}\) of the elements of intelligent manufacturing transformation, only the rows and columns where the key elements B s of intelligent manufacturing transformation are located are retained, and then the direct correlation group evaluation matrix \(\Psi ={\left[{\varphi }_{\alpha \beta }\right]}_{N\times N},\alpha ,\beta \in \{\text{1,2},\cdots ,N\}\) of the key elements is obtained. By assembling the elements in \(\Psi \) , the mean value \(\varphi \) of the direct correlation evaluation between the key elements can be obtained, which is calculated as following:

On the basis of \(\Psi ={\left[{\varphi }_{\alpha \beta }\right]}_{N\times N}\) , the mean value \(\overline{\varphi }\) is used as the threshold for judging whether or not there is an association between the elements, and the key element adjacency matrix \(A={\left[{a}_{\alpha \beta }\right]}_{N\times N}\) is constructed, and the formula for calculating the element \({\alpha }_{\alpha \beta }\) in the matrix is:

In the formula, \({a}_{\alpha \beta }=1\) , indicates that there is an influence of key element \(\beta \) on key element \(\alpha \) , i.e., the change of key element \(\beta \) will change the adaptation value of key element \(\alpha \) ; \({a}_{\alpha \beta }=0\) , indicates that there is no influence of key element \(\beta \) on key element \(\alpha \) , i.e., the change of key element \(\beta \) will not change the adaptation value of key element \(\alpha \) or the change is negligible.

Further, assembling the row elements in the matrix \(A={\left[{a}_{\alpha \beta }\right]}_{N\times N}\) yields the key element correlation degree \({k}_{\chi }\) , which is calculated by the formula:

In the formula, \({k}_{\chi }\) denotes the number of key elements that have influence on the key element χ and does not consider the influence of the key element on itself. According to the theory of NK model, the average value of the correlation degree \({k}_{\chi }\) of each key element is the parameter K, which is calculated by the following formula:

In this equation, K represents the average number of times each key element in the system is affected by other elements. Obviously, the larger the value of K, the stronger the interaction between elements in the system and the more complex the system becomes. If the value of K is 0, there is no interaction between the elements in the system and the complexity is the lowest; on the contrary, if the value of K is equal to N minus 1, each element in the system will be affected by the other elements and the complexity will be the highest.

Fitness landscape map generation

Based on the NK model theory, the association relationship expressed in terms of the key element neighbourhood matrix \(A={\left[{a}_{\alpha \beta }\right]}_{N\times N}\) , when the alleles of the key elements Bs are mutated (namely element selection is performed) or the genes of the key elements that have a superordinate relationship with them are mutated, a random number is taken from the (0, 1) uniformly distributed random variable as the fitness value of the key elements Bs. that is:

Based on the constructed adaptation matrix \(E={\left[{e}_{s}^{d}\right]}_{a\times N}\) for key elements of intelligent manufacturing transformation, the adaptation value \({\text{e}}^{d}\) of the allele combination \({M}_{d}\) of key elements of intelligent manufacturing transformation can be obtained, and its calculation formula is:

Combining the bit gene combinations of key elements of intelligent manufacturing transformation and mapping them to the three-dimensional space, the intelligent manufacturing transformation fitness landscape map is constructed, reflecting the interactions between key elements in the process of intelligent manufacturing transformation, as well as all the possible adaptive states of the complex system of intelligent manufacturing, as shown in Fig.  2 below.

Climbing schematics of fitness landscapes. Generated using Matlab R2021b software.

Optimal path selection

Based on the above analysis, it is known that the change exploration process of intelligent manufacturing involves selecting key elements (in general, the number of alleles is 2), and while the state of alleles changes from 0 to 1, the overall adaptive value of the intelligent manufacturing system will continue to increase, which is shown as a state of climbing in the fitness landscape map (Fig.  2 ). According to Fig.  2 , starting from point a, after judging the adaptation value of the surrounding locations, it passes through points b, c, and d, and finally reaches the highest point, point e. Therefore, the process of starting from point a, comparing the overall fitness value of each point, and finally reaching point e after continuous searching is the search process for the transformation path of intelligent manufacturing, and the transformation path of intelligent manufacturing is a → b → c → d → e.

In the transformation process, according to the difference in the number of key elements of intelligent manufacturing transformation involved in each “climb,” the strategy for key elements of intelligent manufacturing transformation can be classified into single-element selection and multiple-element selection. Single-element selection refers to changing the allele values of only one key element at a time and is presented on the fitness landscape map as a process of climbing from one vertex to neighboring vertices, whereas multiple-element selection is a process that can change the allele values of more than one key element at a time and is presented as a process of climbing from a single vertex to more distant vertices. With the increase in the number of key elements for intelligent manufacturing transformation, the probability of finding the global optimal point may increase accordingly, but the risk borne by the enterprise will also increase, so the enterprise should choose properly according to the actual situation. Summarizing the previous section, we can obtain the calculation steps of the intelligent manufacturing transformation path selection method based on the NK model, as shown in Table 5 .

Simulation study of intelligent manufacturing transformation path selection

Based on the composition of the influencing factors of the intelligent manufacturing transformation of pharmaceutical enterprises, a questionnaire was designed to collect data on the direct influence relationship between the influencing factors. The DEMATEL method proposed by Fontela and Gabus 49 was then applied to determine the two key parameters N and K of the NK model. The NK model of adaptability to the intelligent manufacturing transformation was finally constructed, and computer simulation was used to determine the preferred intelligent manufacturing transformation path for pharmaceutical manufacturing enterprises, providing the corresponding selection results and analyses.

Data collection

To obtain the correlation evaluation data between the core elements, an expert committee was set up, consisting of five experts from universities and research institutes of different organizations and five experts from pharmaceutical manufacturing companies. They scored individually and anonymously, and the questionnaire was constructed as a comparative matrix of judgments between the six influencing factors in the form of a five-point Likert scale to evaluate the influence of the row elements on the column elements. The scores ranged from 1 to 5, corresponding to “no correlation” to “very high correlation”, with Z = {Z 0  = NO (no correlation), Z 1  = VL (very low), Z 2  = L (low), Z 3  = H (high), and Z 4  = VH (very high)}. The detailed questionnaire is shown in the appendix.

After evaluating the strengths and weaknesses of the correlations between the different elements mentioned above by the experts, 10 linguistic assessment phrase sets were obtained, and the final elemental correlation assessment matrix was compiled based on these evaluations as:

Determination of key parameters

For the determination of parameter N, the linguistic phrase forms Z 0 , Z 1 , Z 2 , Z 3 , Z 4 provided by the experts are converted to their corresponding numerical values 0, 1, 2, 3, 4 based on the correspondence between the linguistic phrases and their subscripted values. In accordance with Eq. ( 1 ), the constituent element association evaluation matrices (F 1 , F 2 , F 3 , F 4 ) provided by the experts are integrated to arrive at the intelligent manufacturing constituent elements The direct association group evaluation matrix G is:

Based on Eq. ( 2 ), the DEMATEL method is applied to normalise the constituent direct correlation group evaluation matrix G to obtain the normalised direct correlation group evaluation matrix X as follows:

Based on Markov absorptivity (Bosso et al., 1969; Atlaskin et al., 2021), according to matrix X and Eq. ( 3 ), the indirect correlation evaluation matrix Y of intelligent manufacturing transformation elements is constructed as follows:

According to Eq. ( 4 ), the further construction of the comprehensive correlation matrix T of intelligent manufacturing components of pharmaceutical manufacturing enterprises is as follows:

Further, the centrality a i of the constituent elements can be obtained by assembling the row and column elements in the matrix T. The centrality a i indicates the size of the role played by the corresponding intelligent manufacturing transformation element in the set of constituent elements C. According to Eq. ( 5 ), the specific centrality calculation results are shown in Table 6 below.

To extract the key factors of intelligent manufacturing transformation, i.e. to determine the parameter N in the NK model, the centrality extraction threshold \(\xi \) must be determined first, where the maximum centrality percentage \(\chi \) of the intelligent manufacturing transformation factors takes the value of 0.75 according to the experience of previous studies (Kourtellis et al., 2013; Weng et al., 2020). According to formula ( 6 ) and Table 6 , the centrality extraction threshold \(\xi \) = 0.75*11.750 = 8.8125. from Table 6 , the values of centrality a i of key factors are greater than the threshold \(\xi \) , from which it can be obtained that the parameter N = 6.

In this paper, based on the direct correlation group evaluation matrix G, we continue to calculate and analyses the complexity parameter K, keeping the rows and columns where the key element B S is located, and obtain the key element direct correlation group evaluation matrix \(\psi \) . This is shown as follows:

By assembling the elements in \(\psi \) , the mean value \(\overline{\varphi }\) of the direct correlation evaluation between their elements can be calculated, and according to Eq. ( 7 ), the mean value \(\overline{\varphi }\) =77.5/36 = 2.153. Then, according to the threshold value of the mean value \(\overline{\varphi }\) for determining whether there exists a correlation between the elements, and according to Eq. ( 8 ), the element \({\varphi }_{\alpha \beta }\ge \) the mean value \(\overline{\varphi }\) is assigned to be 1, and the element \({\varphi }_{\alpha \beta }<\) the mean value \(\overline{\varphi }\) is assigned to be 0, and the resultant construction of the key element neighbourhood matrix A is shown below:

The number of key elements that have influence on the key element \(\chi \) , i.e., \({k}_{\chi }\) , is obtained by aggregating the row elements in matrix A. According to Eq. ( 9 ), it is calculated that: \({k}_{\chi }\) =21; based on the theory of NK model, the average value of the correlation \({k}_{\chi }\) of each key element is used as the parameter K. K represents the average number of key elements in the system that are affected by the other elements, and according to the Eq. ( 10 ), it is calculated that: K = 3.5.

NK model analysis

The six key factors that may affect intelligent manufacturing transformation are abstracted into six entities, and the entity modeling approach is applied to explore the interactions between the factors and their impact on intelligent manufacturing transformation. Intelligent manufacturing transformation path selection based on the NK model refers to the decision-making process regarding the key elements of intelligent manufacturing transformation during morphological change. This process involves randomly selecting numbers from the interval (0, 1) as the adaptability values, based on the correlation relationship between the key elements as shown by the key elements' neighborhood matrix A. Therefore, based on the state coding of each element of the NK model and the formula for calculating the degree of adaptation, the Matlab R2021b software (version number: R2021b, URL: https://www.mathworks.com/products/matlab.html ) is used to simulate the change process of the degree of adaptation of the factors affecting intelligent manufacturing transformation, as shown in Table 7 . The results are plotted as a fitness landscape map of the intelligent manufacturing transformation, in which the climbing process can be clearly seen, as shown in Fig.  3 .

Fitness landscape and climbing process of intelligent manufacturing transformation in pharmaceutical manufacturing enterprises. Generated using Matlab R2021b software.

To ensure the stability and reliability of the results of the software simulation and analysis, this study simulated the fitness matrix and performed local search 100,000 times. Statistical analyses revealed that: first, the importance of key element B5 in achieving the allele combination of intelligent manufacturing transformation exceeds that of other key elements. Therefore, the first stage of the transformation needs to consider key element B5, namely pharmaceutical policy, which is manifested as a climb from position {000000} to position {000010} on the fitness landscape map, as shown in Fig.  3 a; second, the importance of key element B2 in achieving the B5 allele combination exceeds that of other key elements. Thus, the second stage of the transformation needs to consider key factor B2, namely technological innovation, which is manifested as a process of climbing from position {000010} to position {000010} on the fitness landscape map, as shown in Fig.  3 b; and so on. The results of the simulation and analysis suggest that the intelligent manufacturing transformation of pharmaceutical manufacturing enterprises should be carried out in the order of pharmaceutical policy, technological innovation, organizational management, resource heterogeneity, market demand, and industry competition. In other words, the path of the transformation system adaptability is B5 → B2 → B3 → B6 → B1 → B4.

In the search for the optimal path, the two-dimensional fitness landscape map expression is clearer and more intuitive. Drawing the two-dimensional landscape map, as shown in Fig.  4 below, the first line of numbers in the box represents the combination state of the factors affecting intelligent manufacturing. The second line of numbers represents the fitness value corresponding to the combination state of the factors. Among them, the mountain peaks represent the states of the combinations that have a high fitness value. The line with an arrow is the path of the landscape map, indicating the path from the lower fitness combination state to the neighboring higher fitness combination state. The optimal path of the process of migration of each layer of the factor combination to the point of the global optimal state is shown as a dashed arrow route, which is shown as follows:

Two-dimensional fitness landscape map.

000000 → 000010 → 010010 → 011010 → 011011 → 111,011 → 111,111.

Based on Figs.  3 and 4 , the intelligent manufacturing transformation path climbing diagram of pharmaceutical manufacturing enterprises shows the climbing process of the influential elements of intelligent manufacturing transformation from point 00000 to 111,111. This indicates that pharmaceutical manufacturing enterprises in the process of intelligent manufacturing transformation should first consider the pharmaceutical policy, which is the government's macroeconomic policy to guide the development of the industry, adjust the imbalance of the industrial structure, and build a well-ordered market competition. Secondly, they should consider technological innovation, meaning the enterprise develops new technologies or new products and services based on existing technologies. Thirdly, organizational management should be considered, which is the enterprise's ability to effectively integrate internal and external resources to cope with external threats and identify and exploit external opportunities, reflecting its competitive advantage in a dynamic environment. Fourthly, the heterogeneity of resources should be considered, encompassing all the things used by the organization to create value, including assets, knowledge, and various capabilities. Fifthly, market demand should be considered, referring to the quantity of products or services that a particular customer is willing to buy in a particular region, time, marketing environment, and marketing plan. Finally, industry competition should be considered, representing the level of competition in an industry that measures the degree of competition among enterprises, which is a key factor in adjusting for economic policy uncertainty and the investment behaviors of enterprises.

Conclusion and discussion

Conclusions.

Each industrial revolution promotes the transformation of organizational forms, bringing new phenomena and issues for business transformation, which contribute to the continuous development of organizational theory knowledge systems 15 , 40 . Some scholars have studied the influencing factors and patterns of intelligent manufacturing transformation from different perspectives 3 , 4 , 10 . However, few consider intelligent manufacturing transformation as a complex adaptive system to explore the influencing factors of pharmaceutical manufacturing enterprises and the optimal path choice for intelligent manufacturing transformation. In this study, based on the NK model theory, we use computer simulation technology for the path selection of intelligent manufacturing transformation.

The findings of the study can be summarized as follows: First, the study identified the main factors affecting the intelligent manufacturing transformation of pharmaceutical manufacturing enterprises as technological innovation, pharmaceutical policy, industry competition, organizational management, resource heterogeneity, and market demand. Understanding the interactions between these factors can help pharmaceutical enterprises develop more effective intelligent manufacturing transformation strategies and avoid potential risks. Second, in the context of intelligent transformation, pharmaceutical manufacturing enterprises should consider improving the degree of adaptability of the intelligent manufacturing transformation system in the order of “Pharmaceutical policy → Technological innovation → Organizational management → Resource heterogeneity → Market demand → Industry competition.” The proposed path determines the implementation order of core elements of intelligent manufacturing under a complex dynamic system, which can help pharmaceutical manufacturing enterprises prioritize the development of more critical transformation elements and enhance the performance of intelligent manufacturing transformation under the background of intelligentization.

Theoretical contributions

Firstly, exploring the intelligent manufacturing transformation issue from the perspective of complex systems helps to systematically understand the internal logic and evolutionary process of intelligent manufacturing. Although previous studies have explored the issue of intelligent manufacturing transformation a great deal based on the perspectives of technology, resources, and markets 14 , 23 , they have focused on the enumeration among individual elements and neglected the synergistic effects among the constituent elements of intelligent manufacturing 4 , 25 , the lack of systematic thinking on intelligent manufacturing transformation, but also did not explain the order in which these elements can form the optimal path of intelligent manufacturing transformation, cannot give the real ‘path’. Based on the theory of complex adaptive systems, this paper considers the interactions between elements, explores the synergistic influence of multiple intelligent manufacturing elements on the overall transformation through the perspective of complex systems, and selects the optimal path of intelligent manufacturing transformation with the sequential order of the elements by using the simulation method of the NK model, which expands the research on the path of intelligent manufacturing transformation, and provides new references for the systematic understanding of intelligent manufacturing transformation.

Secondly, based on the NK model theory, the sequence of elements in the optimal path of intelligent manufacturing transformation of pharmaceutical manufacturing enterprises is clearly delineated. Since the application of NK model theory to organisational management, scholars have mainly applied it to the relationship between enterprises 34 , 35 , the organisational performance of enterprises 30 , 33 , and very few researches have paid attention to the selection of paths for intelligent manufacturing transformation in pharmaceutical manufacturing enterprises. In contrast, this paper explores the optimal path selection for intelligent manufacturing transformation of pharmaceutical manufacturing enterprises based on the NK model theory and through the Matlab computer simulation method, which clearly delineates the sequential order of the elements that form the optimal path for intelligent manufacturing transformation of pharmaceutical manufacturing enterprises and expands the application field of the NK model theory.

Practical implications

First, for manufacturing enterprises, the interrelationship between influencing factors should be comprehensively considered in the process of intelligent manufacturing transformation, and systematic thinking should be adopted to formulate transformation strategies. At the same time, manufacturing enterprises are at different stages and levels in the process of intelligent manufacturing transformation. For manufacturing enterprises in different conditions or states, they can draw on simulation methods to select the path most suitable for their development by systematically analyzing and evaluating the advantages and disadvantages of different transformation paths. This method can help enterprises better understand the various possible influencing factors and provide a scientific basis for transformation decisions.

Second, for the government, it should formulate more precise and targeted policies to support the intelligent manufacturing transformation of pharmaceutical manufacturing enterprises. At the same time, the government can build an industry-university-research cooperation platform to promote cooperation and exchanges between pharmaceutical manufacturing enterprises and scientific research institutions, universities, and other relevant units.

Research limitations and prospects

Although this paper explains the path selection of intelligent manufacturing transformation of pharmaceutical manufacturing enterprises and forms some valuable findings, there are still some shortcomings: first, this paper only explores the intelligent manufacturing transformation influencing factors and path selection of pharmaceutical manufacturing enterprises, and there may be different evolutionary development path in other traditional manufacturing industries, which is worthy of in-depth exploration in the future. It is possible to further extend the study to other industries, and explore the commonalities and differences of intelligent manufacturing transformation paths in different industries to form a more general theoretical framework. Secondly, intelligent manufacturing transformation of manufacturing enterprises has gradually become a necessary path for the transformation and upgrading development of enterprises, but the relevant research is still insufficient, and the path selection can be tested in the future using empirical and other research methods. Empirical data can be further collected to validate the path selection framework proposed in this paper and explore more effective path selection methods.

Ethical approval

This study involved human participants. Informed consent was obtained from all subjects. The participants were fully informed about the purpose and procedure of the study, and their participation was entirely voluntary. Data collection was conducted anonymously to protect the participants' privacy.

Data availability

All data generated or analysed during this study are included in this published article.

Wu, L., Sun, L., Chang, Q., Zhang, D. & Qi, P. How do digitalization capabilities enable open innovation in manufacturing enterprises? A multiple case study based on resource integration perspective. Technol. Forecast. Soc. Chang. 184 , 122019 (2022).

Article   Google Scholar  

Kristoffersen, E., Blomsma, F., Mikalef, P. & Li, J. The smart circular economy: A digital-enabled circular strategies framework for manufacturing companies. J. Bus. Res. 120 , 241–261 (2020).

Liu, Y., Zhang, J. Z., Jasimuddin, S. & Babai, M. Z. Exploring servitization and digital transformation of manufacturing enterprises: Evidence from an industrial internet platform in China. Int. J. Production Res. 62 (8), 2812–2831 (2024).

Kusiak, A. Smart manufacturing. In Springer Handbook of Automation. Springer International Publishing, 973-985. (2023).

Smith, W. K. & Besharov, M. L. Bowing before dual gods: How structured flexibility sustains organizational hybridity. Admin. Sci. Quarterly 64 (1), 1–44 (2019).

Yin, C. & Song, W. Research on the driving mechanism of intelligent renovation and digital transformation in traditional enterprises. J. Elect. Syst. 20 (2), 713–724 (2024).

Li, J. Research on the development trend and prospect of the pharmaceutical industry under the impact of COVID-19. Highlights in Bus. Econ. Manag. 15 , 58–63 (2023).

Ding, H., Pu, B. & Ying, J. Direct and spillover portfolio effects of COVID-19. Res. Int. Bus. Finance 65 , 101932 (2023).

Article   PubMed   PubMed Central   Google Scholar  

Ardolino, M. et al. The Impacts of digital technologies on coping with the COVID-19 pandemic in the manufacturing industry: A systematic literature review. Int. J. Prod. Res. 62 (5), 1953–1976 (2024).

Zhu, W., Ouyang, P. & Kong, M. Research on the evolution mechanism of intelligent manufacturing transformation of Chinese pharmaceutical manufacturing enterprises based on system dynamics. Heliyon https://doi.org/10.1016/j.heliyon.2024.e33959 (2024).

Opazo-Basáez, M., Vendrell-Herrero, F., Bustinza, O. F., Vaillant, Y. & Marić, J. Is digital transformation equally attractive to all manufacturers? Contextualizing the operational and customer benefits of smart manufacturing. Int. J. Phys. Distrib. Logistics Manag. 53 (4), 489–511 (2023).

Sharma, D., Patel, P. & Shah, M. A comprehensive study on Industry 4.0 in the pharmaceutical industry for sustainable development. Environ. Sci. Poll. Res. 30 (39), 90088–90098 (2023).

Article   CAS   Google Scholar  

Agolla, J. E. Human capital in the smart manufacturing and industry 4.0 revolution. Dig. Trans. Smart Manufact. 28 (2), 41–58 (2018).

Google Scholar  

Hagiu, A. & Wright, J. Multi-sided platforms. Int. J. Ind. Organ. 43 , 162–174 (2015).

Lin, D., Lee, C. K., Lau, H. & Yang, Y. Strategic response to Industry 4.0: an empirical investigation on the Chinese automotive industry. Ind. Manag. Data Syst. 118 (3), 589–605 (2018).

Xing, X., Chen, T., Yang, X. & Liu, T. Digital transformation and innovation performance of China’s manufacturers? A configurational approach. Technol. Soc. 75 , 102356 (2023).

Barrett, M., Davidson, E., Prabhu, J. & Vargo, S. L. Service innovation in the digital age. MIS Quarterly 39 (1), 135–154 (2015).

Kauffman, S. A. The origins of order: Self-organization and selection in evolution (Oxford University Press, 1993).

Book   Google Scholar  

Wright, P. K. & Bourne, D. A. Manufacturing intelligence (Addison-Wesley Longman Publishing Co., 1988).

Wang, Z. et al. A design method for an intelligent manufacturing and service system for rehabilitation assistive devices and special groups. Adv. Eng. Inf. 51 , 101504 (2022).

Wang, L., Chen, X. & Liu, Q. A lightweight intelligent manufacturing system based on cloud computing for plate production. Mobile Netw. Appl. 22 , 1170–1181 (2017).

Yang, T., Yi, X., Lu, S., Johansson, K. H. & Chai, T. Intelligent manufacturing for the process industry driven by industrial artificial intelligence. Engineering 7 (9), 1224–1230 (2021).

Ren, S. et al. A comprehensive review of big data analytics throughout product lifecycle to support sustainable smart manufacturing: A framework, challenges and future research directions. J. Cleaner Prod. 210 , 1343–1365 (2019).

Battistoni, E., Gitto, S., Murgia, G. & Campisi, D. Adoption paths of digital transformation in manufacturing SME. Int. J. Prod. Econ. 255 , 108675 (2023).

Osterrieder, P., Budde, L. & Friedli, T. The smart factory as a key construct of industry 4.0: A systematic literature review. Int. J. Prod. Econ. 221 , 107476 (2020).

Ying, W., Pee, L. G. & Jia, S. Social informatics of intelligent manufacturing ecosystems: A case study of KuteSmart. Int. J. Inf. Manag. 42 , 102–105 (2018).

Frenken, K. & Mendritzki, S. Optimal modularity: a demonstration of the evolutionary advantage of modular architectures. J. Evolut. Econ. 22 , 935–956 (2012).

Pathak, S. D., Day, J. M., Nair, A., Sawaya, W. J. & Kristal, M. M. Complexity and adaptivity in supply networks: Building supply network theory using a complex adaptive systems perspective. Decision Sci. 38 (4), 547–580 (2007).

Arévalo, L. E. B. & Espinosa, A. Theoretical approaches to managing complexity in organizations: A comparative analysis. Estudios Gerenciales 31 (134), 20–29 (2015).

Deng, J., Liu, S., Xie, C. & Liu, K. Risk coupling characteristics of maritime accidents in Chinese inland and coastal waters based on NK model. J. Marine Sci. Eng. 10 (1), 4 (2021).

Mitleton-Kelly, E. Ten principles of complexity and enabling infrastructures. Complex Syst. Evolu. Persp. Organ. Appl. Complexity Theory Organ. 1 , 23–50 (2003).

Levinthal, D. A. Adaptation on rugged landscapes. Manag. Sci. 43 (7), 934–950 (1997).

Zhang, G. NK model and the construction of enterprise management strategy. Acad. J. Bus. Manag. 6 (5), 95–101 (2024).

Arend, R. J. Balancing the perceptions of NK modelling with critical insights. J. Innov. Entrep. 11 (1), 23 (2022).

Hsu, C. W., Kuo, T. C., Chen, S. H. & Hu, A. H. Using DEMATEL to develop a carbon management model of supplier selection in green supply chain management. J. Clean. Prod. 56 , 164–172 (2013).

Joseph, J. & Gaba, V. Organizational structure, information processing, and decision-making: A retrospective and road map for research. Acad. Manag. Ann. 14 (1), 267–302 (2020).

Li, F., Chen, J. & Ying, Y. Innovation search scope, technological complexity, and environmental turbulence: A NK simulation. Sustainability 11 (16), 4279 (2019).

Ma, J., Xi, Y., Li, P. & Guo, J. E. Evolution of organizational adaptability: Application of Hexie management theory. Int. J. Comp. Intell. Res. 3 (1), 85–90 (2007).

Kusiak, A. Smart manufacturing. Int. J. Prod. Res. 56 (1–2), 508–517 (2018).

Thoben, K. D., Wiesner, S. & Wuest, T. “Industrie 4.0” and smart manufacturing-a review of research issues and application examples. Int. J. Auto. Technol. 11 (1), 4–16 (2017).

Mittal, S., Khan, M. A., Romero, D. & Wuest, T. Smart manufacturing: Characteristics, technologies and enabling factors. In: Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture, 233(5), 1342-1361. (2019).

Shaughnessy, J. M. Misconceptions of probability: An experiment with a small-group, activity-based, model building approach to introductory probability at the college level. Edu. Stud. Math. 8 (3), 295–316 (1977).

Madihally, S. V. & Matthew, H. W. Porous chitosan scaffolds for tissue engineering. Biomaterials 20 (12), 1133–1142 (1999).

Article   CAS   PubMed   Google Scholar  

Moser, A. & Korstjens, I. Series: Practical guidance to qualitative research. Part 3: Sampling, data collection and analysis. Eur. J. Gen. Pract. 24 (1), 9–18 (2018).

Article   PubMed   Google Scholar  

Irving, J. et al. Ras pathway mutations are prevalent in relapsed childhood acute lymphoblastic leukemia and confer sensitivity to MEK inhibition. Blood J. Am. Soc. Hematol. 124 (23), 3420–3430 (2014).

CAS   Google Scholar  

Glaser, N., Jackson, V., Holzmann, M. J., Franco-Cereceda, A. & Sartipy, U. Aortic valve replacement with mechanical vs. biological prostheses in patients aged 50–69 years. Eur. Heart J. 37 (34), 2658–2667 (2016).

Bosso, J. A., Sorarrain, O. M. & Favret, E. E. Application of finite absorbent Markov chains to sib mating populations with selection. Biometrics 25 , 17–26 (1969).

Article   MathSciNet   CAS   PubMed   Google Scholar  

Atlaskin, A. A. et al. Towards the potential of trihexyltetradecylphosphonium indazolide with aprotic heterocyclic ionic liquid as an efficient absorbent for membrane-assisted gas absorption technique for acid gas removal applications. Sep. Purif. Technol. 257 , 117835 (2021).

Fontela, E. & Gabus, A. The DEMATEL observer. DEMATEL 1976 Report. Switzerland, Geneva, Battelle Geneva Research Center. (1976).

Download references

Acknowledgements

The authors would like to thank the editors and reviewers of Scientific Reports for improving this study.

This research was supported by the National Social Science Foundation of China (23BGL313); National Natural Science Foundation of China (72362025); Jiangxi University Humanities and Social Sciences Research Project (GL23133); Jiangxi Province Vocational Education Teaching Reform Research Project (JXJG-23-49-11); Jiangxi Provincial Social Science Foundation Project (24GL65D).

Author information

Authors and affiliations.

School of Economics and Management, Jiangxi Polytechnic University, Jiujiang, 332000, China

Wei Zhu, Ping Ouyang & Xue Ke

Research Center of Management Science and Engineering, Jiangxi Normal University, Nanchang, 330022, China

Wei Zhu, Shanshan Qiu, Shuqin Li & Zhensong Jiang

You can also search for this author in PubMed   Google Scholar

Contributions

W.Z: Conceptualization, methodology, data curation, writing-review and editing; P.O: methodology, validation, writing-review and editing; X.K: investigation, data curation; S.Q: Data collection, data analysis, Supervision; S.L: Data collection, data analysis; Z.J: Data collection, data analysis.All authors reviewed the manuscript.

Corresponding authors

Correspondence to Wei Zhu or Ping Ouyang .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary information 1., supplementary information 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Zhu, W., Ouyang, P., Ke, X. et al. NK model simulation study of intelligent manufacturing transformation path selection in pharmaceutical manufacturing enterprises. Sci Rep 14 , 19646 (2024). https://doi.org/10.1038/s41598-024-70502-7

Download citation

Received : 07 May 2024

Accepted : 18 August 2024

Published : 23 August 2024

DOI : https://doi.org/10.1038/s41598-024-70502-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Pharmaceutical manufacturing enterprises
• Influencing factors
• Path selection

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Scatterplot selection for dimensionality reduction in multidimensional data visualization

• Regular Paper
• Published: 23 August 2024

Cite this article

• Kaya Okada   ORCID: orcid.org/0000-0002-2552-0163 1 &
• Takayuki Itoh 1  

18 Altmetric

Explore all metrics

Dimensionality reduction (DR) techniques for multidimensional data serve as powerful tools for visualization and understanding of the structure of the data. Various DR methods have been developed to extract specific features of the data over the years. However, selection of the optimal DR method and fine-tuning parameters are still challenging, as these choices vary based on the characteristics of the dataset. Consequently, data scientists often rely on their experience or undertake extensive experimentation to identify the most suitable approach. This paper proposes a semi-automatic method for selecting appropriate DR techniques through scatterplot evaluation. Initially, our approach applies a range of DR methods to the given multidimensional data to compute two-dimensional values. Next, we generate scatterplots from the two-dimensional data and calculate scores reflecting the distribution and spatial relationships among the points. Scatterplots that provide insights achieve higher scores, enabling an efficient selection of DR methods based on their visualization. We demonstrate the effectiveness of the presented method through two case studies: The first one is an e-commerce review dataset, and the second focuses on a dataset derived from music feature extraction.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Explore related subjects

• Artificial Intelligence

Agis D, Pozo F (2019) A frequency-based approach for the detection and classification of structural changes using t-sne. Sensors 19(23):5097

Article   Google Scholar  

Anowar F, Sadaoui S, Selim B (2021) Conceptual and empirical comparison of dimensionality reduction algorithms (pca, kpca, lda, mds, svd, lle, isomap, le, ica, t-sne). Comput Sci Rev 40:100378

Article   MathSciNet   Google Scholar  

Aupetit M, Sedlmair M (2016) Sepme: 2002 new visual separation measures. In: 2016 IEEE pacific visualization symposium (PacificVis), pp. 1–8. IEEE

Ayesha S, Hanif MK, Talib R (2020) Overview and comparative study of dimensionality reduction techniques for high dimensional data. Inf Fus 59:44–58

Caliński T, Harabasz J (1974) A dendrite method for cluster analysis. Commun Stat Theory Methods 3(1):1–27

Dang TN, Wilkinson L (2014) Scagexplorer: Exploring scatterplots by their scagnostics. In: 2014 IEEE Pacific visualization symposium, pp 73–80. IEEE

Engel D, Hüttenberger L, Hamann B (2012) A survey of dimension reduction methods for high-dimensional data analysis and visualization. In: Visualization of large and unstructured data sets: applications in geospatial planning, modeling and engineering-proceedings of IRTG 1131 Workshop 2011. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik

Fodor IK (2002) A survey of dimension reduction techniques. Technical report, Lawrence Livermore National Lab., CA (US)

Gao T (2021) Simcse: simple contrastive learning of sentence embeddings

Gao T, Yao X, Chen D (2021) Simcse: simple contrastive learning of sentence embeddings. arXiv preprint. arXiv:2104.08821

Harrison L, Yang F, Franconeri S, Chang R (2014) Ranking visualizations of correlation using weber’s law. IEEE Trans Visual Comput Graph 20(12):1943–1952

Nadia Syed HS, Jamil NW (2023) A comparative study of hybrid dimension reduction techniques to enhance the classification of high-dimensional microarray data. In: 2023 IEEE 11th conference on systems, process & control (ICSPC), pp 240–245

Heiser CN, Lau KS (2020) A quantitative framework for evaluating single-cell data structure preservation by dimensionality reduction techniques. Cell Rep 31(5)

Huang H, Wang Y, Rudin C, Browne EP (2022) Towards a comprehensive evaluation of dimension reduction methods for transcriptomic data visualization. Commun Biol 5(1):719

Huang S, Ward MO, Rundensteiner EA (2005) Exploration of dimensionality reduction for text visualization. In: Coordinated and multiple views in exploratory visualization (CMV’05), pp 63–74. IEEE

Itoh T, Nakabayashi A, Hagita M (2023) Multidimensional data visualization applying a variety-oriented scatterplot selection technique. J Visual 26(1):199–210

Lee JH, McDonnell KT, Zelenyuk A, Imre D, Mueller K (2013) A structure-based distance metric for high-dimensional space exploration with multidimensional scaling. IEEE Trans Visual Comput Graph 20(3):351–364

Malik HK, Al-Anber NJ, Al-Mekhlafi FAE (2023) Comparison of feature selection and feature extraction role in dimensionality reduction of big data. J Tech 5(1):184–192

Google Scholar  

Matute J, Telea AC, Linsen L (2017) Skeleton-based scagnostics. IEEE Trans Visual Comput Graph 24(1):542–552

Nanga S, Bawah AT, Acquaye BA, Billa MI, Baeta FD, Odai NA, Obeng SK, Nsiah AD (2021) Review of dimension reduction methods. J Data Anal Inf Process 9(3):189–231

Ni J (2018) Amazon review data

Padron-Manrique C, Vázquez-Jiménez A, Esquivel-Hernandez DA, Martinez Lopez YE, Neri-Rosario D, Sánchez-Castañeda JP, Giron-Villalobos D, Resendis-Antonio O (2022) Diffusion on PCA-UMAP manifold captures a well-balance of local, global, and continuum structure to denoise single-cell RNA sequencing data. bioRxiv, pp 2022–06

Remeseiro B, Bolon-Canedo V (2019) A review of feature selection methods in medical applications. Comput Biol Med 112:103375

Saini O, Sharma S (2018) A review on dimension reduction techniques in data mining. Comput Eng Intell Syst 9(1):7–14

Sedlmair M, Tatu A, Munzner T, Tory M (2012) A taxonomy of visual cluster separation factors. In: Computer graphics forum, vol 31, pp 1335–1344. Wiley Online Library

Singh KN, Devi SD, Devi HM, Mahanta AK (2022) A novel approach for dimension reduction using word embedding: an enhanced text classification approach. Int J Inf Manage Data Insights 2(1):100061

Sips M, Neubert B, Lewis JP, Hanrahan P (2009) Selecting good views of high-dimensional data using class consistency. In: Computer graphics forum, vol 28, pp 831–838. Wiley Online Library

Stolarek I, Samelak-Czajka A, Figlerowicz M, Jackowiak P (2022) Dimensionality reduction by umap for visualizing and aiding in classification of imaging flow cytometry data. Iscience 25(10)

Van Der Maaten L, Postma EO, van den Herik HJ et al (2009) Dimensionality reduction: a comparative review. J Mach Learn Res 10(66-71):13

Vashisth P, Meehan K (2020) Gender classification using twitter text data. In: 2020 31st Irish signals and systems conference (ISSC), pp 1–6. IEEE

Wang K, Yang Y, Fangjiang W, Song B, Wang X, Wang T (2023) Comparative analysis of dimension reduction methods for cytometry by time-of-flight data. Nat Commun 14(1):1836

Wang Y, Wang Z, Liu T, Correll M, Cheng Z, Deussen O, Sedlmair M (2019) Improving the robustness of scagnostics. IEEE Trans Visual Comput Graph 26(1):759–769

Wien T (2015) Music information retrieval

Wilkinson L, Anand A, Grossman R (2005) Graph-theoretic scagnostics. In: Information visualization, IEEE symposium on, pp 21–21. IEEE Computer Society

Yamada I, Asai A, Sakuma J, Shindo H, Takeda H, Takefuji Y, Matsumoto Y (2018) Wikipedia2vec: an efficient toolkit for learning and visualizing the embeddings of words and entities from wikipedia. arXiv preprint. arXiv:1812.06280

Download references

Author information

Authors and affiliations.

Ochanomizu University, 2-1-1 Otsuka, Bunkyo-ku, Tokyo, 112-8610, Japan

Kaya Okada & Takayuki Itoh

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Kaya Okada .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Okada, K., Itoh, T. Scatterplot selection for dimensionality reduction in multidimensional data visualization. J Vis (2024). https://doi.org/10.1007/s12650-024-01025-6

Download citation

Received : 02 March 2024

Revised : 26 May 2024

Accepted : 30 July 2024

Published : 23 August 2024

DOI : https://doi.org/10.1007/s12650-024-01025-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Dimensionality reduction
• Multidimensional data visualization
• Scatterplot
• Scatterplot selection
• Evaluation of dimensionality reduction
• Text data visualization
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 23 August 2024

Tackling gender disparities in energy research: a diagnostic tool for equality in research centres

• Sara Sánchez-López 1 ,
• Rocío Poveda-Bautista 1 ,
• Carmen Corona-Sobrino 2 ,
• Paula Otero-Hermida 1 &
• Mónica García-Melón 1  

Energy, Sustainability and Society volume  14 , Article number:  51 ( 2024 ) Cite this article

24 Accesses

Metrics details

In a case study in Spain, the unequal proportion of men and women in a research organization in the energy sector is severe, and long-established dynamics that might determine differences in access to leadership positions and inequalities in research careers are evident. The gender gap in historically masculinized fields, such as energy engineering reflects more than simply the differences in male and female values and personalities. This study seeks to explore the gender gap in energy research centres and to identify barriers that potentially hinder the research careers of women. It proposes the development of a diagnostic tool, based upon indicators, to monitor and evaluate gender roles and inequalities in the management of research centres for identifying and addressing the dynamics and obstacles that hinder women's progress in the energy sector and their potential contribution to the field. This participatory multicriteria-based tool prioritizes the proposed indicators by their influence and importance in the context of energy research and applies it to the monitoring of a specific Spanish energy research centre.

The results are threefold: (i) the methodology is adaptable to different research centres; (ii) the analysis of indicators’ prioritization could lead to recommendations that should be addressed first; (iii) the diagnostic tool used in this in-depth case study of an energy research centre in Spain allowed results to be achieved in terms of gender dynamics. Two indicators stand out as the most relevant in our analysis: gender diversity in leadership positions and uncomplicated application of work–life balance measures. In this case study, the measurement of the first indicator has drawn unsatisfactory results, and the research of the latter is considered still insufficient. In conclusion, this difference becomes a vicious or negative circle for attracting and retaining more women to the research centre. Despite these results, no gender gap seems to be recognized and thus, no measures are being taken to improve the situation.

Conclusions

Comprehensive data and contextualized monitoring are necessary to effectively study and enhance the presence and participation of women in the energy science sector. This approach, combining quantitative and qualitative techniques, is suitable for any research centre that would like to monitor its gender gap, identify potential sources of inequity and address them.

Globally, the limited access to energy disproportionately affects women [ 1 ] and highlights the importance of considering gender in discussions about fair energy distribution and its role in development [ 2 , 3 , 4 , 5 ]. Despite this, the energy sector remains predominantly male-dominated. Not only do women face greater difficulties in accessing energy resources, but they also “continue to be an unrealized potential asset for the development of the energy sector” [ 6 ].

Women bring distinct economic and social capitals to the table [ 7 ], and lack of gender balance might not only be a matter of fairness and social justice, but could also be detrimental to innovative research. The involvement of women in the field of energy, and particularly in the context of sustainability [ 6 , 8 , 9 ], has been identified as pivotal for addressing emerging future advances, governance structures, and frameworks through which we might tackle the required issues, among others [ 10 ]. However, their participation remains limited not only to energy production, but also to the development of alternative consumption and production patterns [ 7 , 11 , 12 ]. This similarly occurs in the production of knowledge within the field where only 15.7 percent of energy scholarship authors have been women [ 13 ].

It is widely argued that one of the problems in an area like engineering or energy is the low number of women students. At a macro level, focusing on the case of Spain, where the study case is located, the proportion of women researchers in Spanish universities is 43.67% [ 14 ]. However, the main problem lies in the unequal distribution according to career progression, with only 25.6% of women reaching the highest category (full professor) and even more at the study phase. In the field of engineering, for instance, only 9% women are to be found in the highest category. In addition, only 52.4% of women have attained permanent positions in the system [ 14 ].

At the meso-level of the university institution examined, 31.98% of the staff are women [ 15 ], of which only 30% have attained permanent positions. In the area of engineering, this percentage drops to 21.39% of women researchers with permanent contracts [ 15 ]. Regarding the presence of women in public research centres in Spain, only 26.8% of women hold a permanent position working in the field of “natural resources”, which includes the energy sector [ 16 ]. This figure improves substantially in the initial categories with 57.7% of doctoral students being women [ 16 ]. Furthermore, in the case of the Spanish energy sector, only 22.2% of Spain’s scientific production on renewable energies in 2022 was led by women researchers [ 17 ].

While a significant body of literature and diverse approaches to addressing the scarcity of women in Science, Technology, Engineering and Mathematics (STEM) are available, a notable gap in the case of the energy sector is evident [ 10 , 13 , 18 ]. The presence and participation of women in the field of energy have been studied in boards and management groups of large energy companies [ 19 ], in decision-making processes in the renewable energy sector [ 20 ] or in energy policymaking [ 12 ]. However, to the best of our knowledge, gender roles and inequalities in the management of research energy centres aimed to address the gender gap remain unexplored in the scientific literature, and our study is the first that seeks to specifically address the monitoring of energy research centres.

Monitoring the gender gap in science: contextualization and indicators used at the organizational level

In Europe, gender gap monitoring in science, research and innovation is highly directed by European Union (EU) approaches. Specifically, the Strategic Vision of the European Research Area has set as a goal for 2030 that half of all scientific personnel, in all disciplines and at all levels of the scientific system, should be women. The aim is to break the horizontal and vertical segregation that currently exists in European science, especially in historically masculinized fields like energy [ 21 ]. Thus, EU members such as Spain have advanced legislation in this area to achieve the Strategic Vision of the European Research Area.

The indicators used to monitor policies in European reports, such as She Figures and their counterpart reports in Spain, tend to focus primarily on providing numbers of men and women. Therefore, despite its relevance, they have scarce information on how gender dynamics work in scientific and innovative working environments, where personnel perform their functions and interact on a daily basis [ 22 , 23 , 24 ].

The meso-organizational level is key in the “quality of equality” which means that inclusion is not merely having women but where—in which areas, in which roles—and how are they included [ 25 ] is essential; and where—without this information, it is not possible to understand why, for instance, many women leave engineering careers or whether women or other underrepresented collectives in the discipline have a similar wellbeing status.

The organizational level includes crucial issues for equal access and quality such as staff awareness of equality measures, the distribution of tasks and responsibilities, management of projects, recognition, work culture, work–life balance culture, and use of time or personnel selection. At this level, there is a concentration of conditioning factors to accumulate merits in a markedly meritocratic science system. In this context, which is depicted as neutral and universal within the meritocratic system, it becomes clear that the system is vulnerable to gender dynamics which apparently affects objective processes such as hiring and promotion [ 26 ] or the definition of academic excellence [ 27 ].

The organizational level is crucial in science development and the lack of indicators may result from intrinsic difficulties in gender monitoring. Monitoring is usually a synonym for quantitative approaches which often tends to focus on public information like how many of each sex are to be found. On the other hand, gender dynamics are difficult to capture without perception and other qualitative indicators [ 28 ] which are more difficult to operationalize, and which often require the preparation of surveys or choosing other methods like organizing focus groups to gather primary data. However, the effort can be worthwhile considering that monitoring is not unambiguous in its use and that indicators are not equally relevant. Monitoring can be applied to control policy development, distribute funding, compare organizations, or check the advance of some implemented measures, for instance. Those purposes and the ultimate justification, such as social justice or achieving efficiency, determine institutional logics that affected the final use of monitoring and resulted in different indicator panels which also reflects a different understanding of the issue that is monitored [ 23 ].

Measuring gender at the organizational level pursues some primary goals: diagnosis and learning. However, indicators receive their significance from institutional practices [ 29 ]. Thus, contextualized monitoring through gathering secondary and primary data, both quantitative and qualitative, is crucial and it becomes essential to go beyond “counting heads” [ 30 ] to understand not only the number of women present in energy research centres, but also the dynamics that hinder the development of women’s careers in these areas, i.e. the distribution of tasks, management, projects, and recognition. In Europe, gender gap indicators are focused on policy monitoring at the national level, while the organizational level still needs to be developed, which is another challenge to be addressed [ 23 ].

Contextualization or context-sensitive monitoring implies a better understanding of different levels as the centre and the research system provides in regards to the conditions of understanding research excellence, access to positions or research funding. This requires expert integration in the monitoring process, to interpret the relevance and cross-influence of the indicators, as is given in more detail in the methods part. Thus, it is crucial to include the context where gendered energy research takes place and to provide a systemic contextualization [ 12 ].

We developed a tool based upon performance indicators to monitor and evaluate gender roles and inequalities in research centres. The tool provides feedback to the literature review and quantitative and qualitative inputs at the organizational level which is a sensible step within the overall gender and science context, with a focus on Spain in this case. This perspective also facilitates the integration of the inherent complexity of measuring relational dynamics in organizations, which contributes to the gender gap [ 31 ]. The gender gap should be understood as a multi-dimensional concept: people involved, relational dynamics [ 31 ], and organizational culture. Therefore, it should be treated as a multi-criteria problem and studied using multi-criteria decision-making methods (MCDM). These methods are highly appreciated for developing monitoring tools [ 22 ], for example, the work of [ 32 ] where a multi-criteria decision model is used to measure sustainable energy development efficiency [ 22 ]. See [ 33 ] for more information on MCDM.

We propose a methodology for an in-depth study of research centres investigating energy-related issues. This methodology can be adapted to develop tools to monitor and diagnose different research centres and their specific contexts.

Our proposal will make three contributions to the energy research field: first, by presenting the possible indicators at the organizational level in research centres and a methodology to prioritize them according to the centre’s needs; second, by monitoring and presenting results of a specific research centre in the energy field; and third, by including recommendations to address the gender dynamics contributing to indicate gender gaps within the monitored centre.

The rest of the paper is organized as follows: “ Methodology ” section presents the research methodology and methods employed. “ Results ” section shows the results and “ Discussion ” section discusses the implications and is divided into the case of study recommendations and general contributions to the energy field. Finally, “ Conclusions ” section summarizes the conclusions of the research.

Methodology

The proposed methodological approach of this research is presented in Fig.  1 . It is developed through two main stages: the design of the general methodology and the application to a specific energy research centre.

Methodology diagram. The stages of the methodology correspond to the sections where these tasks are described in the paper

Our diagnostic tool is developed in two stages. The first is generic and useful for any public research organization. The second is specific to a particular research centre in the energy field. In other words, we obtain generic indicators that can be used to measure any organization and we adapt them to the context and then use them to monitor and diagnose a particular Spanish Energy Research Centre.

The goal of the first is to identify all the relevant perspectives and dimensions related to the gender gap and to determine a specific list of performance indicators to monitor and evaluate gender roles and inequalities in research centres. This general methodology employs an integrated MCDM-based approach using a combination of Decision-Making Trial and Evaluation Laboratory (DEMATEL) and Analytic Network Process (ANP) to determine the most influential criteria for the gender gap in research centres. The combination of these two methodologies (DANP) is novel in the context of gender policies and integrates the benefits of both methods. ANP [ 34 ] allows us a full analysis of the influence of all the factors that make up a network (see [ 34 ] for more information on ANP). In addition, DEMATEL [ 35 ] permits a cause and effect analysis of the various network elements involved [ 36 , 37 ].

The aim of the second is to monitor the performance of a selected Spanish Energy Research Centre (ERC). All the indicators will be measured and analysed according to the results of importance and influence obtained for each.

This stage of the methodology has involved the thorough documentation of the institution itself (bibliometric indicators and other registered numerical indicators), as well as the document analysis of relevant scientific legislation. The analysis of hiring and promotion processes, among other variables, has entailed an in-depth examination of the laws and regulations governing these processes in Spain. A comprehensive study of documentation on Spanish state-level legislation and university organization was conducted to understand the specificities of the case study in the scientific context. A document analysis was carried out to analyse and perform an in-depth study of the chosen case study. Likewise, the content analysis of scientific Spanish legislation [ 38 ] and scientific reports [ 16 , 28 ] have also served methodologically when designing the script of the in-depth interviews. For methodological reasons, a binary gender system has been assumed in the construction of indicators and in the analysis of results. Nevertheless, in the sociodemographic questions of the questionnaire, the possibility of including other gender identities has been provided. This assumption of the binary system aims to facilitate both the research and the data interpretation. However, the authors are aware of the limitations posed by this binary approach and acknowledge the diverse expressions and identities that may manifest within such contexts.

Finally, some recommendations and guidelines will be addressed to the management team of the research centre in order to target their gender gap.

General methodology

Selection of indicators and relevant aspects.

Focusing on the selection of indicators, [ 22 ] proposed a list for Research and Innovation purposes from a Responsible Perspective for the Spanish context subsequent to an in-depth review of the existing indicators provided by relevant reports, such as the Responsible Research and Innovation (RRI) European Expert Group , the collection of She Figures EU reports, or the Spanish version of She Figures, Científicas en Cifras . Based on [ 22 ] study and previous knowledge, we developed a general, extensive list of indicators that should be considered in the analysis of the gender gap in research centres (Table  1 ). This preliminary list of indicators is organized into 6 main groups, which are the most common when analysing the gender gap in research centres.

Prioritization of indicators by experts

The DANP technique is used to evaluate the gender gap criteria (indicators), allowing us to rank these criteria in relation to the objective: evaluate gender roles and inequalities in research centres.

Selection of experts

Our methodology considers the inclusion of energy stakeholders, not only gender experts [ 39 ], in an interdisciplinary approach that combines experts from social sciences, politics, and engineering for methodological development. This interdisciplinary approach aligns with the underlying debate about the topic in energy studies [ 40 ].

The research team includes various types of experts, encompassing those dedicated to gender issues within the Spanish scientific system and engineers specializing in energy. The MCDM technique we use relies heavily on the quality, rather than the quantity, of these experts due to its semi-quantitative and expert-oriented nature [ 41 ]. It is vital that the experts display both a comprehensive understanding of the implications of their fields within the context of our case study and a holistic perspective on research centre activities. In order to ensure a comprehensive assessment, our DANP model requires multiple experts on each panel for cross-verification purposes. Our panel, comprised five individuals—one political scientist, one sociologist, and three engineers—was initially chosen to define the DANP model and validate context-based indicators suitable for any research centre. Recognizing the need to integrate the specific dynamics of the energy engineering sector, we expanded our expert panel by including two additional individuals with in-depth knowledge of the field of energy research. With a total of 7 experts, this panel evaluated context-based indicators, weighting them based on their impact on gender equity.

A detailed description of the cross-experience of our 7 experts is shown in Table  2

Model definition

The ranking model is built upon a network of criteria that have mutual influence. These criteria are derived from a literature review and the context is validated by the panel of 5 experts through a first participatory session.

The relevance of the indicators is heavily affected by the different contexts. The panel evaluates the criteria (indicators) integrating the context. Expert prioritization pinpoints the causal relations and diverse shapes of a specific problem that refuses to be measured.

The objective will be to identify which are important and which are crucial in the specific setting of the energy research centre within both the university and the Spanish research system, considering a panoramic view of the gathered data. This will contribute significantly to an interpretation of the results and the formation of recommendations. The aim is to tailor the general gender gap indicators for research centres (preliminary list) into indicators suitable for monitoring Spanish energy research centres (list of context-based indicators).

For this purpose, we designed a questionnaire that was used to ask the experts individually to elicit their judgements.

Application to a specific Energy Research Centre

Weighting and interactions of indicators.

After constructing the model and receiving validation from the experts, the DANP method is applied in five steps:

Step 1: Generation of Direct-Relation Matrix A. Firstly, measuring the relationship between criteria requires the design of a comparison scale on a 0–4 scale:

0 (no influence)

1 (low influence)

2 (medium influence)

3 (high influence)

4 (very high influence)

Next, experts make pairwise comparisons of the influences between the criteria. Then, the initial data are obtained as the direct-relation matrix. Matrix A is an nxn matrix in which aij denotes the degree to which criterion i affects criterion j .

Step 2: Normalizing the direct-relation matrix. On the basis of direct-relation matrix A , the normalized direct-relation matrix X can be obtained through equations:

where a ij is the values of the direct-relation matrix.

Step 3: Obtaining the total-relation matrix: T can be obtained by using (Eq.  5 ), in which the I is denoted as the identity matrix:

Step 4: Obtaining the causal diagram of the criteria. Parameters D and Rare obtained for each criterion from matrix T using the following equations:

The cause–effect diagram permits the analysis of the degree of prominence, given by the sum of D and R (horizontal axis), and the degree of cause or effect, given by the subtraction of D and R (vertical axis).

Step 5: Normalizing each column of the T matrix (unweighted) by its sum, we obtain the weighted supermatrix:

where w ij is the values of the weighted supermatrix and t ij is the values of the total-relation matrix.

Step 6: Calculating the limit matrix. In this step, the weighted matrix is multiplied by itself until all its columns become equal, i.e. the values converge, and the process ends. This way, each element’s individual influences on the network’s other elements are obtained from this limit supermatrix.

The criteria values are extracted from the vector of the limit supermatrix and normalized by the sum to obtain their final weights. In this way, we can obtain the criteria ranking, which will allow us to understand the decision profile of the experts.

After receiving individual assessment results of DANP, each expert validates her/his own results. If the results are unsatisfactory, she/he revises the results of the pairwise comparisons to ensure that the results are coherent with her/his knowledge and overall assessment.

Monitoring of the research centre

Online survey: primary data.

In addition to the personal interviews, an online survey was designed and circulated to reach as many people as possible within the organization. The study aimed to obtain qualitative and quantitative information to study the gender gap in Energy Research Centres and to identify barriers that potentially hinder the careers of women. Data were gathered through single and multiple choice, and open-ended questions by using Qualtrics software, Version July 2023. Copyright© 2023 Qualtrics. Gender and year of birth were requested for data analysis purposes but no information that would potentially identify the respondent was collected. The survey consisted of 20 questions addressing the level of seniority achieved, the perception of the formal and informal atmosphere of the organization, awareness of the equity plan, use of work–life balance measures, and experience in leading projects.

The survey was sent to the head of the research centre to be internally disseminated by email to all the workers of the institution. Our population is all the researchers who were registered as members of the ERC in May 2023, i.e. 48 people. A total of 36 people answered and, according to their distribution by professional category and gender, it was concluded that the sample was sufficiently representative. The only a priori bias detected was motivation, i.e. the proportion of women in relation to the total number of employees answering the questionnaire is higher than that of men. However, both gender groups are sufficiently represented.

The content validity of the survey was initially tested on a sample of 10 people from diverse academic backgrounds residing in Spain. The survey was adjusted by integrating the feedback received. The data from the pilot are not included in the results.

Databases: secondary data

The institutional database and the university’s website were consulted for information on the position, professional category, and academic merits (patents, scientific production, projects and other outcomes) of all the members of the research centre.

In-depth interviews: primary data

Twelve in-depth interviews (30–60 min long) were conducted with women and men in permanent and non-permanent positions at the centre (12 in total). The interviewees were selected based on a strategic selection of participants to ensure representativeness and to provide diverse and information-rich perspectives on the research topic. The distribution of interviewees corresponds to the structure of the energy engineering field itself, a highly masculine area as seen above.

The aim of the interviews was to obtain information on the perceptions, opinions, and experiences of the centre’s staff as regards gender issues from a representative number of individuals of differing professional categories and genders. Following the logic of the dimensions of the indicators, the interview guide was structured into these four blocks (see “ Qualitative information ” section). The information extracted was transcribed and analysed according to the qualitative content method [ 42 ]. The N-VIVO software was used as a tool to support the analysis.

We have divided the analysis of the results into two parts. In the first part, we show the results obtained for the indicators. These results are generic, i.e. they could be useful for monitoring the gender gap in any research centre in the Spanish research system.

Secondly, we present the results of the weighting of the context-based indicators and the monitoring of an energy technology research centre. The results have been obtained by measuring the performance of a particular centre for each of the proposed indicators.

Indicators to monitor the gender gap in research centres in Spain

Model description (energy research gender indicators).

Once the preliminary list of indicators was obtained (see “ Selection of indicators and relevant aspects ” section), and based on the context of the research centres, the main indicators were selected for the purposes of monitoring these centres. The final list of indicators, which was drawn up through a second participatory session of scientists with expertise in gender and energy, is shown in Table  3 . These selected indicators will be the DANP elements of the network.

This second expert prioritization phase took place during a comprehensive face-to-face session. The experts were convened for a half-day session. The initial session started with the validation of the indicators, which were thoroughly examined and deliberated over to ensure unanimous agreement on the list. Once the indicators were validated, the facilitators (some authors of this paper) elucidated the DANP principles, enhancing the experts’ comprehension and facilitating the clarity of the subsequent surveys. After that, each expert addressed his/her surveys individually under the guidance of facilitators. The results of each survey were immediately processed and presented to each of the experts for review. Subsequently, the facilitators aggregated all the individual results using the geometric mean, the consensus judgement according to [ 41 ], so as to obtain the group responses.

The criteria are clustered into four categories as shown in Fig.  2 : research management and results, staff configuration and structure, work culture, and gender contents in research.

Overview of the proposed model

Results obtained for the weights and interactions of the indicators

The context-based indicators already defined must be weighted, obtaining the Energy Research Gender Indicators (ERGIs). For this, we use the DEMATEL technique.

The expanded panel of experts (see Table  2 ) will then be asked individually to elicit their judgements. To this end, we designed a questionnaire in which they will be asked to rate the intensity of the influence between each pair of criteria from 0 to 4, in which 0 is no influence and 4 is maximum influence. An example of this questionnaire is shown in Fig.  3 .

Questionnaire used to weight the influence of context-based indicators

The DANP method prioritizes the selected indicators from the most to the least important for the evaluation of gender issues in the ERC, according to the participating experts.

The final prioritization of indicators for the aggregated group of experts obtained with the DANP technique is shown in Table  4 and Fig.  4 .

Prioritization of the indicators

In Fig.  4 , three indicators stand out slightly from the rest. The first is C24: Gender diversity in organizational leadership positions; the second C34: Existence of an equality plan, and the third C11: Gender diversity in research leadership. Two of them are related to the leadership of women.

This graph also shows that indicators related to the organization’s own structure, staff configuration and work culture, are more important than indicators related to research outputs when measuring the gender gap in the institution. The use of the DEMATEL technique also allows us to obtain very detailed and relevant information regarding the influences exerted by the indicators on each other. In Table  5 , we present the matrix of influences obtained by the set of experts. In this matrix, each cell represents the influence that the indicator in the row exerts on the indicator in the column.

The total relationship matrix presented in Table  5 shows the results in three different levels according to the two obtained thresholds for relevance [ 21 ]:

Threshold 1. Moderate influence: mean + 1 standard deviation (0.209)

Threshold 2. High influence: mean + 2 standard deviation (0.292)

Grey values are below threshold 1, black values are above threshold 1, and bold values are above threshold 2.

In addition, in the matrix we present the results of the calculations of factors D and R for each indicator (see Eqs.  1 – 5 ). Recall that factor D indicates the level of influence exerted by an indicator and factor R represents the level of influence that the indicator receives. In this second level of analysis, we can see that the indicators with the greatest influence are C34 and C24, which coincide with the two indicators that Fig.  3 shows as being the most important. We also observe that the two most influential indicators are C34 and C11. In other words, the indicators that stand out for their influence on the network coincide with the most important.

We would like to represent this information in a cause-effect diagram; in Fig.  5 we present the X -axis, which shows the degree of importance of each indicator ( R  +  R ) and the Y -axis, which shows the degree of cause (positive values) or effect (negative values) of each indicator ( D – R ).

Cause–effect diagram of the ERGIs

As can be seen in this diagram, the indicators are classified into four quadrants [ 43 ]. We may observe that the indicators that appear in quadrant II are: C24, gender diversity in organizational leadership positions and C31, ease-of-use of work–life balance measures. These can be considered key factors and should be taken into account when designing gender actions. Indicator C31 has not appeared until now as it belongs neither to the most influential nor to the most important group. However, the combination of both properties places it in quadrant II, which makes it a relevant factor when measuring the gender gap in research centres. Our interpretation of these two key factors is that the role of senior researchers is crucial because the fact that there are women in the relevant positions serves as a mirror in which they look for the other women working in the same research centre. In addition, the dynamics can be different when there are women in leadership since it makes clear to the staff in their charge what position they might occupy in the hierarchy of the organization. This result aligned with those from previous studies [ 30 ]. In our case study, the effect of the low number of women and the fact that part of the staff is not accustomed to mixed-gendered interactions have been clearly stated during the interviews. We find that women with a clear vision of equality problems in the centre have probably created a safe environment, bearing in mind that the younger women under their command have not perceived the problems they had faced. However, that affects only a few of the research groups of the centre, groups in which there is a high concentration of female members by the way.

On the other hand, the availability and ease of use of work–life balance measures are considered highly relevant factors that could make the difference when attempting to attract more women to a field as masculine as energy engineering.

Additional conclusions that can be drawn from Fig.  5 are as follows:

These indicators are isolated: C12, mobility actions carried out ; C33, existence of regulations on conduct in cases of workplace harassment and C14, participation of women in knowledge transfer . This means that they are less influential on others. For instance, according to the national authorities, mobility or knowledge transfer are relevant requirements for career advancement but have limited impact on other gender aspects, as is the case of sexual harassment regulations, which are highly relevant but not so closely related to others.

C41, gender contents in research appears to be of low influence and not of great importance. Not all research leaves room for gender perspective integration as is the case of some research in the field of energy.

Finally, we would like to re-emphasize those indicators that appear to be very influential but not very important: C23, transparency in selection processes ; C35, specific training in gender issues . These would be indicators that have a strong influence on others, but which would not be so important on their own, i.e. without considering their relationships with the rest. It is necessary to consider them whenever their influence is exerted on important indicators. For instance, transparency in selection processes affects the distribution of staff both vertically and horizontally, whereas the fact that staff are trained in gender issues will make them more aware of these issues and more critical of inequalities.

Results of the monitoring of a research centre in energy engineering

Qualitative information.

The three thematic blocks of the information obtained from the personal interviews correspond to the dimensions developed in the Energy Research Gender Indicators (ERGIs): (1) research management and results; (2) staff configuration and structure; (3) working culture; and (4) gender contents in research.

Firstly, as regards research management, a large proportion of the men interviewed stated that the organization, participation, and leadership of research in the centre only respond to meritocratic and hierarchical issues and that gender has no influence whatsoever. A researcher stated “the truth is that we were surprised by your interview because here we do not… here what matters is what matters. (…) women, men, and everything. And the truth is that I believe that no discrimination has ever been made” (I1). However, the women interviewed expressed a more critical attitude towards the management of gender diversity in the centre.

Secondly, the unequal proportion of men and women in the organization is one of the issues most frequently mentioned. It is stated that this is a structural problem of the discipline, which manifests itself as early as undergraduate studies, on which very few women are enrolled, and which is reproduced on the subsequent levels of the scientific career and, therefore, at the centre. Furthermore, they state that in comparison, “there are many women at the entry level, but very few go all the way [to a research career]” (I3). There are no proactive measures to try to reverse the structural inequality in the centre by taking positive discrimination measures in selection processes or by making specific calls to attract more women. The permanent researchers interviewed claim to select members of their group based on their knowledge and specialization, regardless of gender. Furthermore, as will be seen in the quantitative results (“ Quantitative information ” section), there is a clear gender gap in positions of responsibility, both vertically and horizontally.

Thirdly, as regards the work culture, working hours are flexible, which is seen as positive for both the family and private life of men and women. However, these measures are still insufficient, as one of the interviewees stated: “private life still affects women more in terms of career development (…) motherhood (…) leaves you behind” (I7).

It is also important to highlight the participation dynamics and the atmosphere in the centre. The fact that there is a reduced number of women is influential, insofar as the women feel less involved because they are in a smaller proportion. And “I do have to say that it is still noticeable that men are not used to it. There are very, very few …that will surely affect that men are accustomed to dealing with men” (I7).

Finally, the incorporation of the gender perspective in research, which is seen, in most cases, as something alien to the nature of the work due to its object of study (e.g., fundamental science). In these studies, in which this perspective would have a place, it is considered that “[considering] research questions or the object of our work is more difficult for us because we do not have the skills. What we know how to do, we do not know how it can contribute” (I10).

On the other hand, although most of the interviewees know of the existence of an equality plan and sexual harassment protocols, it is only their existence that is acknowledged, not their content or implications in the centre.

In short, most of the male interviewees’ discourse centred around the fact that there is no gender-related problem at the institution, either in the management of research or in the working environment. However, this is not the case for some of the women interviewed who do allude to different problems of gender discrimination, such as “the distribution of tasks and roles in the centre” (I2).

Quantitative information

Primary data results: survey.

Researchers who indicated they had children (60%) were asked whether they had taken maternity/paternity leave. 100% of the women with children reported taking maternity leave, whereas less than 50% of the men did. This is especially relevant since maternity leave was regarded by some of the researchers as a critical point in the development of a woman’s scientific career, leading to them losing advantage compared to their male counterparts. If men took paternity leave as well, the difference would not be so blatant, and the impact would be smaller.

The large majority of respondents (75%) reported not having received any gender equality training (see Fig.  6 ). The primary source of training for those who did was the university.

Gender equality training received

Secondary data results: website of the centre and official university database

The research outcomes of all ERC staff were thoroughly reviewed and analysed. This examination included several variables, such as the quantity of published papers, papers published as first authors, research projects as PI, contracts obtained, number of contracts in which they participate, and patents. The analysis was focused on the data from 2019–2023, which were later on segregated by gender. The aim was to identify key factors contributing to scientific career advancement.

Table 6 shows the members of the monitored centre classified according to professional categories following the Spanish university system. Notably, there are no women in the highest category.

Figures  7 and 8 compare gender distribution in research contracts between private companies and publicly funded R&D projects. A notable difference emerges in leadership roles: senior men predominantly lead contracts with private companies, while senior women tend to lead publicly funded R&D projects. As regards the higher number of women leading public projects (Fig.  7 ), there could be several reasons for this, such as: (i) networking, (ii) the policy of promoting gender equality and (iii) the differences between the motivations and values of men and women in terms of the impact of their results.

Principal investigators of publicly funded R&D projects attending to professional tenure and gender

Principal investigators of private contracts attending to professional tenure and gender

Private contracts lack regulations for PIs, whereas for publicly funded projects, there is a policy promoting gender equality in science that positively evaluates projects led by women. On the other hand, and as far as the higher number of men leading private contracts is concerned (Fig.  8 ), that could be a question of ease-of-access to advertising venues since, as the information is not open, are obtained through contacts. Again, the dynamics of a male-connected engineering environment may be observed.

Monitoring of the Research Centre: gathering of data indicators

The results obtained for ERGIs in the monitoring of a Spanish Energy Research Centre as well as the sources from which these results have been collected are shown in Table  7 .

As regards the results of the DANP model and the ERGI values obtained for the ERC, we propose some recommendations focusing on key factors that should be considered when designing gender actions in this centre. These key factors will be those indicators that are important or influential, (or those that combine both properties), and whose ERGI values are low or qualitatively deficient for the centre. We propose recommendations for those indicators marked with * in Table  7 .

For recommendation purposes, from the total list of 14 indicators we choose those that add up to 50% of the total weight. This prioritization allows us to focus on the most important factors that are likely to have the greatest impact on the outcome and prevents the inclusion of too many recommendations that could lead to excessive complexity and potentially obscure the improvement of the process. Therefore, we will more thoroughly analyse the indicators that add up to 50% of the weighting process according to Fig.  1 : C24, C34, C11, C21, and C35.

Case study: key indicators and recommendations

As far as the results of the DANP model and the ERGI values obtained for the analysed ERC are concerned, we propose recommendations in the following indicators (marked in Table  7 with *):

C24. Gender diversity in organizational leadership positions

C34. Awareness of the existence of an equality plan

C11. Gender diversity in research leadership—% Women principal investigators of projects

C21. Vertical segregation

C35. Gender-specific training

C31. Ease-of-use of work–life balance measures

The lack of women in organizational leadership positions is pronounced in the case of the study as no women are present in head positions. This indicator (C24) affects multiple dimensions. The insufficient or, in this case, inexistent female representation in high-level positions might dissuade women from joining an organization in which they do not foresee career development. The presence of women in management might be perceived as making panels more approachable or more receptive to the acknowledgement and tackling of gender issues than those that are exclusively male. Additionally, the absence of women in high positions can potentially disincentivize them from entering an institution where gender dynamics might pose a challenge. These arguments also apply to indicators C21: Vertical segregation and C11: Gender diversity in research leadership . In the case of indicator C11, it is worth highlighting that being the principal investigator of projects is a requirement for any advancement in academic research in the energy sector in Spain. The low proportion of female principal investigators in projects could explain the absence of women in leadership positions. The difficulty they experience in advancing their scientific careers, and consequently accessing management positions, may be a contributing factor.

There is a prevailing belief within the research centre that only meritocracy drives success, dismissing other factors—especially gender—as irrelevant. Despite the evidence, including the low representation of female leaders in the scientific output of the Spanish energy sector [ 17 ], many members fail to recognize existing differences or inequalities. Consequently, these indicators could serve as a pivotal tool to raise awareness and challenge expected resistance towards equality measures within specific groups.

Specific training on gender (C35) would address two different aspects identified in this case study. Firstly, it would provide training to those people who recognize that there is a gender gap and are willing to work towards reducing it but do not have the tools or knowledge to address it, either as a power figure or as part of the group. And secondly, it would increase the awareness of those who consider that no gender gap exists, nor that changes should be made to accommodate a more inclusive view. Thus, given its importance, gender training should not be the sole responsibility of the University's Equality Committee, but the research centre should also oversee the proposal and development of activities that promote training in equality to make this as cross-cutting and diverse as possible.

Furthermore, there is a significant lack of awareness regarding the content of the equality plan (C34). The actions taken to circulate this content and make it visible to the staff should be promoted by the management. However, this would require an initial acknowledgment of the gender gap by the heads of the institution.

The availability of work–life balance measures (C31) is identified as a crucial indicator for the career development of women in research. Yet, availability is not enough. The acceptance of such measures by the work environment as well as whether men make use of them are important factors that weigh in women’s career development. If men in the institution are making equal use of the measures, women’s careers will be less negatively impacted from a competitive point of view when they use them. Besides, it is also a sign that the organizational culture is more sensitive, and that care is not considered a women-only issue.

Furthermore, the ease with which women can make use of the work–life balance measures is considered a highly relevant factor, which leads us to believe that centres not placing obstacles in their way—considering the reasons why they use them—would be an attractive factor for women. Particularly in a field as masculine as energy engineering.

While work–life balance measures are present they are often informal, and the absence of official guidelines gives line managers the discretion to determine the extent, duration, and timing of these measures. Therefore, a change of manager might entail a change of conditions or some uncertainty towards what their rights will be, as some men pointed out in the interviews.

The Gender Perspective in Research Content (C41) was not identified as a relevant indicator, possibly due to the nature of the research. Indeed, for some of the research conducted, gender perspective cannot be applied; for example, in the study of the disposition of photovoltaic cells. However, it was detected that this possibility had often not been considered. Therefore, future research should assess whether its impact may be different for women and whether it is possible to integrate a gender perspective into both the samples and data collection. Several studies have shown that men and women may have different energy consumption patterns due to varying daily routines, responsibilities, and access to resources [ 44 , 45 , 46 , 47 ]. Taking gender differences into consideration can provide a more comprehensive understanding of the energy needs and these consumption patterns. This is particularly relevant, since women are the primary users of household energy in both developing and industrialized countries [ 8 ]. It could also contribute to the fostering of a broader and fairer approach in policy and technology development; for instance, adapting renewable energy projects to address specific needs of women in rural areas [ 18 , 48 ], or simply having enough understanding not to create inequalities or perpetuate those already existing.

Finally, a recommendation is addressed to the home institution of the research centre. The centre has no expertise in gender equality, while the equality plans are located at university level. We have identified that some gendered distortions that exist throughout the whole Spanish research system- such as gendered precariousness and the impact of family responsibilities- are not mitigated by centre measures and university measures are equally absent and unrecognized. Considering the effort to be made, the research centre cannot do it alone; for its development, it should have the support of the equality unit of the institution to which it belongs. The centre could greatly benefit from developing a tailored equality plan that considers its unique circumstances as regards the gender gap. While rooted in the general measures of the institution's plan, the centre’s plan should incorporate specific aspects that reflect its nuances. This entails integrating measures specifically designed to address and bridge the gender gap within the research centre.

General discussion

To effectively study and enhance the presence and participation of women in the field of energy science, comprehensive data are imperative. Disaggregated data, at least separated into area of knowledge, category, and gender, is vital for both informed decision-making and understanding the reality within research centres, enabling the necessary steps to be taken.

The case study research method carried out aims to provide insights into the workings of a particular process within its context, enabling us to observe the dynamics of the agents and infer explanations. However, it is important to note that this approach is not representative and can only be compared once more cases have been developed [ 49 ]. Expanding the research to monitor additional energy research centres would enhance the model and offer deeper insights into the unique dynamics and challenges within this domain. While each centre may present distinct dynamics, this adaptable model is designed to accommodate and integrate these differences. Therefore, this tool can serve both diagnostic and awareness-raising purposes—a conversation starter rooted in data, demonstrating the persistent existence of the gender gap demanding attention and resolution. The presented indicators possess the potential to serve as diagnostic tools for understanding the gender gap within research institutions and for raising awareness. This is particularly crucial in fields such as energy, where according to the cited literature, the gender gap is often overlooked, as we found out in our case study.

We are currently monitoring other research centres using the same general methodology and replicating the specific methodological part of the case study in two additional scientific sectors: applied technologies and biology sciences. The findings in these two case studies reveal similar dynamics in the underrepresentation of women, especially in leadership roles in research and management positions. What may also be observed is how little aware these organizations are of the existing gender gap.

The development of an indicator tool based on DANP not only serves as a proactive approach to the monitoring and evaluation of gender roles and inequalities in research centres but also contributes to interpreting results and forming recommendations. Applied in a case study, this tool is specifically tailored to incorporate findings from literature reviews and both qualitative and quantitative organizational inputs, considering the broader energy, science, and gender context in Spain.

The tool’s design considers the complexity of measuring relational dynamics within organizations, recognizing that these dynamics significantly contribute to the gender gap. This perspective helps integrate various dimensions, including the individuals involved, relational dynamics, and organizational culture. The gender gap is portrayed as a multi-dimensional problem, extending beyond mere numerical representation. Using a multi-criteria decision-making method, we assess the impact of the indicators on gender equality in order to address the gender gap in a specific research centre. This method entails the selection and grouping of decision criteria, followed by the analysis of interactions within the network model defined, considering the opinion of energy stakeholders, not only gender experts.

Our context-sensitive methodology reveals specific dynamics. For instance, in the analysed centre, a sexist environment emerges through informal comments, such as jokes; men often underuse available life–work balance measures, and there is a gendered pattern in fund access: women primarily lead publicly funded projects, limiting their diversification due to a more limited access to private funds.

The use of indicators serves as powerful diagnostic tool and catalyst for awareness. They highlight the persistent gender gap, particularly in traditionally male-dominated fields, such as energy engineering, where this gap often goes unrecognized and acts as a catalyst for conversations about the changes required.

It is advisable to expand the research so as to monitor more centres for the purposes of refining the model and better understanding the nuances of the gender gap. An ongoing monitoring would help to identify existing disparities and instigate actions toward gender equality. For this reason, the research would benefit from the monitoring of more energy research centres to further adapt the model and better understand the particularities that this area might include. Our findings, from an in-depth case study, can be discussed in the development of further research avenues on gender and energy. However, each organization is unique and the methodology we propose is designed to fit the specific dynamics of each centre.

Limitations and future research lines

We acknowledge and emphasize the intrinsic limitation of our in-depth case study methodology and encourage further research that can provide additional insights and patterns into the gender dynamics in energy research centres. Our purpose is to follow up with other energy engineering institutes in Spain, as well as to establish comparisons with other geographical contexts, such as with similar studies conducted in Germany [ 50 ]. This could provide a broader view and strengthen the applicability of the proposed tool in various contexts. We are currently monitoring several research institutes in differing areas of knowledge, also in Spain, with results that are very similar to those presented in this analysis. We intend to continue this study by applying the proposed methodology and carrying out a comparative study of these institutions.

Moreover, we also want to highlight some other limitations regarding data gathering. We were unable to employ any strategies to mitigate non-response bias in the collection of primary data, such as follow-up contacts or incentives for participation due to confidentiality reasons of the monitored centre. This could have compromised the representativeness of the quantitative results through the survey.

Finally, as regards the interpretation of some of the qualitative results, we know that there is an interpretation bias in the results on paternity leave due to the fact that the length of this leave in Spain has varied greatly over the last 20 years from 4 days to 6 months. Since age was not asked in order to anonymize the responses to the questionnaire, it is not possible to relate the length of leave to whether it was taken or not. Further research could have an impact on this issue, since no data has been available until recently.

Availability of data and materials

The datasets generated and analysed during the current study are not publicly available due to the need to protect the privacy of study participants but are available from the corresponding author on reasonable request.

Abbreviations

Energy Research Centre

Analytic Network Process

Combination of DEMATEL and ANP (DANP)

Decision-Making Trial and Evaluation Laboratory

Energy Research Gender Indicators (ERGIs)

European Union

Multi-criteria decision methods

Gayoso Heredia M, Sánchez-Guevara Sánchez C, Núñez Peiró M, Sanz Fernández A, López-Bueno JA, Muñoz Gómez G (2022) Mainstreaming a gender perspective into the study of energy poverty in the city of Madrid. Energy Sustain Dev 70:290–300. https://doi.org/10.1016/j.esd.2022.08.007

Article   Google Scholar  

Hanke F, Guyet R (2023) The struggle of energy communities to enhance energy justice: insights from 113 German cases. Energy Sustain Soc 13:1–16. https://doi.org/10.1186/s13705-023-00388-2

Feenstra M, Özerol G (2021) Energy justice as a search light for gender-energy nexus: Towards a conceptual framework. Renew Sustain Energy Rev 138:1–11. https://doi.org/10.1016/j.rser.2020.110668

Farhar BC, Osnes B, Lowry EA (2014) Energy and gender. In: Press OU (ed) Energy poverty: global challenges and local solutions. Oxford University Press, Oxford, pp 152–179

Chapter   Google Scholar  

Shailaja R (2000) Women, energy and sustainable development. Energy Sustain Dev 4:45–64. https://doi.org/10.1016/S0973-0826(08)60231-8

Oparaocha S, Dutta S (2011) Gender and energy for sustainable development. Curr Opin Environ Sustain 3:265–271. https://doi.org/10.1016/j.cosust.2011.07.003

Standal K, Talevi M, Westskog H (2020) Engaging men and women in energy production in Norway and the United Kingdom: the significance of social practices and gender relations. Energy Res Soc Sci 60:101338. https://doi.org/10.1016/j.erss.2019.101338

Cecelski E (2000) The role of women in sustainable energy development. National Renewable Energy Lab. (NREL), Golden

Book   Google Scholar  

Imbulana Arachchi J, Managi S (2021) Preferences for energy sustainability: different effects of gender on knowledge and importance. Renew Sustain Energy Rev 141:110767. https://doi.org/10.1016/j.rser.2021.110767

Cannon CEB, Chu EK (2021) Gender, sexuality, and feminist critiques in energy research: a review and call for transversal thinking. Energy Res Soc Sci 75:102005. https://doi.org/10.1016/j.erss.2021.102005

Lazoroska D, Palm J, Bergek A (2021) Perceptions of participation and the role of gender for the engagement in solar energy communities in Sweden. Energy Sustain Soc 11:1–12. https://doi.org/10.1186/s13705-021-00312-6

Fraune C (2015) Gender matters: Women, renewable energy, and citizen participation in Germany. Energy Res Soc Sci 7:55–65. https://doi.org/10.1016/j.erss.2015.02.005

Sovacool BK (2014) What are we doing here? Analyzing fifteen years of energy scholarship and proposing a social science research agenda. Energy Res Soc Sci 1:1–29. https://doi.org/10.1016/j.erss.2014.02.003

Innovación M de C e (2023) Científicas En Cifras. Estadísticas e indicadores la (des)igualdad género en la Form. y profesión científica. In: Ministerio de Ciencia e Innovación. https://www.ciencia.gob.es/InfoGeneralPortal/documento/f4f6bb28-cae5-4da2-85f4-067508c410eb . Accessed 21 Mar 2024

Estadística IIN de (2023) Estadística de Personal de las Universidades. Curso 2021–2022. In: Ministerio de Universidades. Gobierno de España.  https://www.universidades.gob.es/estadisticas-de-personal-de-las-universidades/ . Accessed 22 Mar 2024

del Carmen Mayoral Gastón M, Sala AML, Zafra R, González TS, Garcés E, Pozo-Bayón MÁ, Campillo NE, Orozco AO, Faraldos M, González-Sampériz P, Criado-Boado F, Iglesias TV-S, Ruiz MC, de Mujeres y Ciencia del CSIC C (2023) Informe Mujeres Investigadoras 2023. In: Digital CSIC https://digital.csic.es/handle/10261/312328 . Accessed 21 Mar 2024

Estudios D De, Fundación D, Fecyt T (2023) Energías renovables: Inquietudes sociales y nuevos desarrollos científico-tecnológicos. In: Informe Tendecncias Fecyt https://www.fecyt.es/es/publicacion/informe-tendencias-energias-renovables-inquietudes-sociales-y-nuevos-desarrollos . Accessed 21 Mar 2024

Ryan SE (2014) Rethinking gender and identity in energy studies. Energy Res Soc Sci 1:96–105. https://doi.org/10.1016/j.erss.2014.02.008

Carlsson-Kanyama A, Ripa Juliá I, Röhr U (2010) Unequal representation of women and men in energy company boards and management groups: are there implications for mitigation? Energy Policy 38:4737–4740. https://doi.org/10.1016/j.enpol.2010.03.072

Baruah B (2017) Renewable inequity? Women’s employment in clean energy in industrialized, emerging and developing economies. Natural resources forum. Blackwell Publishing Ltd, Oxford, pp 18–29

Google Scholar  

Mort H (2019) A review of energy and gender research in the global north. Technical University of Vienna, Vienna

Otero-Hermida P, García-Melón M (2018) Gender equality indicators for research and innovation from a responsible perspective: the case of Spain. Sustain 10:2980. https://doi.org/10.3390/su10092980

Otero-Hermida P (2020) For responsible and transformative innovation : putting people at the centre. exploring windows for change in a state initiative on gender and innovation monitoring within the European merge of governance frames. In: Velez-Cuartas, Romero-Goyeneche U of A (ed) Transformative metrics. Fondo Editorial FCSH, Antioquia, pp 106–124

Mergaert L, EL, (2014) Resistance to implementing gender mainstreaming in. Eur Integr online Pap 18:1–21. https://doi.org/10.1695/2014005

Adamson M, Kelan EK, Lewis P, Rumens N, Slíwa M (2016) The quality of equality: thinking differently about gender inclusion in organizations. Hum Resour Manag Int Dig 24:8–11. https://doi.org/10.1108/HRMID-04-2016-0060

Nielsen MW (2016) Limits to meritocracy? Gender in academic recruitment and promotion processes. Sci Public Policy 43:386–399. https://doi.org/10.1093/scipol/scv052

Benschop Y (2011) Gender practices in the construction of academic excellence: sheep with five legs. Organization 10:507–524. https://doi.org/10.1177/1350508411414293

European Commission (2015) Indicators for promoting and monitoring Responsible Research and Innovation Report from the Expert Group on Policy Indicators. In: Eur. Comm. https://op.europa.eu/en/publication-detail/-/publication/306a7ab4-f3cb-46cb-b675-9697caf5df19/language-en . Accessed 21 Mar 2024

Leydesdorff L, Wouters P, Bornmann L (2016) Professional and citizen bibliometrics: complementarities and ambivalences in the development and use. Scientometrics 109:2129–2150. https://doi.org/10.1007/s11192-016-2150-8

Corona-Sobrino C, García-Melón M, Poveda-Bautista R, González-Urango H (2020) Closing the gender gap at academic conferences: a tool for monitoring and assessing academic events. PLoS ONE 15:1–23. https://doi.org/10.1371/journal.pone.0243549

West C, Zimmerman DH (1987) Doing gender. Gend Soc 1:125–151

Suganthi L (2020) Sustainability indices for energy utilization using a multi-criteria decision model. Energy Sustain Soc 10:1–31. https://doi.org/10.1186/s13705-020-00249-2

Belton V, Stewart T (2002) Multiple criteria decision analysis: an integrated approach. Springer Science & Business Media, Kluwer Academic Publisher, Boston

Saaty TL (2001) The analytic network process: decision making with dependence and feedback. RWS Publications, Pittsburgh

Gabus A, Fontela EJBGRC (1972) World problems, an invitation to further thought within the framework of DEMATEL. Battelle Geneva Res Cent 1:12–14

Wu WW (2008) Choosing knowledge management strategies by using a combined ANP and DEMATEL approach. Expert Syst Appl 35:828–835. https://doi.org/10.1016/j.eswa.2007.07.025

Kadoić N, Divjak B, Begičević Ređep N (2019) Integrating the DEMATEL with the analytic network process for effective decision-making. Cent Eur J Oper Res 27:653–678. https://doi.org/10.1007/s10100-018-0601-4

Article   MathSciNet   Google Scholar  

Jefatura de Estado del Gobierno de España (2023) Ley Orgánica 2/2023, de 22 de marzo, del Sistema Universitario. In: Boletín Oficial del Estado 1–73. https://www.boe.es/eli/es/lo/2023/03/22/2/con . Accessed 21 Mar 2024

Schmidt-Scheele R, Hauser W, Scheel O, Minn F, Becker L, Buchgeister J, Hottenroth H, Junne T, Lehr U, Naegler T, Simon S, Sutardhio C, Tietze I, Ulrich P, Viere T, Weidlich A (2022) Sustainability assessments of energy scenarios: citizens’ preferences for and assessments of sustainability indicators. Energy Sustain Soc 12:1–23. https://doi.org/10.1186/s13705-022-00366-0

Xu X, Goswami S, Gulledge J, Wullschleger SD, Thornton PE (2016) Interdisciplinary research in climate and energy sciences. Wiley Interdiscip Rev Energy Environ 5:49–56. https://doi.org/10.1002/wene.180

Saaty TL, Peniwati K (2013) Group decision making: drawing out and reconciling differences. RWS publications, Pittsburgh

Hsieh HF, Shannon SE (2005) Three approaches to qualitative content analysis. Qual Health Res 15:1277–1288. https://doi.org/10.1177/1049732305276687

Li Y, Zhao K, Zhang F (2023) Identification of key influencing factors to Chinese coal power enterprises transition in the context of carbon neutrality: a modified fuzzy DEMATEL approach. Energy 263:125427. https://doi.org/10.1016/j.energy.2022.125427

Clancy J, Roehr U (2003) Gender and energy: is there a Northern perspective? Energy Sustain Dev 7:44–49. https://doi.org/10.1016/s0973-0826(08)60364-6

Lutzenhiser L (1992) A cultural model of household energy consumption. Energy 17:47–60

Shrestha B, Tiwari SR, Bajracharya SB, Keitsch MM, Rijal HB (2021) Review on the importance of gender perspective in household energy-saving behavior and energy transition for sustainability. Energies 14:7571. https://doi.org/10.3390/en14227571

Raaij V, Raaij WF Van (1984) Patterns of residential energy behavior. In: P. Ester (Ed.), Consumer behavior and energy policy: Selected/ed. proceedings of the 1st international conference, Noordwijkerhout, September, 1982. North-Holland Publishing Company, Amsterdam, pp. 97–118

Søraa RA, Anfinsen M, Foulds C, Korsnes M, Lagesen V, Robison R, Ryghaug M (2020) Diversifying diversity: Inclusive engagement, intersectionality, and gender identity in a European Social Sciences and Humanities Energy research project. Energy Res Soc Sci 62:101380. https://doi.org/10.1016/j.erss.2019.101380

Sayer RA (1992) Method in social science: A realist approach. Psychology Press, Routledge, New York

Thronicker, I., Poppen, J., Tartaroti, V., Heller, K., Schmidtchen J (2023) Diversity Monitoring - Key Indicator Report 2021. In: Helmholtz-Zentrum für Umweltforschung - UFZ, Interner Bericht. https://www.ufz.de/index.php?de=46546 . Accessed 20 Feb 2024

Download references

Acknowledgements

This is an extended and updated version of a paper originally presented at the 18th Conference on Sustainable Development of Energy, Water and Environment Systems (SDEWES 2022) held in Paphos, Cyprus, over the period 24th to 29th September 2023 (denoted then as paper SDEWES2023.00425 Proposal of an Energy Research Gender Indicator as a diagnostic tool for energy research organizations). Thanks to everyone who contributed to the study by commenting on, filling out and sharing the questionnaire and being interviewed. We would also like to thank the panel of experts in the surveys for their willingness to participate. Finally, we would like to thank Michael Colin Bennett for assisting us with the English revision of the final version of this paper.

Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was supported by project INVISIBLES funded by the regional public administration of Valencia under the grant (AICO/2021/133).

This work was also partly supported by the Ministry of Universities through the EU-funded Margarita Salas programme NextGeneration EU, Valencia University under the Grant (2021-1099).

This work was also partly supported by European Commission H2020 Scientific Understanding and Provision of an Enhanced and Robust Monitoring system for RRI SUPER_MoRRI (H2020-SWAFS/0467-Grant-agreement nº 824671).

Author information

Authors and affiliations.

INGENIO (CSIC-UPV), Universitat Politècnica de València, Camino de Vera, s/n, 46022, Valencia, Spain

Sara Sánchez-López, Rocío Poveda-Bautista, Paula Otero-Hermida & Mónica García-Melón

Departament Sociologia i Antropologia Social, Universitat de València, Av. dels Tarongers s/n, 46022, Valencia, Spain

Carmen Corona-Sobrino

You can also search for this author in PubMed   Google Scholar

Contributions

Sara Sánchez-López: writing—original draft, methodology, visualization, data curation. Rocío Poveda-Bautista: conceptualization, methodology, writing—original draft, writing—review & editing, supervision. Carmen Corona-Sobrino: conceptualization, methodology, software, data curation, visualization, writing—original draft, writing review & editing. Paula Otero-Hermida: conceptualization, writing—review & editing, supervision. Mónica García-Melón: methodology, data-curation, writing—review and editing.

Corresponding author

Correspondence to Sara Sánchez-López .

Ethics declarations

Ethics approval and consent to participate.

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Universitat Politècnica de València (P17_10_01_20, 10 January 2020). The participants provided their written informed consent to participate in this study. The questionnaire and interviews did not collect personally identifiable data, according to Delegación de Protección de Datos, IRB of Universitat Politècnica de València, and national regulations Law 3/2018, 5th of December, Protection of Personal Data and guarantee of digital rights, article 7, published in BOE núm. 294, 06/12/2018, (Reference: BOE-A-2018–16673). The purpose of the study was explained to all the participants at the beginning of the questionnaire and interviews. The participants were also informed that they have the right to leave the questionnaire or interview at any time whenever they feel uncomfortable or do not want to answer any further questions.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Sánchez-López, S., Poveda-Bautista, R., Corona-Sobrino, C. et al. Tackling gender disparities in energy research: a diagnostic tool for equality in research centres. Energ Sustain Soc 14 , 51 (2024). https://doi.org/10.1186/s13705-024-00479-8

Download citation

Received : 13 December 2023

Accepted : 14 July 2024

Published : 23 August 2024

DOI : https://doi.org/10.1186/s13705-024-00479-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Energy research
• Research centres

Energy, Sustainability and Society

ISSN: 2192-0567

• General enquiries: [email protected]