selection criteria in case study

• Search Menu
• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Urban Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Papyrology
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Evolution
• Language Reference
• Language Acquisition
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Media
• Music and Religion
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Science
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Lifestyle, Home, and Garden
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Neuroanaesthesia
• Clinical Neuroscience
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Ethics
• Business Strategy
• Business History
• Business and Technology
• Business and Government
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic History
• Economic Systems
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Natural Disasters (Environment)
• Social Impact of Environmental Issues (Social Science)
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• International Political Economy
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Behaviour
• Political Economy
• Political Institutions
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Theory
• Politics and Law
• Public Policy
• Public Administration
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Developmental and Physical Disabilities Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

28 Case Selection for Case‐Study Analysis: Qualitative and Quantitative Techniques

John Gerring is Professor of Political Science, Boston University.

• Published: 02 September 2009
• Cite Icon Cite
• Permissions Icon Permissions

This article presents some guidance by cataloging nine different techniques for case selection: typical, diverse, extreme, deviant, influential, crucial, pathway, most similar, and most different. It also indicates that if the researcher is starting from a quantitative database, then methods for finding influential outliers can be used. In particular, the article clarifies the general principles that might guide the process of case selection in case-study research. Cases are more or less representative of some broader phenomenon and, on that score, may be considered better or worse subjects for intensive analysis. The article then draws attention to two ambiguities in case-selection strategies in case-study research. The first concerns the admixture of several case-selection strategies. The second concerns the changing status of a case as a study proceeds. Some case studies follow only one strategy of case selection.

Case ‐study analysis focuses on one or several cases that are expected to provide insight into a larger population. This presents the researcher with a formidable problem of case selection: Which cases should she or he choose?

In large‐sample research, the task of case selection is usually handled by some version of randomization. However, in case‐study research the sample is small (by definition) and this makes random sampling problematic, for any given sample may be wildly unrepresentative. Moreover, there is no guarantee that a few cases, chosen randomly, will provide leverage into the research question of interest.

In order to isolate a sample of cases that both reproduces the relevant causal features of a larger universe (representativeness) and provides variation along the dimensions of theoretical interest (causal leverage), case selection for very small samples must employ purposive (nonrandom) selection procedures. Nine such methods are discussed in this chapter, each of which may be identified with a distinct case‐study “type:” typical, diverse, extreme, deviant, influential, crucial, pathway, most‐similar , and most‐different . Table 28.1 summarizes each type, including its general definition, a technique for locating it within a population of potential cases, its uses, and its probable representativeness.

While each of these techniques is normally practiced on one or several cases (the diverse, most‐similar, and most‐different methods require at least two), all may employ additional cases—with the proviso that, at some point, they will no longer offer an opportunity for in‐depth analysis and will thus no longer be “case studies” in the usual sense ( Gerring 2007 , ch. 2 ). It will also be seen that small‐ N case‐selection procedures rest, at least implicitly, upon an analysis of a larger population of potential cases (as does randomization). The case(s) identified for intensive study is chosen from a population and the reasons for this choice hinge upon the way in which it is situated within that population. This is the origin of the terminology—typical, diverse, extreme, et al. It follows that case‐selection procedures in case‐study research may build upon prior cross‐case analysis and that they depend, at the very least, upon certain assumptions about the broader population.

In certain circumstances, the case‐selection procedure may be structured by a quantitative analysis of the larger population. Here, several caveats must be satisfied. First, the inference must pertain to more than a few dozen cases; otherwise, statistical analysis is problematic. Second, relevant data must be available for that population, or a significant sample of that population, on key variables, and the researcher must feel reasonably confident in the accuracy and conceptual validity of these variables. Third, all the standard assumptions of statistical research (e.g. identification, specification, robustness) must be carefully considered, and wherever possible, tested. I shall not dilate further on these familiar issues except to warn the researcher against the unreflective use of statistical techniques. 1 When these requirements are not met, the researcher must employ a qualitative approach to case selection.

The point of this chapter is to elucidate general principles that might guide the process of case selection in case‐study research, building upon earlier work by Harry Eckstein, Arend Lijphart, and others. Sometimes, these principles can be applied in a quantitative framework and sometimes they are limited to a qualitative framework. In either case, the logic of case selection remains quite similar, whether practiced in small‐ N or large‐ N contexts.

Before we begin, a bit of notation is necessary. In this chapter “ N ” refers to cases, not observations. Here, I am concerned primarily with causal inference, rather than inferences that are descriptive or predictive in nature. Thus, all hypotheses involve at least one independent variable ( X ) and one dependent variable ( Y ). For convenience, I shall label the causal factor of special theoretical interest X   1 , and the control variable, or vector of controls (if there are any), X   2 . If the writer is concerned to explain a puzzling outcome, but has no preconceptions about its causes, then the research will be described as Y‐centered . If a researcher is concerned to investigate the effects of a particular cause, with no preconceptions about what these effects might be, the research will be described as X‐centered . If a researcher is concerned to investigate a particular causal relationship, the research will be described as X   1 / Y‐centered , for it connects a particular cause with a particular outcome. 2   X ‐ or Y ‐centered research is exploratory; its purpose is to generate new hypotheses. X   1 / Y‐centered research, by contrast, is confirmatory/disconfirmatory; its purpose is to test an existing hypothesis.

1 Typical Case

In order for a focused case study to provide insight into a broader phenomenon it must be representative of a broader set of cases. It is in this context that one may speak of a typical‐case approach to case selection. The typical case exemplifies what is considered to be a typical set of values, given some general understanding of a phenomenon. By construction, the typical case is also a representative case.

Some typical cases serve an exploratory role. Here, the author chooses a case based upon a set of descriptive characteristics and then probes for causal relationships. Robert and Helen Lynd (1929/1956) selected a single city “to be as representative as possible of contemporary American life.” Specifically, they were looking for a city with

1) a temperate climate; 2) a sufficiently rapid rate of growth to ensure the presence of a plentiful assortment of the growing pains accompanying contemporary social change; 3) an industrial culture with modern, high‐speed machine production; 4) the absence of dominance of the city's industry by a single plant (i.e., not a one‐industry town); 5) a substantial local artistic life to balance its industrial activity …; and 6) the absence of any outstanding peculiarities or acute local problems which would mark the city off from the midchannel sort of American community. ( Lynd and Lynd 1929/1956 , quoted in Yin 2004 , 29–30)

After examining a number of options the Lynds decided that Muncie, Indiana, was more representative than, or at least as representative as, other midsized cities in America, thus qualifying as a typical case.

This is an inductive approach to case selection. Note that typicality may be understood according to the mean, median, or mode on a particular dimension; there may be multiple dimensions (as in the foregoing example); and each may be differently weighted (some dimensions may be more important than others). Where the selection criteria are multidimensional and a large sample of potential cases is in play, some form of factor analysis may be useful in identifying the most‐typical case(s).

However, the more common employment of the typical‐case method involves a causal model of some phenomenon of theoretical interest. Here, the researcher has identified a particular outcome ( Y ), and perhaps a specific X   1 / Y hypothesis, which she wishes to investigate. In order to do so, she looks for a typical example of that causal relationship. Intuitively, one imagines that a case selected according to the mean values of all parameters must be a typical case relative to some causal relationship. However, this is by no means assured.

Suppose that the Lynds were primarily interested in explaining feelings of trust/distrust among members of different social classes (one of the implicit research goals of the Middletown study). This outcome is likely to be affected by many factors, only some of which are included in their six selection criteria. So choosing cases with respect to a causal hypothesis involves, first of all, identifying the relevant parameters. It involves, secondly, the selection of a case that has a “typical” value relative to the overall causal model; it is well explained. Cases with untypical scores on a particular dimension (e.g. very high or very low) may still be typical examples of a causal relationship. Indeed, they may be more typical than cases whose values lie close to the mean. Thus, a descriptive understanding of typicality is quite different from a causal understanding of typicality. Since it is the latter version that is more common, I shall adopt this understanding of typicality in the remainder of the discussion.

From a qualitative perspective, causal typicality involves the selection of a case that conforms to expectations about some general causal relationship. It performs as expected. In a quantitative setting, this notion is measured by the size of a case's residual in a large‐ N cross‐case model. Typical cases lie on or near the regression line; their residuals are small. Insofar as the model is correctly specified, the size of a case's residual (i.e. the number of standard deviations that separate the actual value from the fitted value) provides a helpful clue to how representative that case is likely to be. “Outliers” are unlikely to be representative of the target population.

Of course, just because a case has a low residual does not necessarily mean that it is a representative case (with respect to the causal relationship of interest). Indeed, the issue of case representativeness is an issue that can never be definitively settled. When one refers to a “typical case” one is saying, in effect, that the probability of a case's representativeness is high, relative to other cases. This test of typicality is misleading if the statistical model is mis‐specified. And it provides little insurance against errors that are purely stochastic. A case may lie directly on the regression line but still be, in some important respect, atypical. For example, it might have an odd combination of values; the interaction of variables might be different from other cases; or additional causal mechanisms might be at work. For this reason, it is important to supplement a statistical analysis of cases with evidence drawn from the case in question (the case study itself) and with our deductive knowledge of the world. One should never judge a case solely by its residual. Yet, all other things being equal, a case with a low residual is less likely to be unusual than a case with a high residual, and to this extent the method of case selection outlined here may be a helpful guide to case‐study researchers faced with a large number of potential cases.

By way of conclusion, it should be noted that because the typical case embodies a typical value on some set of causally relevant dimensions, the variance of interest to the researcher must lie within that case. Specifically, the typical case of some phenomenon may be helpful in exploring causal mechanisms and in solving identification problems (e.g. endogeneity between X   1 and Y , an omitted variable that may account for X   1   and Y , or some other spurious causal association). Depending upon the results of the case study, the author may confirm an existing hypothesis, disconfirm that hypothesis, or reframe it in a way that is consistent with the findings of the case study. These are the uses of the typical‐case study.

2 Diverse Cases

A second case‐selection strategy has as its primary objective the achievement of maximum variance along relevant dimensions. I refer to this as a diverse‐case method. For obvious reasons, this method requires the selection of a set of cases—at minimum, two—which are intended to represent the full range of values characterizing X   1 , Y , or some particular X   1 / Y relationship. 3

Where the individual variable of interest is categorical (on/off, red/black/blue, Jewish/Protestant/Catholic), the identification of diversity is readily apparent. The investigator simply chooses one case from each category. For a continuous variable, the choices are not so obvious. However, the researcher usually chooses both extreme values (high and low), and perhaps the mean or median as well. The researcher may also look for break‐points in the distribution that seem to correspond to categorical differences among cases. Or she may follow a theoretical hunch about which threshold values count, i.e. which are likely to produce different values on Y .

Another sort of diverse case takes account of the values of multiple variables (i.e. a vector), rather than a single variable. If these variables are categorical, the identification of causal types rests upon the intersection of each category. Two dichotomous variables produce a matrix with four cells. Three trichotomous variables produce a matrix of eight cells. And so forth. If all variables are deemed relevant to the analysis, the selection of diverse cases mandates the selection of one case drawn from within each cell. Let us say that an outcome is thought to be affected by sex, race (black/white), and marital status. Here, a diverse‐case strategy of case selection would identify one case within each of these intersecting cells—a total of eight cases. Things become slightly more complicated when one or more of the factors is continuous, rather than categorical. Here, the diversity of case values do not fall neatly into cells. Rather, these cells must be created by fiat—e.g. high, medium, low.

It will be seen that where multiple variables are under consideration, the logic of diverse‐case analysis rests upon the logic of typological theorizing—where different combinations of variables are assumed to have effects on an outcome that vary across types ( Elman 2005 ; George and Bennett 2005 , 235; Lazarsfeld and Barton 1951 ). George and Smoke, for example, wish to explore different types of deterrence failure—by “fait accompli,” by “limited probe,” and by “controlled pressure.” Consequently, they wish to find cases that exemplify each type of causal mechanism. 4

Diversity may thus refer to a range of variation on X or Y , or to a particular combination of causal factors (with or without a consideration of the outcome). In each instance, the goal of case selection is to capture the full range of variation along the dimension(s) of interest.

Since diversity can mean many things, its employment in a large‐ N setting is necessarily dependent upon how this key term is defined. If it is understood to pertain only to a single variable ( X   1 or Y ), then the task is fairly simple. A categorical variable mandates the choice of at least one case from each category—two if dichotomous, three if trichotomous, and so forth. A continuous variable suggests the choice of at least one “high” and “low” value, and perhaps one drawn from the mean or median. But other choices might also be justified, according to one's hunch about the underlying causal relationship or according to natural thresholds found in the data, which may be grouped into discrete categories. Single‐variable traits are usually easy to discover in a large‐ N setting through descriptive statistics or through visual inspection of the data.

Where diversity refers to particular combinations of variables, the relevant cross‐ case technique is some version of stratified random sampling (in a probabilistic setting) or Qualitative Comparative Analysis (in a deterministic setting) ( Ragin 2000 ). If the researcher suspects that a causal relationship is affected not only by combinations of factors but also by their sequencing , then the technique of analysis must incorporate temporal elements ( Abbott 2001 ; Abbott and Forrest 1986 ; Abbott and Tsay 2000 ). Thus, the method of identifying causal types rests upon whatever method of identifying causal relationships is employed in the large‐ N sample.

Note that the identification of distinct case types is intended to identify groups of cases that are internally homogeneous (in all respects that might affect the causal relationship of interest). Thus, the choice of cases within each group should not be problematic, and may be accomplished through random sampling or purposive case selection. However, if there is suspected diversity within each category, then measures should be taken to assure that the chosen cases are typical of each category. A case study should not focus on an atypical member of a subgroup.

Indeed, considerations of diversity and typicality often go together. Thus, in a study of globalization and social welfare systems, Duane Swank (2002) first identifies three distinctive groups of welfare states: “universalistic” (social democratic), “corporatist conservative,” and “liberal.” Next, he looks within each group to find the most‐typical cases. He decides that the Nordic countries are more typical of the universalistic model than the Netherlands since the latter has “some characteristics of the occupationally based program structure and a political context of Christian Democratic‐led governments typical of the corporatist conservative nations” ( Swank 2002 , 11; see also Esping‐Andersen 1990 ). Thus, the Nordic countries are chosen as representative cases within the universalistic case type, and are accompanied in the case‐study portion of his analysis by other cases chosen to represent the other welfare state types (corporatist conservative and liberal).

Evidently, when a sample encompasses a full range of variation on relevant parameters one is likely to enhance the representativeness of that sample (relative to some population). This is a distinct advantage. Of course, the inclusion of a full range of variation may distort the actual distribution of cases across this spectrum. If there are more “high” cases than “low” cases in a population and the researcher chooses only one high case and one low case, the resulting sample of two is not perfectly representative. Even so, the diverse‐case method probably has stronger claims to representativeness than any other small‐ N sample (including the standalone typical case). The selection of diverse cases has the additional advantage of introducing variation on the key variables of interest. A set of diverse cases is, by definition, a set of cases that encompasses a range of high and low values on relevant dimensions. There is, therefore, much to recommend this method of case selection. I suspect that these advantages are commonly understood and are applied on an intuitive level by case‐study researchers. However, the lack of a recognizable name—and an explicit methodological defense—has made it difficult for case‐study researchers to utilize this method of case selection, and to do so in an explicit and self‐conscious fashion. Neologism has its uses.

3 Extreme Case

The extreme‐case method selects a case because of its extreme value on an independent ( X   1 ) or dependent ( Y ) variable of interest. Thus, studies of domestic violence may choose to focus on extreme instances of abuse ( Browne 1987 ). Studies of altruism may focus on those rare individuals who risked their lives to help others (e.g. Holocaust resisters) ( Monroe 1996 ). Studies of ethnic politics may focus on the most heterogeneous societies (e.g. Papua New Guinea) in order to better understand the role of ethnicity in a democratic setting ( Reilly 2000–1 ). Studies of industrial policy often focus on the most successful countries (i.e. the NICS) ( Deyo 1987 ). And so forth. 5

Often an extreme case corresponds to a case that is considered to be prototypical or paradigmatic of some phenomena of interest. This is because concepts are often defined by their extremes, i.e. their ideal types. Italian Fascism defines the concept of Fascism, in part, because it offered the most extreme example of that phenomenon. However, the methodological value of this case, and others like it, derives from its extremity (along some dimension of interest), not its theoretical status or its status in the literature on a subject.

The notion of “extreme” may now be defined more precisely. An extreme value is an observation that lies far away from the mean of a given distribution. This may be measured (if there are sufficient observations) by a case's “Z score”—the number of standard deviations between a case and the mean value for that sample. Extreme cases have high Z scores, and for this reason may serve as useful subjects for intensive analysis.

For a continuous variable, the distance from the mean may be in either direction (positive or negative). For a dichotomous variable (present/absent), extremeness may be interpreted as unusual . If most cases are positive along a given dimension, then a negative case constitutes an extreme case. If most cases are negative, then a positive case constitutes an extreme case. It should be clear that researchers are not simply concerned with cases where something “happened,” but also with cases where something did not. It is the rareness of the value that makes a case valuable, in this context, not its positive or negative value. 6 Thus, if one is studying state capacity, a case of state failure is probably more informative than a case of state endurance simply because the former is more unusual. Similarly, if one is interested in incest taboos a culture where the incest taboo is absent or weak is probably more useful than a culture where it is present or strong. Fascism is more important than nonfascism. And so forth. There is a good reason, therefore, why case studies of revolution tend to focus on “revolutionary” cases. Theda Skocpol (1979) had much more to learn from France than from Austro‐Hungary since France was more unusual than Austro‐Hungary within the population of nation states that Skocpol was concerned to explain. The reason is quite simple: There are fewer revolutionary cases than nonrevolutionary cases; thus, the variation that we explore as a clue to causal relationships is encapsulated in these cases, against a background of nonrevolutionary cases.

Note that the extreme‐case method of case selection appears to violate the social science folk wisdom warning us not to “select on the dependent variable.” 7 Selecting cases on the dependent variable is indeed problematic if a number of cases are chosen, all of which lie on one end of a variable's spectrum (they are all positive or negative), and if the researcher then subjects this sample to cross‐case analysis as if it were representative of a population. 8 Results for this sort of analysis would almost assuredly be biased. Moreover, there will be little variation to explain since the values of each case are explicitly constrained.

However, this is not the proper employment of the extreme‐case method. (It is more appropriately labeled an extreme‐ sample method.) The extreme‐case method actually refers back to a larger sample of cases that lie in the background of the analysis and provide a full range of variation as well as a more representative picture of the population. It is a self‐conscious attempt to maximize variance on the dimension of interest, not to minimize it. If this population of cases is well understood— either through the author's own cross‐case analysis, through the work of others, or through common sense—then a researcher may justify the selection of a single case exemplifying an extreme value for within‐case analysis. If not, the researcher may be well advised to follow a diverse‐case method, as discussed above.

By way of conclusion, let us return to the problem of representativeness. It will be seen that an extreme case may be typical or deviant. There is simply no way to tell because the researcher has not yet specified an X   1 / Y causal proposition. Once such a causal proposition has been specified one may then ask whether the case in question is similar to some population of cases in all respects that might affect the X   1 / Y relationship of interest (i.e. unit homogeneous). It is at this point that it becomes possible to say, within the context of a cross‐case statistical model, whether a case lies near to, or far from, the regression line. However, this sort of analysis means that the researcher is no longer pursuing an extreme‐case method. The extreme‐case method is purely exploratory—a way of probing possible causes of Y , or possible effects of X , in an open‐ended fashion. If the researcher has some notion of what additional factors might affect the outcome of interest, or of what relationship the causal factor of interest might have with Y , then she ought to pursue one of the other methods explored in this chapter. This also implies that an extreme‐case method may transform into a different kind of approach as a study evolves; that is, as a more specific hypothesis comes to light. Useful extreme cases at the outset of a study may prove less useful at a later stage of analysis.

4 Deviant Case

The deviant‐case method selects that case(s) which, by reference to some general understanding of a topic (either a specific theory or common sense), demonstrates a surprising value. It is thus the contrary of the typical case. Barbara Geddes (2003) notes the importance of deviant cases in medical science, where researchers are habitually focused on that which is “pathological” (according to standard theory and practice). The New England Journal of Medicine , one of the premier journals of the field, carries a regular feature entitled Case Records of the Massachusetts General Hospital. These articles bear titles like the following: “An 80‐Year‐Old Woman with Sudden Unilateral Blindness” or “A 76‐Year‐Old Man with Fever, Dyspnea, Pulmonary Infiltrates, Pleural Effusions, and Confusion.” 9 Another interesting example drawn from the field of medicine concerns the extensive study now devoted to a small number of persons who seem resistant to the AIDS virus ( Buchbinder and Vittinghoff 1999 ; Haynes, Pantaleo, and Fauci 1996 ). Why are they resistant? What is different about these people? What can we learn about AIDS in other patients by observing people who have built‐in resistance to this disease?

Likewise, in psychology and sociology case studies may be comprised of deviant (in the social sense) persons or groups. In economics, case studies may consist of countries or businesses that overperform (e.g. Botswana; Microsoft) or underperform (e.g. Britain through most of the twentieth century; Sears in recent decades) relative to some set of expectations. In political science, case studies may focus on countries where the welfare state is more developed (e.g. Sweden) or less developed (e.g. the United States) than one would expect, given a set of general expectations about welfare state development. The deviant case is closely linked to the investigation of theoretical anomalies. Indeed, to say deviant is to imply “anomalous.” 10

Note that while extreme cases are judged relative to the mean of a single distribution (the distribution of values along a single variable), deviant cases are judged relative to some general model of causal relations. The deviant‐case method selects cases which, by reference to some (presumably) general relationship, demonstrate a surprising value. They are “deviant” in that they are poorly explained by the multivariate model. The important point is that deviant‐ness can only be assessed relative to the general (quantitative or qualitative) model. This means that the relative deviant‐ness of a case is likely to change whenever the general model is altered. For example, the United States is a deviant welfare state when this outcome is gauged relative to societal wealth. But it is less deviant—and perhaps not deviant at all—when certain additional (political and societal) factors are included in the model, as discussed in the epilogue. Deviance is model dependent. Thus, when discussing the concept of the deviant case it is helpful to ask the following question: Relative to what general model (or set of background factors) is Case A deviant?

Conceptually, we have said that the deviant case is the logical contrary of the typical case. This translates into a directly contrasting statistical measurement. While the typical case is one with a low residual (in some general model of causal relations), a deviant case is one with a high residual. This means, following our previous discussion, that the deviant case is likely to be an un representative case, and in this respect appears to violate the supposition that case‐study samples should seek to reproduce features of a larger population.

However, it must be borne in mind that the primary purpose of a deviant‐case analysis is to probe for new—but as yet unspecified—explanations. (If the purpose is to disprove an extant theory I shall refer to the study as crucial‐case, as discussed below.) The researcher hopes that causal processes identified within the deviant case will illustrate some causal factor that is applicable to other (more or less deviant) cases. This means that a deviant‐case study usually culminates in a general proposition, one that may be applied to other cases in the population. Once this general proposition has been introduced into the overall model, the expectation is that the chosen case will no longer be an outlier. Indeed, the hope is that it will now be typical , as judged by its small residual in the adjusted model. (The exception would be a circumstance in which a case's outcome is deemed to be “accidental,” and therefore inexplicable by any general model.)

This feature of the deviant‐case study should help to resolve questions about its representativeness. Even if it is not possible to measure the new causal factor (and thus to introduce it into a large‐ N cross‐case model), it may still be plausible to assert (based on general knowledge of the phenomenon) that the chosen case is representative of a broader population.

5 Influential Case

Sometimes, the choice of a case is motivated solely by the need to verify the assumptions behind a general model of causal relations. Here, the analyst attempts to provide a rationale for disregarding a problematic case or a set of problematic cases. That is to say, she attempts to show why apparent deviations from the norm are not really deviant, or do not challenge the core of the theory, once the circumstances of the special case or cases are fully understood. A cross‐case analysis may, after all, be marred by several classes of problems including measurement error, specification error, errors in establishing proper boundaries for the inference (the scope of the argument), and stochastic error (fluctuations in the phenomenon under study that are treated as random, given available theoretical resources). If poorly fitting cases can be explained away by reference to these kinds of problems, then the theory of interest is that much stronger. This sort of deviant‐case analysis answers the question, “What about Case A (or cases of type A)? How does that, seemingly disconfirming, case fit the model?”

Because its underlying purpose is different from the usual deviant‐case study, I offer a new term for this method. The influential case is a case that casts doubt upon a theory, and for that reason warrants close inspection. This investigation may reveal, after all, that the theory is validated—perhaps in some slightly altered form. In this guise, the influential case is the “case that proves the rule.” In other instances, the influential‐case analysis may contribute to disconfirming, or reconceptualizing, a theory. The key point is that the value of the case is judged relative to some extant cross‐case model.

A simple version of influential‐case analysis involves the confirmation of a key case's score on some critical dimension. This is essentially a question of measurement. Sometimes cases are poorly explained simply because they are poorly understood. A close examination of a particular context may reveal that an apparently falsifying case has been miscoded. If so, the initial challenge presented by that case to some general theory has been obviated.

However, the more usual employment of the influential‐case method culminates in a substantive reinterpretation of the case—perhaps even of the general model. It is not just a question of measurement. Consider Thomas Ertman's (1997) study of state building in Western Europe, as summarized by Gerardo Munck. This study argues

that the interaction of a) the type of local government during the first period of statebuilding, with b) the timing of increases in geopolitical competition, strongly influences the kind of regime and state that emerge. [Ertman] tests this hypothesis against the historical experience of Europe and finds that most countries fit his predictions. Denmark, however, is a major exception. In Denmark, sustained geopolitical competition began relatively late and local government at the beginning of the statebuilding period was generally participatory, which should have led the country to develop “patrimonial constitutionalism.” But in fact, it developed “bureaucratic absolutism.” Ertman carefully explores the process through which Denmark came to have a bureaucratic absolutist state and finds that Denmark had the early marks of a patrimonial constitutionalist state. However, the country was pushed off this developmental path by the influence of German knights, who entered Denmark and brought with them German institutions of local government. Ertman then traces the causal process through which these imported institutions pushed Denmark to develop bureaucratic absolutism, concluding that this development was caused by a factor well outside his explanatory framework. ( Munck 2004 , 118)

Ertman's overall framework is confirmed insofar as he has been able to show, by an in‐depth discussion of Denmark, that the causal processes stipulated by the general theory hold even in this apparently disconfirming case. Denmark is still deviant, but it is so because of “contingent historical circumstances” that are exogenous to the theory ( Ertman 1997 , 316).

Evidently, the influential‐case analysis is similar to the deviant‐case analysis. Both focus on outliers. However, as we shall see, they focus on different kinds of outliers. Moreover, the animating goals of these two research designs are quite different. The influential‐case study begins with the aim of confirming a general model, while the deviant‐case study has the aim of generating a new hypothesis that modifies an existing general model. The confusion stems from the fact that the same case study may fulfill both objectives—qualifying a general model and, at the same time, confirming its core hypothesis.

Thus, in their study of Roberto Michels's “iron law of oligarchy,” Lipset, Trow, and Coleman (1956) choose to focus on an organization—the International Typographical Union—that appears to violate the central presupposition. The ITU, as noted by one of the authors, has “a long‐term two‐party system with free elections and frequent turnover in office” and is thus anything but oligarchic ( Lipset 1959 , 70). As such, it calls into question Michels's grand generalization about organizational behavior. The authors explain this curious result by the extraordinarily high level of education among the members of this union. Michels's law is shown to be true for most organizations, but not all. It is true, with qualifications. Note that the respecification of the original model (in effect, Lipset, Trow, and Coleman introduce a new control variable or boundary condition) involves the exploration of a new hypothesis. In this instance, therefore, the use of an influential case to confirm an existing theory is quite similar to the use of a deviant case to explore a new theory.

In a quantitative idiom, influential cases are those that, if counterfactually assigned a different value on the dependent variable, would most substantially change the resulting estimates. They may or may not be outliers (high‐residual cases). Two quantitative measures of influence are commonly applied in regression diagnostics ( Belsey, Kuh, and Welsch 2004 ). The first, often referred to as the leverage of a case, derives from what is called the hat matrix . Based solely on each case's scores on the independent variables, the hat matrix tells us how much a change in (or a measurement error on) the dependent variable for that case would affect the overall regression line. The second is Cook's distance , a measure of the extent to which the estimates of all the parameters would change if a given case were omitted from the analysis. Cases with a large leverage or Cook's distance contribute quite a lot to the inferences drawn from a cross‐case analysis. In this sense, such cases are vital for maintaining analytic conclusions. Discovering a significant measurement error on the dependent variable or an important omitted variable for such a case may dramatically revise estimates of the overall relationships. Hence, it may be quite sensible to select influential cases for in‐depth study.

Note that the use of an influential‐case strategy of case selection is limited to instances in which a researcher has reason to be concerned that her results are being driven by one or a few cases. This is most likely to be true in small to moderate‐sized samples. Where N is very large—greater than 1,000, let us say—it is extremely unlikely that a small set of cases (much less an individual case) will play an “influential” role. Of course, there may be influential sets of cases, e.g. countries within a particular continent or cultural region, or persons of Irish extraction. Sets of influential observations are often problematic in a time‐series cross‐section data‐set where each unit (e.g. country) contains multiple observations (through time), and hence may have a strong influence on aggregate results. Still, the general rule is: the larger the sample, the less important individual cases are likely to be and, hence, the less likely a researcher is to use an influential‐case approach to case selection.

6 Crucial Case

Of all the extant methods of case selection perhaps the most storied—and certainly the most controversial—is the crucial‐case method, introduced to the social science world several decades ago by Harry Eckstein. In his seminal essay, Eckstein (1975 , 118) describes the crucial case as one “that must closely fit a theory if one is to have confidence in the theory's validity, or, conversely, must not fit equally well any rule contrary to that proposed.” A case is crucial in a somewhat weaker—but much more common—sense when it is most, or least, likely to fulfill a theoretical prediction. A “most‐likely” case is one that, on all dimensions except the dimension of theoretical interest, is predicted to achieve a certain outcome, and yet does not. It is therefore used to disconfirm a theory. A “least‐likely” case is one that, on all dimensions except the dimension of theoretical interest, is predicted not to achieve a certain outcome, and yet does so. It is therefore used to confirm a theory. In all formulations, the crucial‐case offers a most‐difficult test for an argument, and hence provides what is perhaps the strongest sort of evidence possible in a nonexperimental, single‐case setting.

Since the publication of Eckstein's influential essay, the crucial‐case approach has been claimed in a multitude of studies across several social science disciplines and has come to be recognized as a staple of the case‐study method. 11 Yet the idea of any single case playing a crucial (or “critical”) role is not widely accepted among most methodologists (e.g. Sekhon 2004 ). (Even its progenitor seems to have had doubts.)

Let us begin with the confirmatory (a.k.a. least‐likely) crucial case. The implicit logic of this research design may be summarized as follows. Given a set of facts, we are asked to contemplate the probability that a given theory is true. While the facts matter, to be sure, the effectiveness of this sort of research also rests upon the formal properties of the theory in question. Specifically, the degree to which a theory is amenable to confirmation is contingent upon how many predictions can be derived from the theory and on how “risky” each individual prediction is. In Popper's (1963 , 36) words, “Confirmations should count only if they are the result of risky predictions ; that is to say, if, unenlightened by the theory in question, we should have expected an event which was incompatible with the theory—and event which would have refuted the theory. Every ‘good’ scientific theory is a prohibition; it forbids certain things to happen. The more a theory forbids, the better it is” (see also Popper 1934/1968 ). A risky prediction is therefore one that is highly precise and determinate, and therefore unlikely to be achieved by the product of other causal factors (external to the theory of interest) or through stochastic processes. A theory produces many such predictions if it is fully elaborated, issuing predictions not only on the central outcome of interest but also on specific causal mechanisms, and if it is broad in purview. (The notion of riskiness may also be conceptualized within the Popperian lexicon as degrees of falsifiability .)

These points can also be articulated in Bayesian terms. Colin Howson and Peter Urbach explain: “The degree to which h [a hypothesis] is confirmed by e [a set of evidence] depends … on the extent to which P(eČh) exceeds P (e) , that is, on how much more probable e is relative to the hypothesis and background assumptions than it is relative just to background assumptions.” Again, “confirmation is correlated with how much more probable the evidence is if the hypothesis is true than if it is false” ( Howson and Urlbach 1989 , 86). Thus, the stranger the prediction offered by a theory—relative to what we would normally expect—the greater the degree of confirmation that will be afforded by the evidence. As an intuitive example, Howson and Urbach (1989 , 86) offer the following:

If a soothsayer predicts that you will meet a dark stranger sometime and you do in fact, your faith in his powers of precognition would not be much enhanced: you would probably continue to think his predictions were just the result of guesswork. However, if the prediction also gave the correct number of hairs on the head of that stranger, your previous scepticism would no doubt be severely shaken.

While these Popperian/Bayesian notions 12 are relevant to all empirical research designs, they are especially relevant to case‐study research designs, for in these settings a single case (or, at most, a small number of cases) is required to bear a heavy burden of proof. It should be no surprise, therefore, that Popper's idea of “riskiness” was to be appropriated by case‐study researchers like Harry Eckstein to validate the enterprise of single‐case analysis. (Although Eckstein does not cite Popper the intellectual lineage is clear.) Riskiness, here, is analogous to what is usually referred to as a “most‐ difficult” research design, which in a case‐study research design would be understood as a “least‐likely” case. Note also that the distinction between a “must‐fit” case and a least‐likely case—that, in the event, actually does fit the terms of a theory—is a matter of degree. Cases are more or less crucial for confirming theories. The point is that, in some circumstances, a paucity of empirical evidence may be compensated by the riskiness of the theory.

The crucial‐case research design is, perforce, a highly deductive enterprise; much depends on the quality of the theory under investigation. It follows that the theories most amenable to crucial‐case analysis are those which are lawlike in their precision, degree of elaboration, consistency, and scope. The more a theory attains the status of a causal law, the easier it will be to confirm, or to disconfirm, with a single case. Indeed, risky predictions are common in natural science fields such as physics, which in turn served as the template for the deductive‐nomological (“covering‐law”) model of science that influenced Eckstein and others in the postwar decades (e.g. Hempel 1942 ).

A frequently cited example is the first important empirical demonstration of the theory of relativity, which took the form of a single‐event prediction on the occasion of the May 29, 1919, solar eclipse ( Eckstein 1975 ; Popper 1963 ). Stephen Van Evera (1997 , 66–7) describes the impact of this prediction on the validation of Einstein's theory.

Einstein's theory predicted that gravity would bend the path of light toward a gravity source by a specific amount. Hence it predicted that during a solar eclipse stars near the sun would appear displaced—stars actually behind the sun would appear next to it, and stars lying next to the sun would appear farther from it—and it predicted the amount of apparent displacement. No other theory made these predictions. The passage of this one single‐case‐study test brought the theory wide acceptance because the tested predictions were unique—there was no plausible competing explanation for the predicted result—hence the passed test was very strong.

The strength of this test is the extraordinary fit between the theory and a set of facts found in a single case, and the corresponding lack of fit between all other theories and this set of facts. Einstein offered an explanation of a particular set of anomalous findings that no other existing theory could make sense of. Of course, one must assume that there was no—or limited—measurement error. And one must assume that the phenomenon of interest is largely invariant; light does not bend differently at different times and places (except in ways that can be understood through the theory of relativity). And one must assume, finally, that the theory itself makes sense on other grounds (other than the case of special interest); it is a plausible general theory. If one is willing to accept these a priori assumptions, then the 1919 “case study” provides a very strong confirmation of the theory. It is difficult to imagine a stronger proof of the theory from within an observational (nonexperimental) setting.

In social science settings, by contrast, one does not commonly find single‐case studies offering knockout evidence for a theory. This is, in my view, largely a product of the looseness (the underspecification) of most social science theories. George and Bennett point out that while the thesis of the democratic peace is as close to a “law” as social science has yet seen, it cannot be confirmed (or refuted) by looking at specific causal mechanisms because the causal pathways mandated by the theory are multiple and diverse. Under the circumstances, no single‐case test can offer strong confirmation of the theory ( George and Bennett 2005 , 209).

However, if one adopts a softer version of the crucial‐case method—the least‐likely (most difficult) case—then possibilities abound. Indeed, I suspect that, implicitly , most case‐study work that makes a positive argument focusing on a single case (without a corresponding cross‐case analysis) relies largely on the logic of the least‐ likely case. Rarely is this logic made explicit, except perhaps in a passing phrase or two. Yet the deductive logic of the “risky” prediction is central to the case‐study enterprise. Whether a case study is convincing or not often rests on the reader's evaluation of how strong the evidence for an argument might be, and this in turn—wherever cross‐ case evidence is limited and no manipulated treatment can be devised—rests upon an estimation of the degree of “fit” between a theory and the evidence at hand, as discussed.

Lily Tsai's (2007) investigation of governance at the village level in China employs several in‐depth case studies of villages which are chosen (in part) because of their least‐likely status relative to the theory of interest. Tsai's hypothesis is that villages with greater social solidarity (based on preexisting religious or familial networks) will develop a higher level of social trust and mutual obligation and, as a result, will experience better governance. Crucial cases, therefore, are villages that evidence a high level of social solidarity but which, along other dimensions, would be judged least likely to develop good governance, e.g. they are poor, isolated, and lack democratic institutions or accountability mechanisms from above. “Li Settlement,” in Fujian province, is such a case. The fact that this impoverished village nonetheless boasts an impressive set of infrastructural accomplishments such as paved roads with drainage ditches (a rarity in rural China) suggests that something rather unusual is going on here. Because her case is carefully chosen to eliminate rival explanations, Tsai's conclusions about the special role of social solidarity are difficult to gainsay. How else is one to explain this otherwise anomalous result? This is the strength of the least‐likely case, where all other plausible causal factors for an outcome have been minimized. 13

Jack Levy (2002 , 144) refers to this, evocatively, as a “Sinatra inference:” if it can make it here, it can make it anywhere (see also Khong 1992 , 49; Sagan 1995 , 49; Shafer 1988 , 14–6). Thus, if social solidarity has the hypothesized effect in Li Settlement it should have the same effect in more propitious settings (e.g. where there is greater economic surplus). The same implicit logic informs many case‐study analyses where the intent of the study is to confirm a hypothesis on the basis of a single case.

Another sort of crucial case is employed for the purpose of dis confirming a causal hypothesis. A central Popperian insight is that it is easier to disconfirm an inference than to confirm that same inference. (Indeed, Popper doubted that any inference could be fully confirmed, and for this reason preferred the term “corroborate.”) This is particularly true of case‐study research designs, where evidence is limited to one or several cases. The key proviso is that the theory under investigation must take a consistent (a.k.a. invariant, deterministic) form, even if its predictions are not terrifically precise, well elaborated, or broad.

As it happens, there are a fair number of invariant propositions floating around the social science disciplines (Goertz and Levy forthcoming; Goertz and Starr 2003 ). It used to be argued, for example, that political stability would occur only in countries that are relatively homogeneous, or where existing heterogeneities are mitigated by cross‐cutting cleavages ( Almond 1956 ; Bentley 1908/1967 ; Lipset 1960/1963 ; Truman 1951 ). Arend Lijphart's (1968) study of the Netherlands, a peaceful country with reinforcing social cleavages, is commonly viewed as refuting this theory on the basis of a single in‐depth case analysis. 14

Granted, it may be questioned whether presumed invariant theories are really invariant; perhaps they are better understood as probabilistic. Perhaps, that is, the theory of cross‐cutting cleavages is still true, probabilistically, despite the apparent Dutch exception. Or perhaps the theory is still true, deterministically, within a subset of cases that does not include the Netherlands. (This sort of claim seems unlikely in this particular instance, but it is quite plausible in many others.) Or perhaps the theory is in need of reframing; it is true, deterministically, but applies only to cross‐ cutting ethnic/racial cleavages, not to cleavages that are primarily religious. One can quibble over what it means to “disconfirm” a theory. The point is that the crucial case has, in all these circumstances, provided important updating of a theoretical prior.

Heretofore, I have treated causal factors as dichotomous. Countries have either reinforcing or cross‐cutting cleavages and they have regimes that are either peaceful or conflictual. Evidently, these sorts of parameters are often matters of degree. In this reading of the theory, cases are more or less crucial. Accordingly, the most useful—i.e. most crucial—case for Lijphart's purpose is one that has the most segregated social groups and the most peaceful and democratic track record. In these respects, the Netherlands was a very good choice. Indeed, the degree of disconfirmation offered by this case study is probably greater than the degree of disconfirmation that might have been provided by other cases such as India or Papua New Guinea—countries where social peace has not always been secure. The point is that where variables are continuous rather than dichotomous it is possible to evaluate potential cases in terms of their degree of crucialness .

Note that the crucial‐case method of case‐selection, whether employed in a confirmatory or disconfirmatory mode, cannot be employed in a large‐ N context. This is because an explicit cross‐case model would render the crucial‐case study redundant. Once one identifies the relevant parameters and the scores of all cases on those parameters, one has in effect constructed a cross‐case model that confirms or disconfirms the theory in question. The case study is thenceforth irrelevant, at least as a means of decisive confirmation or disconfirmation. 15 It remains highly relevant as a means of exploring causal mechanisms, of course. Yet, because this objective is quite different from that which is usually associated with the term, I enlist a new term for this technique.

7 Pathway Case

One of the most important functions of case‐study research is the elucidation of causal mechanisms. But which sort of case is most useful for this purpose? Although all case studies presumably shed light on causal mechanisms, not all cases are equally transparent. In situations where a causal hypothesis is clear and has already been confirmed by cross‐case analysis, researchers are well advised to focus on a case where the causal effect of X   1 on Y can be isolated from other potentially confounding factors ( X   2 ). I shall call this a pathway case to indicate its uniquely penetrating insight into causal mechanisms. In contrast to the crucial case, this sort of method is practicable only in circumstances where cross‐case covariational patterns are well studied and where the mechanism linking X   1 and Y remains dim. Because the pathway case builds on prior cross‐case analysis, the problem of case selection must be situated within that sample. There is no standalone pathway case.

The logic of the pathway case is clearest in situations of causal sufficiency—where a causal factor of interest, X   1 , is sufficient by itself (though perhaps not necessary) to account for Y 's value (0 or 1). The other causes of Y , about which we need make no assumptions, are designated as a vector, X   2 .

Note that wherever various causal factors are substitutable for one another, each factor is conceptualized (individually) as sufficient ( Braumoeller 2003 ). Thus, situations of causal equifinality presume causal sufficiency on the part of each factor or set of conjoint factors. An example is provided by the literature on democratization, which stipulates three main avenues of regime change: leadership‐initiated reform, a controlled opening to opposition, or the collapse of an authoritarian regime ( Colomer 1991 ). The case‐study format constrains us to analyze one at a time, so let us limit our scope to the first one—leadership‐initiated reform. So considered, a causal‐pathway case would be one with the following features: (a) democratization, (b) leadership‐initiated reform, (c) no controlled opening to the opposition, (d) no collapse of the previous authoritarian regime, and (e) no other extraneous factors that might affect the process of democratization. In a case of this type, the causal mechanisms by which leadership‐initiated reform may lead to democratization will be easiest to study. Note that it is not necessary to assume that leadership‐initiated reform always leads to democratization; it may or may not be a deterministic cause. But it is necessary to assume that leadership‐initiated reform can sometimes lead to democratization on its own (given certain background features).

Now let us move from these examples to a general‐purpose model. For heuristic purposes, let us presume that all variables in that model are dichotomous (coded as 0 or 1) and that the model is complete (all causes of Y are included). All causal relationships will be coded so as to be positive: X   1 and Y covary as do X   2 and Y . This allows us to visualize a range of possible combinations at a glance.

Recall that the pathway case is always focused, by definition, on a single causal factor, denoted X   1 . (The researcher's focus may shift to other causal factors, but may only focus on one causal factor at a time.) In this scenario, and regardless of how many additional causes of Y there might be (denoted X   2 , a vector of controls), there are only eight relevant case types, as illustrated in Table 28.2 . Identifying these case types is a relatively simple matter, and can be accomplished in a small‐ N sample by the construction of a truth‐table (modeled after Table 28.2 ) or in a large‐ N sample by the use of cross‐tabs.

Notes : X   1 = the variable of theoretical interest. X   2 = a vector of controls (a score of 0 indicates that all control variables have a score of 0, while a score of 1 indicates that all control variables have a score of 1). Y = the outcome of interest. A–H = case types (the N for each case type is indeterminate). G, H = possible pathway cases. Sample size = indeterminate.

Assumptions : (a) all variables can be coded dichotomously (a binary coding of the concept is valid); (b) all independent variables are positively correlated with Y in the general case; ( c ) X   1 is (at least sometimes) a sufficient cause of Y .

Note that the total number of combinations of values depends on the number of control variables, which we have represented with a single vector, X   2 . If this vector consists of a single variable then there are only eight case types. If this vector consists of two variables ( X   2a , X   2b ) then the total number of possible combinations increases from eight (2 3 ) to sixteen (2 4 ). And so forth. However, none of these combinations is relevant for present purposes except those where X   2a and X   2b have the same value (0 or 1). “Mixed” cases are not causal pathway cases, for reasons that should become clear.

The pathway case, following the logic of the crucial case, is one where the causal factor of interest, X   1 , correctly predicts Y while all other possible causes of Y (represented by the vector, X   2 ) make “wrong” predictions. If X   1 is—at least in some circumstances—a sufficient cause of Y , then it is these sorts of cases that should be most useful for tracing causal mechanisms. There are only two such cases in Ta b l e 28.2—G and H. In all other cases, the mechanism running from X   1 to Y would be difficult to discern either because X   1 and Y are not correlated in the usual way (constituting an unusual case, in the terms of our hypothesis) or because other confounding factors ( X   2 ) intrude. In case A, for example, the positive value on Y could be a product of X   1 or X   2 . An in‐depth examination of this case is not likely to be very revealing.

Keep in mind that because the researcher already knows from her cross‐case examination what the general causal relationships are, she knows (prior to the case‐ study investigation) what constitutes a correct or incorrect prediction. In the crucial‐ case method, by contrast, these expectations are deductive rather than empirical. This is what differentiates the two methods. And this is why the causal pathway case is useful principally for elucidating causal mechanisms rather than verifying or falsifying general propositions (which are already more or less apparent from the cross‐case evidence). Of course, we must leave open the possibility that the investigation of causal mechanisms would invalidate a general claim, if that claim is utterly contingent upon a specific set of causal mechanisms and the case study shows that no such mechanisms are present. However, this is rather unlikely in most social science settings. Usually, the result of such a finding will be a reformulation of the causal processes by which X   1 causes Y —or, alternatively, a realization that the case under investigation is aberrant (atypical of the general population of cases).

Sometimes, the research question is framed as a unidirectional cause: one is interested in why 0 becomes 1 (or vice versa) but not in why 1 becomes 0. In our previous example, we asked why democracies fail, not why countries become democratic or authoritarian. So framed, there can be only one type of causal‐pathway case. (Whether regime failure is coded as 0 or 1 is a matter of taste.) Where researchers are interested in bidirectional causality—a movement from 0 to 1 as well as from 1 to 0—there are two possible causal‐pathway cases, G and H. In practice, however, one of these case types is almost always more useful than the other. Thus, it seems reasonable to employ the term “pathway case” in the singular. In order to determine which of these two case types will be more useful for intensive analysis the researcher should look to see whether each case type exhibits desirable features such as: (a) a rare (unusual) value on X   1 or Y (designated “extreme” in our previous discussion), (b) observable temporal variation in X   1 , ( c ) an X   1 / Y relationship that is easier to study (it has more visible features; it is more transparent), or (d) a lower residual (thus indicating a more typical case, within the terms of the general model). Usually, the choice between G and H is intuitively obvious.

Now, let us consider a scenario in which all (or most) variables of concern to the model are continuous, rather than dichotomous. Here, the job of case selection is considerably more complex, for causal “sufficiency” (in the usual sense) cannot be invoked. It is no longer plausible to assume that a given cause can be entirely partitioned, i.e. rival factors eliminated. However, the search for a pathway case may still be viable. What we are looking for in this scenario is a case that satisfies two criteria: (1) it is not an outlier (or at least not an extreme outlier) in the general model and (2) its score on the outcome ( Y ) is strongly influenced by the theoretical variable of interest ( X   1 ), taking all other factors into account ( X   2 ). In this sort of case it should be easiest to “see” the causal mechanisms that lie between X   1 and Y .

Achieving the second desiderata requires a bit of manipulation. In order to determine which (nonoutlier) cases are most strongly affected by X   1 , given all the other parameters in the model, one must compare the size of the residuals for each case in a reduced form model, Y = Constant + X   2 + Res reduced , with the size of the residuals for each case in a full model, Y = Constant + X   2 + X   1 + Res full . The pathway case is that case, or set of cases, which shows the greatest difference between the residual for the reduced‐form model and the full model (ΔResidual). Thus,

Note that the residual for a case must be smaller in the full model than in the reduced‐ form model; otherwise, the addition of the variable of interest ( X   1 ) pulls the case away from the regression line. We want to find a case where the addition of X   1 pushes the case towards the regression line, i.e. it helps to “explain” that case.

As an example, let us suppose that we are interested in exploring the effect of mineral wealth on the prospects for democracy in a society. According to a good deal of work on this subject, countries with a bounty of natural resources—particularly oil—are less likely to democratize (or once having undergone a democratic transition, are more likely to revert to authoritarian rule) ( Barro 1999 ; Humphreys 2005 ; Ross 2001 ). The cross‐country evidence is robust. Yet as is often the case, the causal mechanisms remain rather obscure. In order to better understand this phenomenon it may be worthwhile to exploit the findings of cross‐country regression models in order to identify a country whose regime type (i.e. its democracy “score” on some general index) is strongly affected by its natural‐research wealth, all other things held constant. An analysis of this sort identifies two countries— the United Arab Emirates and Kuwait—with high Δ Residual values and modest residuals in the full model (signifying that these cases are not outliers). Researchers seeking to explore the effect of oil wealth on regime type might do well to focus on these two cases since their patterns of democracy cannot be well explained by other factors—e.g. economic development, religion, European influence, or ethnic fractionalization. The presence of oil wealth in these countries would appear to have a strong independent effect on the prospects for democratization in these cases, an effect that is well modeled by general theory and by the available cross‐case evidence.

To reiterate, the logic of causal “elimination” is much more compelling where variables are dichotomous and where causal sufficiency can be assumed ( X   1 is sufficient by itself, at least in some circumstances, to cause Y ). Where variables are continuous, the strategy of the pathway case is more dubious, for potentially confounding causal factors ( X   2 ) cannot be neatly partitioned. Even so, we have indicated why the selection of a pathway case may be a logical approach to case‐study analysis in many circumstances.

The exceptions may be briefly noted. Sometimes, where all variables in a model are dichotomous, there are no pathway cases, i.e. no cases of type G or H (in Table 28.2 ). This is known as the “empty cell” problem, or a problem of severe causal multicollinearity. The universe of observational data does not always oblige us with cases that allow us to independently test a given hypothesis. Where variables are continuous, the analogous problem is that of a causal variable of interest ( X   1 ) that has only minimal effects on the outcome of interest. That is, its role in the general model is quite minor. In these situations, the only cases that are strongly affected by X   1 —if there are any at all—may be extreme outliers, and these sorts of cases are not properly regarded as providing confirmatory evidence for a proposition, for reasons that are abundantly clear by now.

Finally, it should be clarified that the identification of a causal pathway case does not obviate the utility of exploring other cases. One might, for example, want to compare both sorts of potential pathway cases—G and H—with each other. Many other combinations suggest themselves. However, this sort of multi‐case investigation moves beyond the logic of the causal‐pathway case.

8 Most‐similar Cases

The most‐similar method employs a minimum of two cases. 16 In its purest form, the chosen pair of cases is similar in all respects except the variable(s) of interest. If the study is exploratory (i.e. hypothesis generating), the researcher looks for cases that differ on the outcome of theoretical interest but are similar on various factors that might have contributed to that outcome, as illustrated in Table 28.3 (A) . This is a common form of case selection at the initial stage of research. Often, fruitful analysis begins with an apparent anomaly: two cases are apparently quite similar, and yet demonstrate surprisingly different outcomes. The hope is that intensive study of these cases will reveal one—or at most several—factors that differ across these cases. These differing factors ( X   1 ) are looked upon as putative causes. At this stage, the research may be described by the second diagram in Table 28.3 (B) . Sometimes, a researcher begins with a strong hypothesis, in which case her research design is confirmatory (hypothesis testing) from the get‐go. That is, she strives to identify cases that exhibit different outcomes, different scores on the factor of interest, and similar scores on all other possible causal factors, as illustrated in the second (hypothesis‐testing) diagram in Table 28.3 (B) .

The point is that the purpose of a most‐similar research design, and hence its basic setup, often changes as a researcher moves from an exploratory to a confirmatory mode of analysis. However, regardless of where one begins, the results, when published, look like a hypothesis‐testing research design. Question marks have been removed: (A) becomes (B) in Table 28.3 .

As an example, let us consider Leon Epstein's classic study of party cohesion, which focuses on two “most‐similar” countries, the United States and Canada. Canada has highly disciplined parties whose members vote together on the floor of the House of Commons while the United States has weak, undisciplined parties, whose members often defect on floor votes in Congress. In explaining these divergent outcomes, persistent over many years, Epstein first discusses possible causal factors that are held more or less constant across the two cases. Both the United States and Canada inherited English political cultures, both have large territories and heterogeneous populations, both are federal, and both have fairly loose party structures with strong regional bases and a weak center. These are the “control” variables. Where they differ is in one constitutional feature: Canada is parliamentary while the United States is presidential. And it is this institutional difference that Epstein identifies as the crucial (differentiating) cause. (For further examples of the most‐similar method see Brenner 1976 ; Hamilton 1977 ; Lipset 1968 ; Miguel 2004 ; Moulder 1977 ; Posner 2004 .)

X   1 = the variable of theoretical interest. X   2 = a vector of controls. Y = the outcome of interest.

Several caveats apply to any most‐similar analysis (in addition to the usual set of assumptions applying to all case‐study analysis). First, each causal factor is understood as having an independent and additive effect on the outcome; there are no “interaction” effects. Second, one must code cases dichotomously (high/low, present/absent). This is straightforward if the underlying variables are also dichotomous (e.g. federal/unitary). However, it is often the case that variables of concern in the model are continuous (e.g. party cohesion). In this setting, the researcher must “dichotomize” the scoring of cases so as to simplify the two‐case analysis. (Some flexibility is admissible on the vector of controls ( X   2 ) that are “held constant” across the cases. Nonidentity is tolerable if the deviation runs counter to the predicted hypothesis. For example, Epstein describes both the United States and Canada as having strong regional bases of power, a factor that is probably more significant in recent Canadian history than in recent American history. However, because regional bases of power should lead to weaker parties, rather than stronger parties, this element of nonidentity does not challenge Epstein's conclusions. Indeed, it sets up a most‐difficult research scenario, as discussed above.)

In one respect the requirements for case control are not so stringent. Specifically, it is not usually necessary to measure control variables (at least not with a high degree of precision) in order to control for them. If two countries can be assumed to have similar cultural heritages one needn't worry about constructing variables to measure that heritage. One can simply assert that, whatever they are, they are more or less constant across the two cases. This is similar to the technique employed in a randomized experiment, where the researcher typically does not attempt to measure all the factors that might affect the causal relationship of interest. She assumes, rather, that these unknown factors have been neutralized across the treatment and control groups by randomization or by the choice of a sample that is internally homogeneous.

The most useful statistical tool for identifying cases for in‐depth analysis in a most‐ similar setting is probably some variety of matching strategy—e.g. exact matching, approximate matching, or propensity‐score matching. 17 The product of this procedure is a set of matched cases that can be compared in whatever way the researcher deems appropriate. These are the “most‐similar” cases. Rosenbaum and Silber (2001 , 223) summarize:

Unlike model‐based adjustments, where [individuals] vanish and are replaced by the coefficients of a model, in matching, ostensibly comparable patterns are compared directly, one by one. Modern matching methods involve statistical modeling and combinatorial algorithms, but the end result is a collection of pairs or sets of people who look comparable, at least on average. In matching, people retain their integrity as people, so they can be examined and their stories can be told individually.

Matching, conclude the authors, “facilitates, rather than inhibits, thick description” ( Rosenbaum and Silber 2001 , 223).

In principle, the same matching techniques that have been used successfully in observational studies of medical treatments might also be adapted to the study of nation states, political parties, cities, or indeed any traditional paired cases in the social sciences. Indeed, the current popularity of matching among statisticians—relative, that is, to garden‐variety regression models—rests upon what qualitative researchers would recognize as a “case‐based” approach to causal analysis. If Rosenbaum and Silber are correct, it may be perfectly reasonable to appropriate this large‐ N method of analysis for case‐study purposes.

As with other methods of case selection, the most‐similar method is prone to problems of nonrepresentativeness. If employed in a qualitative fashion (without a systematic cross‐case selection strategy), potential biases in the chosen case must be addressed in a speculative way. If the researcher employs a matching technique of case selection within a large‐ N sample, the problem of potential bias can be addressed by assuring the choice of cases that are not extreme outliers, as judged by their residuals in the full model. Most‐similar cases should also be “typical” cases, though some scope for deviance around the regression line may be acceptable for purposes of finding a good fit among cases.

X   1 = the variable of theoretical interest. X   2a–d = a vector of controls. Y = the outcome of interest.

9 Most‐different Cases

A final case‐selection method is the reverse image of the previous method. Here, variation on independent variables is prized, while variation on the outcome is eschewed. Rather than looking for cases that are most‐similar, one looks for cases that are most‐ different . Specifically, the researcher tries to identify cases where just one independent variable ( X   1 ), as well as the dependent variable ( Y ), covary, while all other plausible factors ( X   2a–d ) show different values. 18

The simplest form of this two‐case comparison is illustrated in Table 28.4 . Cases A and B are deemed “most different,” though they are similar in two essential respects— the causal variable of interest and the outcome.

As an example, I follow Marc Howard's (2003) recent work, which explores the enduring impact of Communism on civil society. 19 Cross‐national surveys show a strong correlation between former Communist regimes and low social capital, controlling for a variety of possible confounders. It is a strong result. Howard wonders why this relationship is so strong and why it persists, and perhaps even strengthens, in countries that are no longer socialist or authoritarian. In order to answer this question, he focuses on two most‐different cases, Russia and East Germany. These two countries were quite different—in all ways other than their Communist experience— prior to the Soviet era, during the Soviet era (since East Germany received substantial subsidies from West Germany), and in the post‐Soviet era, as East Germany was absorbed into West Germany. Yet, they both score near the bottom of various cross‐ national indices intended to measure the prevalence of civic engagement in the current era. Thus, Howard's (2003 , 6–9) case selection procedure meets the requirements of the most‐different research design: Variance is found on all (or most) dimensions aside from the key factor of interest (Communism) and the outcome (civic engagement).

What leverage is brought to the analysis from this approach? Howard's case studies combine evidence drawn from mass surveys and from in‐depth interviews of small, stratified samples of Russians and East Germans. (This is a good illustration, incidentally, of how quantitative and qualitative evidence can be fruitfully combined in the intensive study of several cases.) The product of this analysis is the identification of three causal pathways that, Howard (2003 , 122) claims, help to explain the laggard status of civil society in post‐Communist polities: “the mistrust of communist organizations, the persistence of friendship networks, and the disappointment with post‐communism.” Simply put, Howard (2003 , 145) concludes, “a great number of citizens in Russia and Eastern Germany feel a strong and lingering sense of distrust of any kind of public organization, a general satisfaction with their own personal networks (accompanied by a sense of deteriorating relations within society overall), and disappointment in the developments of post‐communism.”

The strength of this most‐different case analysis is that the results obtained in East Germany and Russia should also apply in other post‐Communist polities (e.g. Lithuania, Poland, Bulgaria, Albania). By choosing a heterogeneous sample, Howard solves the problem of representativeness in his restricted sample. However, this sample is demonstrably not representative across the population of the inference, which is intended to cover all countries of the world.

More problematic is the lack of variation on key causal factors of interest— Communism and its putative causal pathways. For this reason, it is difficult to reach conclusions about the causal status of these factors on the basis of the most‐different analysis alone. It is possible, that is, that the three causal pathways identified by Howard also operate within polities that never experienced Communist rule.

Nor does it seem possible to conclusively eliminate rival hypotheses on the basis of this most‐different analysis. Indeed, this is not Howard's intention. He wishes merely to show that whatever influence on civil society might be attributed to economic, cultural, and other factors does not exhaust this subject.

My considered judgment is that the most‐different research design provides minimal leverage into the problem of why Communist systems appear to suppress civic engagement, years after their disappearance. Fortunately, this is not the only research design employed by Howard in his admirable study. Indeed, the author employs two other small‐ N cross‐case methods, as well as a large‐ N cross‐country statistical analysis. These methods do most of the analytic work. East Germany may be regarded as a causal pathway case (see above). It has all the attributes normally assumed to foster civic engagement (e.g. a growing economy, multiparty competition, civil liberties, a free press, close association with Western European culture and politics), but nonetheless shows little or no improvement on this dimension during the post‐ transition era ( Howard 2003 , 8). It is plausible to attribute this lack of change to its Communist past, as Howard does, in which case East Germany should be a fruitful case for the investigation of causal mechanisms. The contrast between East and West Germany provides a most‐similar analysis since the two polities share virtually everything except a Communist past. This variation is also deftly exploited by Howard.

I do not wish to dismiss the most‐different research method entirely. Surely, Howard's findings are stronger with the intensive analysis of Russia than they would be without. Yet his book would not stand securely on the empirical foundation provided by most‐different analysis alone. If one strips away the pathway‐case (East Germany) and the most‐similar analysis (East/West Germany) there is little left upon which to base an analysis of causal relations (aside from the large‐ N cross‐national analysis). Indeed, most scholars who employ the most‐different method do so in conjunction with other methods. 20 It is rarely, if ever, a standalone method. 21

Generalizing from this discussion of Marc Howard's work, I offer the following summary remarks on the most‐different method of case analysis. (I leave aside issues faced by all case‐study analyses, issues that are explored in Gerring 2007 .)

Let us begin with a methodological obstacle that is faced by both Millean styles of analysis—the necessity of dichotomizing every variable in the analysis. Recall that, as with most‐similar analysis, differences across cases must generally be sizeable enough to be interpretable in an essentially dichotomous fashion (e.g. high/low, present/absent) and similarities must be close enough to be understood as essentially identical (e.g. high/high, present/present). Otherwise the results of a Millean style analysis are not interpretable. The problem of “degrees” is deadly if the variables under consideration are, by nature, continuous (e.g. GDP). This is a particular concern in Howard's analysis, where East Germany scores somewhat higher than Russia in civic engagement; they are both low, but Russia is quite a bit lower. Howard assumes that this divergence is minimal enough to be understood as a difference of degrees rather than of kinds, a judgment that might be questioned. In these respects, most‐different analysis is no more secure—but also no less—than most‐similar analysis.

In one respect, most‐different analysis is superior to most‐similar analysis. If the coding assumptions are sound, the most‐different research design may be quite useful for eliminating necessary causes . Causal factors that do not appear across the chosen cases—e.g. X   2a–d in Table 28.4 —are evidently unnecessary for the production of Y . However, it does not follow that the most‐different method is the best method for eliminating necessary causes. Note that the defining feature of this method is the shared element across cases— X   1 in Table 28.4 . This feature does not help one to eliminate necessary causes. Indeed, if one were focused solely on eliminating necessary causes one would presumably seek out cases that register the same outcomes and have maximum diversity on other attributes. In Table 28.4 , this would be a set of cases that satisfy conditions X   2a–d , but not X   1 . Thus, even the presumed strength of the most‐different analysis is not so strong.

Usually, case‐study analysis is focused on the identification (or clarification) of causal relations, not the elimination of possible causes. In this setting, the most‐ different technique is useful, but only if assumptions of causal uniqueness hold. By “causal uniqueness,” I mean a situation in which a given outcome is the product of only one cause: Y cannot occur except in the presence of X . X is necessary, and in some situations (given certain background conditions) sufficient, to cause Y . 22

Consider the following hypothetical example. Suppose that a new disease, about which little is known, has appeared in Country A. There are hundreds of infected persons across dozens of affected communities in that country. In Country B, located at the other end of the world, several new cases of the disease surface in a single community. In this setting, we can imagine two sorts of Millean analyses. The first examines two similar communities within Country A, one of which has developed the disease and the other of which has not. This is the most‐similar style of case comparison, and focuses accordingly on the identification of a difference between the two cases that might account for variation across the sample. A second approach focuses on communities where the disease has appeared across the two countries and searches for any similarities that might account for these similar outcomes. This is the most‐different research design.

Both are plausible approaches to this particular problem, and we can imagine epidemiologists employing them simultaneously. However, the most‐different design demands stronger assumptions about the underlying factors at work. It supposes that the disease arises from the same cause in any setting. This is often a reasonable operating assumption when one is dealing with natural phenomena, though there are certainly many exceptions. Death, for example, has many causes. For this reason, it would not occur to us to look for most‐different cases of high mortality around the world. In order for the most‐different research design to effectively identify a causal factor at work in a given outcome, the researcher must assume that X   1 —the factor held constant across the diverse cases—is the only possible cause of Y (see Table 28.4 ). This assumption rarely holds in social‐scientific settings. Most outcomes of interest to anthropologists, economists, political scientists, and sociologists have multiple causes. There are many ways to win an election, to build a welfare state, to get into a war, to overthrow a government, or—returning to Marc Howard's work—to build a strong civil society. And it is for this reason that most‐different analysis is rarely applied in social science work and, where applied, is rarely convincing.

If this seems a tad severe, there is a more charitable way of approaching the most‐different method. Arguably, this is not a pure “method” at all but merely a supplement, a way of incorporating diversity in the sub‐sample of cases that provide the unusual outcome of interest. If the unusual outcome is revolutions, one might wish to encompass a wide variety of revolutions in one's analysis. If the unusual outcome is post‐Communist civil society, it seems appropriate to include a diverse set of post‐Communist polities in one's sample of case studies, as Marc Howard does. From this perspective, the most‐different method (so‐called) might be better labeled a diverse‐case method, as explored above.

10 Conclusions

In order to be a case of something broader than itself, the chosen case must be representative (in some respects) of a larger population. Otherwise—if it is purely idiosyncratic (“unique”)—it is uninformative about anything lying outside the borders of the case itself. A study based on a nonrepresentative sample has no (or very little) external validity. To be sure, no phenomenon is purely idiosyncratic; the notion of a unique case is a matter that would be difficult to define. One is concerned, as always, with matters of degree. Cases are more or less representative of some broader phenomenon and, on that score, may be considered better or worse subjects for intensive analysis. (The one exception, as noted, is the influential case.)

Of all the problems besetting case‐study analysis, perhaps the most persistent— and the most persistently bemoaned—is the problem of sample bias ( Achen and Snidal 1989 ; Collier and Mahoney 1996 ; Geddes 1990 ; King, Keohane, and Verba 1994 ; Rohlfing 2004 ; Sekhon 2004 ). Lisa Martin (1992 , 5) finds that the overemphasis of international relations scholars on a few well‐known cases of economic sanctions— most of which failed to elicit any change in the sanctioned country—“has distorted analysts view of the dynamics and characteristics of economic sanctions.” Barbara Geddes (1990) charges that many analyses of industrial policy have focused exclusively on the most successful cases—primarily the East Asian NICs—leading to biased inferences. Anna Breman and Carolyn Shelton (2001) show that case‐study work on the question of structural adjustment is systematically biased insofar as researchers tend to focus on disaster cases—those where structural adjustment is associated with very poor health and human development outcomes. These cases, often located in sub‐Saharan Africa, are by no means representative of the entire population. Consequently, scholarship on the question of structural adjustment is highly skewed in a particular ideological direction (against neoliberalism) (see also Gerring, Thacker, and Moreno 2005) .

These examples might be multiplied many times. Indeed, for many topics the most‐studied cases are acknowledged to be less than representative. It is worth reflecting upon the fact that our knowledge of the world is heavily colored by a few “big” (populous, rich, powerful) countries, and that a good portion of the disciplines of economics, political science, and sociology are built upon scholars' familiarity with the economics, political science, and sociology of one country, the United States. 23 Case‐study work is particularly prone to problems of investigator bias since so much rides on the researcher's selection of one (or a few) cases. Even if the investigator is unbiased, her sample may still be biased simply by virtue of “random” error (which may be understood as measurement error, error in the data‐generation process, or as an underlying causal feature of the universe).

There are only two situations in which a case‐study researcher need not be concerned with the representativeness of her chosen case. The first is the influential case research design, where a case is chosen because of its possible influence on a cross‐case model, and hence is not expected to be representative of a larger sample. The second is the deviant‐case method, where the chosen case is employed to confirm a broader cross‐case argument to which the case stands as an apparent exception. Yet even here the chosen case is expected to be representative of a broader set of cases—those, in particular, that are poorly explained by the extant model.

In all other circumstances, cases must be representative of the population of interest in whatever ways might be relevant to the proposition in question. Note that where a researcher is attempting to disconfirm a deterministic proposition the question of representativeness is perhaps more appropriately understood as a question of classification: Is the chosen case appropriately classified as a member of the designated population? If so, then it is fodder for a disconfirming case study.

If the researcher is attempting to confirm a deterministic proposition, or to make probabilistic arguments about a causal relationship, then the problem of representativeness is of the more usual sort: Is case A unit‐homogeneous relative to other cases in the population? This is not an easy matter to test. However, in a large‐ N context the residual for that case (in whatever model the researcher has greatest confidence in) is a reasonable place to start. Of course, this test is only as good as the model at hand. Any incorrect specifications or incorrect modeling procedures will likely bias the results and give an incorrect assessment of each case's “typicality.” In addition, there is the possibility of stochastic error, errors that cannot be modeled in a general framework. Given the explanatory weight that individual cases are asked to bear in a case‐study analysis, it is wise to consider more than just the residual test of representativeness. Deductive logic and an in‐depth knowledge of the case in question are often more reliable tools than the results of a cross‐case model.

In any case, there is no dispensing with the question. Case studies (with the two exceptions already noted) rest upon an assumed synecdoche: The case should stand for a population. If this is not true, or if there is reason to doubt this assumption, then the utility of the case study is brought severely into question.

Fortunately, there is some safety in numbers. Insofar as case‐study evidence is combined with cross‐case evidence the issue of sample bias is mitigated. Indeed, the suspicion of case‐study work that one finds in the social sciences today is, in my view, a product of a too‐literal interpretation of the case‐study method. A case study tout court is thought to mean a case study tout seul . Insofar as case studies and cross‐case studies can be enlisted within the same investigation (either in the same study or by reference to other studies in the same subfield), problems of representativeness are less worrisome. This is the virtue of cross‐level work, a.k.a. “triangulation.”

11 Ambiguities

Before concluding, I wish to draw attention to two ambiguities in case‐selection strategies in case‐study research. The first concerns the admixture of several case‐ selection strategies. The second concerns the changing status of a case as a study proceeds.

Some case studies follow only one strategy of case selection. They are typical , diverse , extreme , deviant , influential , crucial , pathway , most‐similar , or most‐different research designs, as discussed. However, many case studies mix and match among these case‐selection strategies. Indeed, insofar as all case studies seek representative samples, they are always in search of “typical” cases. Thus, it is common for writers to declare that their case is, for example, both extreme and typical; it has an extreme value on X   1 or Y but is not, in other respects, idiosyncratic. There is not much that one can say about these combinations of strategies except that, where the cases allow for a variety of empirical strategies, there is no reason not to pursue them. And where the same cases can serve several functions at once (without further effort on the researcher's part), there is little cost to a multi‐pronged approach to case analysis.

The second issue that deserves emphasis is the changing status of a case during the course of a researcher's investigation—which may last for years, if not decades. The problem is acute wherever a researcher begins in an exploratory mode and proceeds to hypothesis‐testing (that is, she develops a specific X   1 / Y proposition) or where the operative hypothesis or key control variable changes (a new causal factor is discovered or another outcome becomes the focus of analysis). Things change. And it is the mark of a good researcher to keep her mind open to new evidence and new insights. Too often, methodological discussions give the misleading impression that hypotheses are clear and remain fixed over the course of a study's development. Nothing could be further from the truth. The unofficial transcripts of academia— accessible in informal settings, where researchers let their guards down (particularly if inebriated)—are filled with stories about dead‐ends, unexpected findings, and drastically revised theory chapters. It would be interesting, in this vein, to compare published work with dissertation prospectuses and fellowship applications. I doubt if the correlation between these two stages of research is particularly strong.

Research, after all, is about discovery, not simply the verification or falsification of static hypotheses. That said, it is also true that research on a particular topic should move from hypothesis generating to hypothesis‐testing. This marks the progress of a field, and of a scholar's own work. As a rule, research that begins with an open‐ended ( X ‐ or Y ‐centered) analysis should conclude with a determinate X   1 / Y hypothesis.

The problem is that research strategies that are ideal for exploration are not always ideal for confirmation. The extreme‐case method is inherently exploratory since there is no clear causal hypothesis; the researcher is concerned merely to explore variation on a single dimension ( X or Y ). Other methods can be employed in either an open‐ ended (exploratory) or a hypothesis‐testing (confirmatory/disconfirmatory) mode. The difficulty is that once the researcher has arrived at a determinate hypothesis the originally chosen research design may no longer appear to be so well designed.

This is unfortunate, but inevitable. One cannot construct the perfect research design until (a) one has a specific hypothesis and (b) one is reasonably certain about what one is going to find “out there” in the empirical world. This is particularly true of observational research designs, but it also applies to many experimental research designs: Usually, there is a “good” (informative) finding, and a finding that is less insightful. In short, the perfect case‐study research design is usually apparent only ex post facto .

There are three ways to handle this. One can explain, straightforwardly, that the initial research was undertaken in an exploratory fashion, and therefore not constructed to test the specific hypothesis that is—now—the primary argument. Alternatively, one can try to redesign the study after the new (or revised) hypothesis has been formulated. This may require additional field research or perhaps the integration of additional cases or variables that can be obtained through secondary sources or through consultation of experts. A final approach is to simply jettison, or de‐emphasize, the portion of research that no longer addresses the (revised) key hypothesis. A three‐case study may become a two‐case study, and so forth. Lost time and effort are the costs of this downsizing.

In the event, practical considerations will probably determine which of these three strategies, or combinations of strategies, is to be followed. (They are not mutually exclusive.) The point to remember is that revision of one's cross‐case research design is normal and perhaps to be expected. Not all twists and turns on the meandering trail of truth can be anticipated.

12 Are There Other Methods of Case Selection?

At the outset of this chapter I summarized the task of case selection as a matter of achieving two objectives: representativeness (typicality) and variation (causal leverage). Evidently, there are other objectives as well. For example, one wishes to identify cases that are independent of each other. If chosen cases are affected by each other (sometimes known as Galton's problem or a problem of diffusion), this problem must be corrected before analysis can take place. I have neglected this issue because it is usually apparent to the researcher and, in any case, there are no simple techniques that might be utilized to correct for such biases. (For further discussion of this and other factors impinging upon case selection see Gerring 2001 , 178–81.)

I have also disregarded pragmatic/logistical issues that might affect case selection. Evidently, case selection is often influenced by a researcher's familiarity with the language of a country, a personal entrée into that locale, special access to important data, or funding that covers one archive rather than another. Pragmatic considerations are often—and quite rightly—decisive in the case‐selection process.

A final consideration concerns the theoretical prominence of a particular case within the literature on a subject. Researchers are sometimes obliged to study cases that have received extensive attention in previous studies. These are sometimes referred to as “paradigmatic” cases or “exemplars” ( Flyvbjerg 2004 , 427).

However, neither pragmatic/logistical utility nor theoretical prominence qualifies as a methodological factor in case selection. That is, these features of a case have no bearing on the validity of the findings stemming from a study. As such, it is appropriate to grant these issues a peripheral status in this chapter.

One final caveat must be issued. While it is traditional to distinguish among the tasks of case selection and case analysis, a close look at these processes shows them to be indistinct and overlapping. One cannot choose a case without considering the sort of analysis that it might be subjected to, and vice versa. Thus, the reader should consider choosing cases by employing the nine techniques laid out in this chapter along with any considerations that might be introduced by virtue of a case's quasi‐experimental qualities, a topic taken up elsewhere ( Gerring 2007 , ch. 6 ).

Abadie, A. , Drukker, D. , Herr, J. L. , and Imbens, G. W.   2001 . Implementing matching estimators for average treatment effects in Stata.   Stata Journal , 1: 1–18.

Google Scholar

Abbott, A.   2001 . Time Matters: On Theory and Method . Chicago: University of Chicago Press.

Google Preview

——  and Tsay, A.   2000 . Sequence analysis and optimal matching methods in sociology.   Sociological Methods and Research , 29: 3–33. 10.1177/0049124100029001001

——  and Forrest, J.   1986 . Optimal matching methods for historical sequences.   Journal of Interdisciplinary History , 16: 471–94. 10.2307/204500

Achen, C. H. , and Snidal, D.   1989 . Rational deterrence theory and comparative case studies.   World Politics , 41: 143–69. 10.2307/2010405

Allen, W. S.   1965 . The Nazi Seizure of Power: The Experience of a Single German Town, 1930–1935 . New York: Watts.

Almond, G. A.   1956 . Comparative political systems.   Journal of Politics , 18: 391–409.

Amenta, E.   1991 . Making the most of a case study: theories of the welfare state and the American experience. Pp. 172–94 in Issues and Alternatives in Comparative Social Research ed. C. C. Ragin . Leiden: E. J. Brill.

Barro, R. J.   1999 . Determinants of democracy.   Journal of Political Economy , 107: 158–83. 10.1086/250107

Belsey, D. A. , Kuh, E. , and Welsch, R. E.   2004 . Regression Diagnostics: Identifying Influential Data and Sources of Collinearity . New York: Wiley.

Bennett, A. , Lepgold, J. , and Unger, D.   1994 . Burden‐sharing in the Persian Gulf War.   International Organization , 48: 39–75. 10.1017/S0020818300000813

Bentley, A. 1908/ 1967 . The Process of Government . Cambridge, Mass.: Harvard University Press.

Brady, H. E. , and Collier, D. (eds.) 2004 . Rethinking Social Inquiry: Diverse Tools, Shared Standards . Lanham, Md.: Rowman and Littlefield.

Braumoeller, B. F.   2003 . Causal complexity and the study of politics.   Political Analysis , 11: 209–33. 10.1093/pan/mpg012

Breman, A. , and Shelton, C. 2001. Structural adjustment and health: a literature review of the debate, its role‐players and presented empirical evidence. CMH Working Paper Series, Paper No. WG6: 6. WHO, Commission on Macroeconomics and Health.

Brenner, R.   1976 . Agrarian class structure and economic development in pre‐industrial Europe.   Past and Present , 70: 30–75. 10.1093/past/70.1.30

Browne, A.   1987 . When Battered Women Kill . New York: Free Press.

Buchbinder, S. , and Vittinghoff, E.   1999 . HIV‐infected long‐term nonprogressors: epidemiology, mechanisms of delayed progression, and clinical and research implications.   Microbes Infect , 1: 1113–20. 10.1016/S1286-4579(99)00204-X

Cohen, M. R. , and Nagel, E.   1934 . An Introduction to Logic and Scientific Method . New York: Harcourt, Brace and Company.

Collier, D. , and Mahoney, J.   1996 . Insights and pitfalls: selection bias in qualitative research.   World Politics , 49: 56–91. 10.1353/wp.1996.0023

Collier, R. B. , and Collier, D. 1991/ 2002 . Shaping the Political Arena: Critical Junctures, the Labor Movement, and Regime Dynamics in Latin America . Notre Dame, Ind.: University of Notre Dame Press.

Colomer, J. M.   1991 . Transitions by agreement: modeling the Spanish way.   American Political Science Review , 85: 1283–302. 10.2307/1963946

Converse, P. E. , and Dupeux, G.   1962 . Politicization of the electorate in France and the United States.   Public Opinion Quarterly , 16: 1–23. 10.1086/267067

Coppedge, M. J. 2004. The conditional impact of the economy on democracy in Latin America. Presented at the conference “Democratic Advancements and Setbacks: What Have We Learnt?”, Uppsala University, June 11–13.

De Felice, E. G.   1986 . Causal inference and comparative methods.   Comparative Political Studies , 19: 415–37. 10.1177/0010414086019003005

Desch, M. C.   2002 . Democracy and victory: why regime type hardly matters.   International Security , 27: 5–47. 10.1162/016228802760987815

Deyo, F. (ed.) 1987 . The Political Economy of the New Asian Industrialism . Ithaca, NY: Cornell University Press.

Dion, D.   1998 . Evidence and inference in the comparative case study.   Comparative Politics , 30: 127–45. 10.2307/422284

Eckstein, H.   1975 . Case studies and theory in political science. In Handbook of Political Science , vii: Political Science: Scope and Theory , ed. F. I. Greenstein and N. W. Polsby . Reading, Mass.: Addison‐Wesley.

Eggan, F.   1954 . Social anthropology and the method of controlled comparison.   American Anthropologist , 56: 743–63. 10.1525/aa.1954.56.5.02a00020

Elman, C.   2003 . Lessons from Lakatos. In Progress in International Relations Theory: Appraising the Field , ed. C. Elman and M. F. Elman . Cambridge, Mass.: MIT Press.

——  2005 . Explanatory typologies in qualitative studies of international politics.   International Organization , 59: 293–326.

Emigh, R.   1997 . The power of negative thinking: the use of negative case methodology in the development of sociological theory.   Theory and Society , 26: 649–84. 10.1023/A:1006896217647

Epstein, L. D.   1964 . A comparative study of Canadian parties.   American Political Science Review , 58: 46–59. 10.2307/1952754

Ertman, T.   1997 . Birth of the Leviathan: Building States and Regimes in Medieval and Early Modern Europe . Cambridge: Cambridge University Press.

Esping‐Andersen, G.   1990 . The Three Worlds of Welfare Capitalism . Princeton, NJ: Princeton University Press.

Flyvbjerg, B.   2004 . Five misunderstandings about case‐study research. Pp. 420–34 in Qualitative Research Practice , ed. C. Seale , G. Gobo , J. F. Gubrium , and D. Silverman . London: Sage.

Geddes, B.   1990 . How the cases you choose affect the answers you get: selection bias in comparative politics. In Political Analysis , vol. ii, ed. J. A. Stimson . Ann Arbor: University of Michigan Press.

——  2003 . Paradigms and Sand Castles: Theory Building and Research Design in Comparative Politics . Ann Arbor: University of Michigan Press.

George, A. L. , and Bennett, A.   2005 . Case Studies and Theory Development . Cambridge, Mass.: MIT Press.

——  and Smoke, R.   1974 . Deterrence in American Foreign Policy: Theory and Practice . New York: Columbia University Press.

Gerring, J.   2001 . Social Science Methodology: A Criterial Framework . Cambridge: Cambridge University Press.

——  2007 . Case Study Research: Principles and Practices . Cambridge: Cambridge University Press.

——  Thacker, S. and Moreno, C. 2005. Do neoliberal policies save lives? Unpublished manuscript.

Goertz, G. and Starr, H. (eds.) 2003 . Necessary Conditions: Theory, Methodology and Applications . New York: Rowman and Littlefield.

——  and Levy, J. (eds.) forthcoming. Causal explanations, necessary conditions, and case studies: World War I and the end of the Cold War. Manuscript.

Goodin, R. E. and Smitsman, A.   2000 . Placing welfare states: the Netherlands as a crucial test case.   Journal of Comparative Policy Analysis , 2: 39–64. 10.1080/13876980008412635

Gujarati, D. N.   2003 . Basic Econometrics , 4th edn. New York: McGraw‐Hill.

Hamilton, G. G.   1977 . Chinese consumption of foreign commodities: a comparative perspective.   American Sociological Review , 42: 877–91. 10.2307/2094574

Haynes, B. F.   Pantaleo, G. and Fauci, A. S.   1996 . Toward an understanding of the correlates of protective immunity to HIV infection.   Science , 271: 324–8. 10.1126/science.271.5247.324

Hempel, C. G.   1942 . The function of general laws in history.   Journal of Philosophy , 39: 35–48. 10.2307/2017635

Ho, D. E.   Imai, K.   King, G. and Stuart, E. A. 2004. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Manuscript.

Howard, M. M.   2003 . The Weakness of Civil Society in Post‐Communist Europe . Cambridge: Cambridge University Press.

Howson, C. and Urbach, P.   1989 . Scientific Reasoning: The Bayesian Approach . La Salle, Ill.: Open Court.

Humphreys, M.   2005 . Natural resources, conflict, and conflict resolution: uncovering the mechanisms.   Journal of Conflict Resolution , 49: 508–37. 10.1177/0022002705277545

Jenicek, M.   2001 . Clinical Case Reporting in Evidence‐Based Medicine , 2nd edn. Oxford: Oxford University Press.

Karl, T. L.   1997 . The Paradox of Plenty: Oil Booms and Petro‐states . Berkeley: University of California Press.

Kazancigil, A.   1994 . The deviant case in comparative analysis: high stateness in comparative analysis. Pp. 213–38 in Comparing Nations: Concepts, Strategies, Substance , ed. M. Dogan and A. Kazancigil . Cambridge: Blackwell.

Kemp, K. A.   1986 . Race, ethnicity, class and urban spatial conflict: Chicago as a crucial case   Urban Studies , 23: 197–208. 10.1080/00420988620080231

Kendall, P. L. and Wolf, K. M. 1949/ 1955 . The analysis of deviant cases in communications research. In Communications Research, 1948–1949 , ed. P. F. Lazarsfeld and F. N. Stanton. New York: Harper and Brothers. Reprinted as pp. 167–70 in The Language of Social Research , ed. P. F. Lazarsfeld and M. Rosenberg . New York: Free Press.

Kennedy, C. H.   2005 . Single‐case Designs for Educational Research . Boston: Allyn and Bacon.

Kennedy, P.   2003 . A Guide to Econometrics , 5th edn. Cambridge, Mass.: MIT Press.

Khong, Y. F.   1992 . Analogies at War: Korea, Munich, Dien Bien Phu, and the Vietnam Decisions of 1965 . Princeton, NJ: Princeton University Press.

King, G.   Keohane, R. O. and Verba, S.   1994 . Designing Social Inquiry: Scientific Inference in Qualitative Research . Princeton, NJ: Princeton University Press.

Lakatos, I.   1978 . The Methodology of Scientific Research Programmes . Cambridge: Cambridge University Press.

Lazarsfeld, P. F. and Barton, A. H.   1951 . Qualitative measurement in the social sciences: classification, typologies, and indices. In The Policy Sciences , ed. D. Lerner and H. D. Lass‐ well. Stanford, Calif.: Stanford University Press.

Levy, J. S.   2002 . Qualitative methods in international relations. In Evaluating Methodology in International Studies , ed. F. P. Harvey and M. Brecher. Ann Arbor: University of Michigan Press.

Lijphart, A.   1968 . The Politics of Accommodation: Pluralism and Democracy in the Netherlands . Berkeley: University of California Press.

——  1969 . Consociational democracy.   World Politics , 21: 207–25. 10.2307/2009820

——  1971 . Comparative politics and the comparative method. American Political Science Review , 65: 682–93.

——  1975 . The comparable cases strategy in comparative research.   Comparative Political Studies , 8: 158–77.

Lipset, S. M.   1959 . Some social requisites of democracy: economic development and political development.   American Political Science Review , 53: 69–105. 10.2307/1951731

——  1960/ 1963 . Political Man: The Social Bases of Politics . Garden City, NY: Anchor.

——  1968 . Agrarian Socialism: The Cooperative Commonwealth Federation in Saskatchewan. A Study in Political Sociology . Garden City, NY: Doubleday.

——  Trow, M. A. and Coleman, J. S.   1956 . Union Democracy: The Internal Politics of the International Typographical Union . New York: Free Press.

Lynd, R. S. and Lynd, H. M. 1929/ 1956 . Middletown: A Study in American Culture . New York: Harcourt, Brace.

Mahoney, J. and Goertz, G.   2004 . The possibility principle: choosing negative cases in comparative research.   American Political Science Review , 98: 653–69.

Martin, L. L.   1992 . Coercive Cooperation: Explaining Multilateral Economic Sanctions .Princeton, NJ: Princeton University Press.

Mayo, D. G.   1996 . Error and the Growth of Experimental Knowledge . Chicago: University of Chicago Press.

Meckstroth, T.   1975 . “Most different systems” and “most similar systems:” a study in the logic of comparative inquiry.   Comparative Political Studies , 8: 133–77.

Miguel, E.   2004 . Tribe or nation: nation‐building and public goods in Kenya versus Tanzania.   World Politics , 56: 327–62. 10.1353/wp.2004.0018

Mill, J. S. 1843/ 1872 . The System of Logic , 8th edn. London: Longmans, Green.

Monroe, K. R.   1996 . The Heart of Altruism: Perceptions of a Common Humanity . Princeton, NJ: Princeton University Press.

Moore, B., Jr.   1966 . Social Origins of Dictatorship and Democracy: Lord and Peasant in the Making of the Modern World . Boston: Beacon Press.

Morgan, S. L. and Harding, D. J. 2005. Matching estimators of causal effects: from stratification and weighting to practical data analysis routines. Manuscript.

Moulder, F. V.   1977 . Japan, China and the Modern World Economy: Toward a Reinterpretation of East Asian Development ca. 1600 to ca. 1918 . Cambridge: Cambridge University Press.

Munck, G. L.   2004 . Tools for qualitative research. Pp. 105–21 in Rethinking Social Inquiry: Diverse Tools, Shared Standards , ed. H. E. Brady and D. Collier . Lanham, Md. : Rowman and Littlefield.

Njolstad, O.   1990 . Learning from history? Case studies and the limits to theory‐building. Pp. 220–46 in Arms Races: Technological and Political Dynamics , ed. O. Njolstad . Thousand Oaks, Calif.: Sage.

Patton, M. Q.   2002 . Qualitative Evaluation and Research Methods . Newbury Park, Calif.: Sage.

Popper, K. 1934/ 1968 . The Logic of Scientific Discovery . New York: Harper and Row.

——  1963 . Conjectures and Refutations . London: Routledge and Kegan Paul.

Posner, D.   2004 . The political salience of cultural difference: why Chewas and Tumbukas are allies in Zambia and adversaries in Malawi.   American Political Science Review , 98: 529–46.

Przeworski, A. and Teune, H.   1970 . The Logic of Comparative Social Inquiry . New York: John Wiley.

Queen, S.   1928 . Round table on the case study in sociological research.   Publications of the American Sociological Society, Papers and Proceedings , 22: 225–7.

Ragin, C. C.   2000 . Fuzzy‐set Social Science . Chicago: University of Chicago Press.

——  2004 . Turning the tables. Pp. 123–38 in Rethinking Social Inquiry: Diverse Tools, Shared Standards , ed. H. E. Brady and D. Collier.   Lanham, Md. : Rowman and Littlefield.

Reilly, B.   2000 –1. Democracy, ethnic fragmentation, and internal conflict: confused theories, faulty data, and the “crucial case” of Papua New Guinea.   International Security , 25: 162–85. 10.1162/016228800560552

——  and Phillpot, R.   2003 . “Making democracy work” in Papua New Guinea: social capital and provincial development in an ethnically fragmented society.   Asian Survey , 42: 906–27. 10.1525/as.2002.42.6.906

Rogowski, R.   1995 . The role of theory and anomaly in social‐scientific inference.   American Political Science Review , 89: 467–70. 10.2307/2082443

Rohlfing, I. 2004. Have you chosen the right case? Uncertainty in case selection for single case studies. Working Paper, International University, Bremen.

Rosenbaum, P. R.   2004 . Matching in observational studies. In Applied Bayesian Modeling and Causal Inference from an Incomplete‐data Perspective , ed. A. Gelman and X.‐L. Meng . New York: John Wiley.

——  and Silber, J. H.   2001 . Matching and thick description in an observational study of mortality after surgery.   Biostatistics , 2: 217–32. 10.1093/biostatistics/2.2.217

Ross, M.   2001 . Does oil hinder democracy?   World Politics , 53: 325–61. 10.1353/wp.2001.0011

Sagan, S. D.   1995 . Limits of Safety: Organizations, Accidents, and Nuclear Weapons . Princeton, NJ: Princeton University Press.

Sekhon, J. S.   2004 . Quality meets quantity: case studies, conditional probability and counter‐ factuals.   Perspectives in Politics , 2: 281–93.

Shafer, M. D.   1988 . Deadly Paradigms: The Failure of U.S. Counterinsurgency Policy . Princeton, NJ: Princeton University Press.

Skocpol, T.   1979 . States and Social Revolutions: A Comparative Analysis of France, Russia, and China . Cambridge: Cambridge University Press.

——  and Somers, M.   1980 . The uses of comparative history in macrosocial inquiry.   Comparative Studies in Society and History , 22: 147–97.

Stinchcombe, A. L.   1968 . Constructing Social Theories . New York: Harcourt, Brace.

Swank, D. H.   2002 . Global Capital, Political Institutions, and Policy Change in Developed Welfare States . Cambridge: Cambridge University Press.

Tendler, J.   1997 . Good Government in the Tropics . Baltimore: Johns Hopkins University Press.

Truman, D. B.   1951 . The Governmental Process . New York: Alfred A. Knopf.

Tsai, L.   2007 . Accountability without Democracy: How Solidary Groups Provide Public Goods in Rural China . Cambridge: Cambridge University Press.

Van Evera, S.   1997 . Guide to Methods for Students of Political Science . Ithaca, NY: Cornell University Press.

Wahlke, J. C.   1979 . Pre‐behavioralism in political science. American Political Science Review , 73: 9–31. 10.2307/1954728

Yashar, D. J.   2005 . Contesting Citizenship in Latin America: The Rise of Indigenous Movements and the Postliberal Challenge . Cambridge: Cambridge University Press.

Yin, R. K.   2004 . Case Study Anthology . Thousand Oaks, Calif.: Sage.

Gujarati (2003) ; Kennedy (2003) . Interestingly, the potential of cross‐case statistics in helping to choose cases for in‐depth analysis is recognized in some of the earliest discussions of the case‐study method (e.g. Queen 1928 , 226).

This expands on Mill (1843/1872 , 253), who wrote of scientific enquiry as twofold: “either inquiries into the cause of a given effect or into the effects or properties of a given cause.”

This method has not received much attention on the part of qualitative methodologists; hence, the absence of a generally recognized name. It bears some resemblance to J. S. Mill's Joint Method of Agreement and Difference ( Mill 1843/1872 ), which is to say a mixture of most‐similar and most‐different analysis, as discussed below. Patton (2002 , 234) employs the concept of “maximum variation (heterogeneity) sampling.”

More precisely, George and Smoke (1974 , 534, 522–36, ch. 18 ; see also discussion in Collier and Mahoney 1996 , 78) set out to investigate causal pathways and discovered, through the course of their investigation of many cases, these three causal types. Yet, for our purposes what is important is that the final sample includes at least one representative of each “type.”

For further examples see Collier and Mahoney (1996) ; Geddes (1990) ; Tendler (1997) .

Traditionally, methodologists have conceptualized cases as having “positive” or “negative” values (e.g. Emigh 1997 ; Mahoney and Goertz 2004 ; Ragin 2000 , 60; 2004 , 126).

Geddes (1990) ; King, Keohane, and Verba (1994) . See also discussion in Brady and Collier (2004) ; Collier and Mahoney (1996) ; Rogowski (1995) .

The exception would be a circumstance in which the researcher intends to disprove a deterministic argument ( Dion 1998 ).

Geddes (2003 , 131). For other examples of casework from the annals of medicine see “Clinical reports” in the Lancet , “Case studies” in Canadian Medical Association Journal , and various issues of the Journal of Obstetrics and Gynecology , often devoted to clinical cases (discussed in Jenicek 2001 , 7). For examples from the subfield of comparative politics see Kazancigil (1994) .

For a discussion of the important role of anomalies in the development of scientific theorizing see Elman (2003) ; Lakatos (1978) . For examples of deviant‐case research designs in the social sciences see Amenta (1991) ; Coppedge (2004) ; Eckstein (1975) ; Emigh (1997) ; Kendall and Wolf (1949/1955) .

For examples of the crucial‐case method see Bennett, Lepgold, and Unger (1994) ; Desch (2002) ; Goodin and Smitsman (2000) ; Kemp (1986) ; Reilly and Phillpot (2003) . For general discussion see George and Bennett (2005) ; Levy (2002) ; Stinchcombe (1968 , 24–8).

A third position, which purports to be neither Popperian or Bayesian, has been articulated by Mayo (1996 , ch. 6 ). From this perspective, the same idea is articulated as a matter of “severe tests.”

It should be noted that Tsai's conclusions do not rest solely on this crucial case. Indeed, she employs a broad range of methodological tools, encompassing case‐study and cross‐case methods.

See also the discussion in Eckstein (1975) and Lijphart (1969) . For additional examples of case studies disconfirming general propositions of a deterministic nature see Allen (1965); Lipset, Trow, and Coleman (1956) ; Njolstad (1990) ; Reilly (2000–1) ; and discussion in Dion (1998) ; Rogowski (1995) .

Granted, insofar as case‐study analysis provides a window into causal mechanisms, and causal mechanisms are integral to a given theory, a single case may be enlisted to confirm or disconfirm a proposition. However, if the case study upholds a posited pattern of X/Y covariation, and finds fault only with the stipulated causal mechanism, it would be more accurate to say that the study forces the reformulation of a given theory, rather than its confirmation or disconfirmation. See further discussion in the following section.

Sometimes, the most‐similar method is known as the “method of difference,” after its inventor ( Mill 1843/1872 ). For later treatments see Cohen and Nagel (1934) ; Eggan (1954) ; Gerring (2001 , ch. 9 ); Lijphart (1971 ; 1975) ; Meckstroth (1975) ; Przeworski and Teune (1970) ; Skocpol and Somers (1980) .

For good introductions see Ho et al. (2004) ; Morgan and Harding (2005) ; Rosenbaum (2004) ; Rosenbaum and Silber (2001) . For a discussion of matching procedures in Stata see Abadie et al. (2001) .

The most‐different method is also sometimes referred to as the “method of agreement,” following its inventor, J. S. Mill (1843/1872) . See also De Felice (1986) ; Gerring (2001 , 212–14); Lijphart (1971 ; 1975) ; Meckstroth (1975) ; Przeworski and Teune (1970) ; Skocpol and Somers (1980) . For examples of this method see Collier and Collier (1991/2002) ; Converse and Dupeux (1962) ; Karl (1997) ; Moore (1966) ; Skocpol (1979) ; Yashar (2005 , 23). However, most of these studies are described as combining most‐similar and most‐different methods.

In the following discussion I treat the terms social capital, civil society, and civic engagement interchangeably.

E.g. Collier and Collier (1991/2002) ; Karl (1997) ; Moore (1966) ; Skocpol (1979) ; Yashar (2005 , 23). Karl (1997) , which affects to be a most‐different system analysis (20), is a particularly clear example of this. Her study, focused ostensibly on petro‐states (states with large oil reserves), makes two sorts of inferences. The first concerns the (usually) obstructive role of oil in political and economic development. The second sort of inference concerns variation within the population of petro‐states, showing that some countries (e.g. Norway, Indonesia) manage to avoid the pathologies brought on elsewhere by oil resources. When attempting to explain the constraining role of oil on petro‐states, Karl usually relies on contrasts between petro‐states and nonpetro‐states (e.g. ch. 10 ). Only when attempting to explain differences among petro‐states does she restrict her sample to petro‐states. In my opinion, very little use is made of the most‐different research design.

This was recognized, at least implicitly, by Mill (1843/1872 , 258–9). Skepticism has been echoed by methodologists in the intervening years (e.g. Cohen and Nagel 1934 , 251–6; Gerring 2001 ; Skocpol and Somers 1980 ). Indeed, explicit defenses of the most‐different method are rare (but see De Felice 1986 ).

Another way of stating this is to say that X is a “nontrivial necessary condition” of Y .

Wahlke (1979 , 13) writes of the failings of the “behavioralist” mode of political science analysis: “It rarely aims at generalization; research efforts have been confined essentially to case studies of single political systems, most of them dealing …with the American system.”

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options

• Related Documents

Case Selection for Case‐Study Analysis: Qualitative and Quantitative Techniques

This article presents some guidance by cataloging nine different techniques for case selection: typical, diverse, extreme, deviant, influential, crucial, pathway, most similar, and most different. It also indicates that if the researcher is starting from a quantitative database, then methods for finding influential outliers can be used. In particular, the article clarifies the general principles that might guide the process of case selection in case-study research. Cases are more or less representative of some broader phenomenon and, on that score, may be considered better or worse subjects for intensive analysis. The article then draws attention to two ambiguities in case-selection strategies in case-study research. The first concerns the admixture of several case-selection strategies. The second concerns the changing status of a case as a study proceeds. Some case studies follow only one strategy of case selection.

Case Selection and Hypothesis Testing

This chapter discusses quantitative and qualitative practices of case-study selection when the goal of the analysis is to evaluate causal hypotheses. More specifically, it considers how the different causal models used in the qualitative and quantitative research cultures shape the kind of cases that provide the most leverage for hypothesis testing. The chapter examines whether one should select cases based on their value on the dependent variable. It also evaluates the kinds of cases that provide the most leverage for causal inference when conducting case-study research. It shows that differences in research goals between quantitative and qualitative scholars yield distinct ideas about best strategies of case selection. Qualitative research places emphasis on explaining particular cases; quantitative research does not.

Paying Attention to the Trees in the Forest, or a Call to Examine Agency-Specific Stories

Public administration scholarship needs to strike a better balance between large sample studies and in-depth case studies. The availability of large data sets has led us to engage in empirical research that is broad in scope but is frequently devoid of rich context. In-depth case studies can help to explain why we observe particular relationships and can help us to clarify gaps and inconsistencies in theory. Our argument for more case studies aims to encourage researchers to bridge insights from qualitative and quantitative research through triangulation. We describe the value of case study research, and qualitative and quantitative design options. We then propose opportunities for case study research in public personnel scholarship on patronage pressures, performance management, and diversity management.

Case Selection Techniques in Case Study Research

Blending the focused ethnographic method and case study research: implications regarding case selection and generalization of results.

In this article, we present the benefits of blending the methodological characteristics of the focused ethnographic method (FEM) and case study research (CSR) for a study on auxiliary work processes in a hospital in Barcelona, Spain. We argue that incorporating CSR logic and principles in the FEM produces a better form of inquiry, as this combination improves the quality of data and the ability to make comparisons in addition to enhancing the transferability of findings. These better outcomes are achieved because the characteristics of the case study methodological design reinforce many of the strengths of the FEM, such as its thematic concreteness and its targeted data collection. The role played by the theoretical framework of the study thus makes it easier to focus on the key variables of the initial theoretical model and to introduce a logic of multiple case comparison.

Qualitative Case Study Research Design

Qualitative case study research can be a valuable tool for answering complex, real-world questions. This method is often misunderstood or neglected due to a lack of understanding by researchers and reviewers. This tutorial defines the characteristics of qualitative case study research and its application to a broader understanding of stuttering that cannot be defined through other methodologies. This article will describe ways that data can be collected and analyzed.

Intersecting Mixed Methods and Case Study Research: Design Possibilities and Challenges

Robert k. yin. (2014). case study research design and methods (5th ed.). thousand oaks, ca: sage. 282 pages., cooperative banks & local government units - areas of collaboration (poland case study - research results), from the ‘frying-pan’ into the ‘fire’ antidotes to confirmatory bias in case-study research, export citation format, share document.

Case-Control Studies

•   1  
• |   2  
• |   3  
• |   4  
• |   5  
• |   6  
• |   7  
• |   8  

Selecting & Defining Cases and Controls

The "case" definition, sources of cases, selection of the controls.

E pi_Tools.XLSX

All Modules

Careful thought should be given to the case definition to be used. If the definition is too broad or vague, it is easier to capture people with the outcome of interest, but a loose case definition will also capture people who do not have the disease. On the other hand, an overly restrictive case definition is employed, fewer cases will be captured, and the sample size may be limited. Investigators frequently wrestle with this problem during outbreak investigations. Initially, they will often use a somewhat broad definition in order to identify potential cases. However, as an outbreak investigation progresses, there is a tendency to narrow the case definition to make it more precise and specific, for example by requiring confirmation of the diagnosis by laboratory testing. In general, investigators conducting case-control studies should thoughtfully construct a definition that is as clear and specific as possible without being overly restrictive.

Investigators studying chronic diseases generally prefer newly diagnosed cases, because they tend to be more motivated to participate, may remember relevant exposures more accurately, and because it avoids complicating factors related to selection of longer duration (i.e., prevalent) cases. However, it is sometimes impossible to have an adequate sample size if only recent cases are enrolled.

Typical sources for cases include:

• Patient rosters at medical facilities
• Death certificates
• Disease registries (e.g., cancer or birth defect registries; the SEER Program [Surveillance, Epidemiology and End Results] is a federally funded program that identifies newly diagnosed cases of cancer in population-based registries across the US )
• Cross-sectional surveys (e.g., NHANES, the National Health and Nutrition Examination Survey)

As noted above, it is always useful to think of a case-control study as being nested within some sort of a cohort, i.e., a source population that produced the cases that were identified and enrolled. In view of this there are two key principles that should be followed in selecting controls:

• The comparison group ("controls") should be representative of the source population that produced the cases.
• The "controls" must be sampled in a way that is independent of the exposure, meaning that their selection should not be more (or less) likely if they have the exposure of interest.

If either of these principles are not adhered to, selection bias can result (as discussed in detail in the module on Bias ).

Note that in the earlier example of a case-control study conducted in the Massachusetts population, we specified that our sampling method was random so that exposed and unexposed members of the population had an equal chance of being selected. Therefore, we would expect that about 1,000 would be exposed and 5,000 unexposed (the same ratio as in the whole population), and came up with an odds ratio that was same as the hypothetical risk ratio we would have had if we had collected exposure information from the whole population of six million:

What if we had instead been more likely to sample those who were exposed, so that we instead found 1,500 exposed and 4,500 unexposed among the 6,000 controls?   Then the odds ratio would have been:

This odds ratio is biased because it differs from the true odds ratio.   In this case, the bias stemmed from the fact that we violated the second principle in selection of controls. Depending on which category is over or under-sampled, this type of bias can result in either an underestimate or an overestimate of the true association.

A hypothetical case-control study was conducted to determine whether lower socioeconomic status (the exposure) is associated with a higher risk of cervical cancer (the outcome). The "cases" consisted of 250 women with cervical cancer who were referred to Massachusetts General Hospital for treatment for cervical cancer. They were referred from all over the state. The cases were asked a series of questions relating to socioeconomic status (household income, employment, education, etc.). The investigators identified control subjects by going door-to-door in the community around MGH from 9:00 AM to 5:00  PM. Many residents are not home, but they persist and eventually enroll enough controls. The problem is that the controls were selected by a different mechanism than the cases, AND the selection mechanism may have tended to select individuals of different socioeconomic status, since women who were at home may have been somewhat more likely to be unemployed. In other words, the controls were more likely to be enrolled (selected) if they had the exposure of interest (lower socioeconomic status). 

What is the purpose of the control group in a case-control study?

a.  To provide information on the disease distribution in the population that gave rise to the cases.

b.  To provide information on the exposure distribution in the population that gave rise to the cases.

return to top | previous page | next page

Content ©2016. All Rights Reserved. Date last modified: June 7, 2016. Wayne W. LaMorte, MD, PhD, MPH

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons

Margin Size

• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

2.3: Case Selection (Or, How to Use Cases in Your Comparative Analysis)

• Last updated
• Save as PDF
• Page ID 135832

• Dino Bozonelos, Julia Wendt, Charlotte Lee, Jessica Scarffe, Masahiro Omae, Josh Franco, Byran Martin, & Stefan Veldhuis
• Victor Valley College, Berkeley City College, Allan Hancock College, San Diego City College, Cuyamaca College, Houston Community College, and Long Beach City College via ASCCC Open Educational Resources Initiative (OERI)

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

\( \newcommand{\Span}{\mathrm{span}}\)

\( \newcommand{\id}{\mathrm{id}}\)

\( \newcommand{\kernel}{\mathrm{null}\,}\)

\( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\)

\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\)

\( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

\( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vectorC}[1]{\textbf{#1}} \)

\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

Learning Objectives

By the end of this section, you will be able to:

• Discuss the importance of case selection in case studies.
• Consider the implications of poor case selection.

Introduction

Case selection is an important part of any research design. Deciding how many cases, and which cases, to include, will clearly help determine the outcome of our results. If we decide to select a high number of cases, we often say that we are conducting large-N research. Large-N research is when the number of observations or cases is large enough where we would need mathematical, usually statistical, techniques to discover and interpret any correlations or causations. In order for a large-N analysis to yield any relevant findings, a number of conventions need to be observed. First, the sample needs to be representative of the studied population. Thus, if we wanted to understand the long-term effects of COVID, we would need to know the approximate details of those who contracted the virus. Once we know the parameters of the population, we can then determine a sample that represents the larger population. For example, women make up 55% of all long-term COVID survivors. Thus, any sample we generate needs to be at least 55% women.

Second, some kind of randomization technique needs to be involved in large-N research. So not only must your sample be representative, it must also randomly select people within that sample. In other words, we must have a large selection of people that fit within the population criteria, and then randomly select from those pools. Randomization would help to reduce bias in the study. Also, when cases (people with long-term COVID) are randomly chosen they tend to ensure a fairer representation of the studied population. Third, your sample needs to be large enough, hence the large-N designation for any conclusions to have any external validity. Generally speaking, the larger the number of observations/cases in the sample, the more validity we can have in the study. There is no magic number, but if using the above example, our sample of long-term COVID patients should be at least over 750 people, with an aim of around 1,200 to 1,500 people.

When it comes to comparative politics, we rarely ever reach the numbers typically used in large-N research. There are about 200 fully recognized countries, with about a dozen partially recognized countries, and even fewer areas or regions of study, such as Europe or Latin America. Given this, what is the strategy when one case, or a few cases, are being studied? What happens if we are only wanting to know the COVID-19 response in the United States, and not the rest of the world? How do we randomize this to ensure our results are not biased or are representative? These and other questions are legitimate issues that many comparativist scholars face when completing research. Does randomization work with case studies? Gerring suggests that it does not, as “any given sample may be widely representative” (pg. 87). Thus, random sampling is not a reliable approach when it comes to case studies. And even if the randomized sample is representative, there is no guarantee that the gathered evidence would be reliable.

One can make the argument that case selection may not be as important in large-N studies as they are in small-N studies. In large-N research, potential errors and/or biases may be ameliorated, especially if the sample is large enough. This is not always what happens, errors and biases most certainly can exist in large-N research. However, incorrect or biased inferences are less of a worry when we have 1,500 cases versus 15 cases. In small-N research, case selection simply matters much more.

This is why Blatter and Haverland (2012) write that, “case studies are ‘case-centered’, whereas large-N studies are ‘variable-centered’". In large-N studies we are more concerned with the conceptualization and operationalization of variables. Thus, we want to focus on which data to include in the analysis of long-term COVID patients. If we wanted to survey them, we would want to make sure we construct questions in appropriate ways. For almost all survey-based large-N research, the question responses themselves become the coded variables used in the statistical analysis.

Case selection can be driven by a number of factors in comparative politics, with the first two approaches being the more traditional. First, it can derive from the interests of the researcher(s). For example, if the researcher lives in Germany, they may want to research the spread of COVID-19 within the country, possibly using a subnational approach where the researcher may compare infection rates among German states. Second, case selection may be driven by area studies. This is still based on the interests of the researcher as generally speaking scholars pick areas of studies due to their personal interests. For example, the same researcher may research COVID-19 infection rates among European Union member-states. Finally, the selection of cases selected may be driven by the type of case study that is utilized. In this approach, cases are selected as they allow researchers to compare their similarities or their differences. Or, a case might be selected that is typical of most cases, or in contrast, a case or cases that deviate from the norm. We discuss types of case studies and their impact on case selection below.

Types of Case Studies: Descriptive vs. Causal

There are a number of different ways to categorize case studies. One of the most recent ways is through John Gerring. He wrote two editions on case study research (2017) where he posits that the central question posed by the researcher will dictate the aim of the case study. Is the study meant to be descriptive? If so, what is the researcher looking to describe? How many cases (countries, incidents, events) are there? Or is the study meant to be causal, where the researcher is looking for a cause and effect? Given this, Gerring categorizes case studies into two types: descriptive and causal.

Descriptive case studies are “not organized around a central, overarching causal hypothesis or theory” (pg. 56). Most case studies are descriptive in nature, where the researchers simply seek to describe what they observe. They are useful for transmitting information regarding the studied political phenomenon. For a descriptive case study, a scholar might choose a case that is considered typical of the population. An example could involve researching the effects of the pandemic on medium-sized cities in the US. This city would have to exhibit the tendencies of medium-sized cities throughout the entire country. First, we would have to conceptualize what we mean by ‘a medium-size city’. Second, we would then have to establish the characteristics of medium-sized US cities, so that our case selection is appropriate. Alternatively, cases could be chosen for their diversity . In keeping with our example, maybe we want to look at the effects of the pandemic on a range of US cities, from small, rural towns, to medium-sized suburban cities to large-sized urban areas.

Causal case studies are “organized around a central hypothesis about how X affects Y” (pg. 63). In causal case studies, the context around a specific political phenomenon or phenomena is important as it allows for researchers to identify the aspects that set up the conditions, the mechanisms, for that outcome to occur. Scholars refer to this as the causal mechanism , which is defined by Falleti & Lynch (2009) as “portable concepts that explain how and why a hypothesized cause, in a given context, contributes to a particular outcome”. Remember, causality is when a change in one variable verifiably causes an effect or change in another variable. For causal case studies that employ causal mechanisms, Gerring divides them into exploratory case-selection, estimating case-selection, and diagnostic case-selection. The differences revolve around how the central hypothesis is utilized in the study.

Exploratory case studies are used to identify a potential causal hypothesis. Researchers will single out the independent variables that seem to affect the outcome, or dependent variable, the most. The goal is to build up to what the causal mechanism might be by providing the context. This is also referred to as hypothesis generating as opposed to hypothesis testing. Case selection can vary widely depending on the goal of the researcher. For example, if the scholar is looking to develop an ‘ideal-type’, they might seek out an extreme case. An ideal-type is defined as a “conception or a standard of something in its highest perfection” (New Webster Dictionary). Thus, if we want to understand the ideal-type capitalist system, we want to investigate a country that practices a pure or ‘extreme’ form of the economic system.

Estimating case studies start with a hypothesis already in place. The goal is to test the hypothesis through collected data/evidence. Researchers seek to estimate the ‘causal effect’. This involves determining if the relationship between the independent and dependent variables is positive, negative, or ultimately if no relationship exists at all. Finally, diagnostic case studies are important as they help to “confirm, disconfirm, or refine a hypothesis” (Gerring 2017). Case selection can also vary in diagnostic case studies. For example, scholars can choose an least-likely case, or a case where the hypothesis is confirmed even though the context would suggest otherwise. A good example would be looking at Indian democracy, which has existed for over 70 years. India has a high level of ethnolinguistic diversity, is relatively underdeveloped economically, and a low level of modernization through large swaths of the country. All of these factors strongly suggest that India should not have democratized, or should have failed to stay a democracy in the long-term, or have disintegrated as a country.

Most Similar/Most Different Systems Approach

The discussion in the previous subsection tends to focus on case selection when it comes to a single case. Single case studies are valuable as they provide an opportunity for in-depth research on a topic that requires it. However, in comparative politics, our approach is to compare. Given this, we are required to select more than one case. This presents a different set of challenges. First, how many cases do we pick? This is a tricky question we addressed earlier. Second, how do we apply the previously mentioned case selection techniques, descriptive vs. causal? Do we pick two extreme cases if we used an exploratory approach, or two least-likely cases if choosing a diagnostic case approach?

Thankfully, an English scholar by the name of John Stuart Mill provided some insight on how we should proceed. He developed several approaches to comparison with the explicit goal of isolating a cause within a complex environment. Two of these methods, the 'method of agreement' and the 'method of difference' have influenced comparative politics. In the 'method of agreement' two or more cases are compared for their commonalities. The scholar looks to isolate the characteristic, or variable, they have in common, which is then established as the cause for their similarities. In the 'method of difference' two or more cases are compared for their differences. The scholar looks to isolate the characteristic, or variable, they do not have in common, which is then identified as the cause for their differences. From these two methods, comparativists have developed two approaches.

What Is the Most Similar Systems Design (MSSD)?

This approach is derived from Mill’s ‘method of difference’. In a Most Similar Systems Design Design, the cases selected for comparison are similar to each other, but the outcomes differ in result. In this approach we are interested in keeping as many of the variables the same across the elected cases, which for comparative politics often involves countries. Remember, the independent variable is the factor that doesn’t depend on changes in other variables. It is potentially the ‘cause’ in the cause and effect model. The dependent variable is the variable that is affected by, or dependent on, the presence of the independent variable. It is the ‘effect’. In a most similar systems approach the variables of interest should remain the same.

A good example involves the lack of a national healthcare system in the US. Other countries, such as New Zealand, Australia, Ireland, UK and Canada, all have robust, publicly accessible national health systems. However, the US does not. These countries all have similar systems: English heritage and language use, liberal market economies, strong democratic institutions, and high levels of wealth and education. Yet, despite these similarities, the end results vary. The US does not look like its peer countries. In other words, why do we have similar systems producing different outcomes?

What Is the Most Different Systems Design (MDSD)?

This approach is derived from Mill’s ‘method of agreement’. In a Most Different System Design, the cases selected are different from each other, but result in the same outcome. In this approach, we are interested in selecting cases that are quite different from one another, yet arrive at the same outcome. Thus, the dependent variable is the same. Different independent variables exist between the cases, such as democratic v. authoritarian regime, liberal market economy v. non-liberal market economy. Or it could include other variables such as societal homogeneity (uniformity) vs. societal heterogeneity (diversity), where a country may find itself unified ethnically/religiously/racially, or fragmented along those same lines.

A good example involves the countries that are classified as economically liberal. The Heritage Foundation lists countries such as Singapore, Taiwan, Estonia, Australia, New Zealand, as well as Switzerland, Chile and Malaysia as either free or mostly free. These countries differ greatly from one another. Singapore and Malaysia are considered flawed or illiberal democracies (see chapter 5 for more discussion), whereas Estonia is still classified as a developing country. Australia and New Zealand are wealthy, Malaysia is not. Chile and Taiwan became economically free countries under the authoritarian military regimes, which is not the case for Switzerland. In other words, why do we have different systems producing the same outcome?

Communicating the Value of Research: Contractor's Final Report (2009)

Chapter: chapter 3: case study selection criteria and recommended transportation research projects.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

NCHRP 20-78: Final Report Page 29 CHAPTER 3: CASE STUDY SELECTION CRITERIA AND RECOMMENDED TRANSPORTATION RESEARCH PROJECTS Introduction Task 3 in the study work plan required the study team to describe the characteristics of transportation research projects that would be useful for in-depth evaluation and to identify a suitable number of projects for Task 4 case study evaluation. We conducted seven in-depth case studies, and each individual case study consisted of a “whole” study, in which facts were gathered from various sources about the entire lifecycle of the transportation research project—from research proposal to funding (where possible) through implementation—and conclusions were drawn on those facts and are presented here. In analyzing the case studies, our tasks were to:  Map the communication flow and content from initiation through implementation,  Determine the communication practices that were used,  Elicit the participants’ understanding of their roles as communicators and advocates of the project,  Determine patterns of responses (from audiences) to the communication strategy, and  Assess effectiveness of communications for securing research (and implementation) support. Our overall research strategy was to conduct case studies of a wide range of successful transportation research project investment decision-making. Our primary objectives were to highlight lessons learned about effective communication practices from each case study and across all case studies. Each case study covered the following information:  Context: Background information about the research project/program; political, institutional or other situational factors of interest; history of conditions influencing the situation; and current concerns for issues, as well as descriptions of the key actors and the key audiences for communications.  Facts about the Case: Detail on “the value of the research”; narrative that describes the goals and objectives of communication approaches or messaging strategies; map of the communication flow (message senders/receivers, key messages and actors involved); patterns of response from audiences; whether or not participants understood their roles as advocates for the project/program and as communicators of the value of the research.  Challenges Encountered: Perspectives of the various actors relating to the challenges that emerged and how (if) these were overcome; actions taken; and any changes made to strategy.  Outcomes: Analysis of post-communication situation—assessment of the effectiveness of communications for securing research (and implementation) support; the outcomes they expected versus the outcomes that resulted; what lessons for communicating the value of research can be learned from the case; which (if any) attributes of effective communications identified played a significant role. This information will provide the best insight into how to replicate the success in other circumstances.

NCHRP 20-78: Final Report Page 30 Hypothesis Testing A primary benefit of the case study approach is that it is useful for both generating and testing hypotheses. From our analysis of the communications efforts leading to the passage of the research components of SAFETEA-LU (Working Paper #1), we identified several strategies and themes that appeared to be associated with effective communications. The case study evaluations will enable us to test the generalizability of these findings to successful cases of investment in transportation research projects, to identify additional effective strategies and tactics, and to expand on the techniques used. The common threads, or attributes, of effective communication practices that were identified in Chapter 2 were the following:  Communicating the national value or grand vision of the research  Building broad coalitions  Defining a “strategic space” for flexibility of action  Building long-term, multidimensional relationships  Working from within  Using multiple messaging strategies: o Providing research-based information o Identifying “sticky” messages, e.g., demonstrating benefits in terms of resolving problems, saving lives, increasing efficiency, etc.  Establishing a basis for exchange or reciprocity  Tailoring the “ask” to the current mood and concerns of the audience and/or constituent interests  Using illustrative success stories  Presenting information in straight-forward, easy-to-understand language  Hiring, training, and/or selecting professional communicators or lobbyists. The testing of the generalizability of these attributes can be enhanced by strategic selection of cases. Strategy for the Selection of Cases There are various known strategies for the selection of cases that are best organized into two approaches: random selection and information-oriented selection. In random selection, cases are randomly selected from a large sample mainly for establishing credibility (i.e., avoiding subjective bias). In information-oriented selection, cases are selected to demonstrate a characteristic or attribute of interest. In our work, we will use the information-oriented selection approach, because random selection of a small number of cases from a very large universe of potential transportation research projects might result in cases that are not applicable to the project objectives. Given the small number of case studies conducted in this project and the information-oriented selection approach, we applied the following specific criteria: 1. A mixture of both “hard science” and “soft science” research. 2. A variety of types of performing organizations (e.g., universities, state DOTs, private sector) responsible for the research (from research proposal to funding to implementation). These organizations also act as the communicators or “senders of communications.”

NCHRP 20-78: Final Report Page 31 3. Diversity in the locations of the performing organizations. 4. Different types of audiences for communications about the value of the research.7 We identified nine transportation research projects that passed panel review that we felt will inform the research objectives. We researched the first seven of these as case studies. The final two were held in reserve in case one or two of the first seven turned out to be uninteresting or infeasible after preliminary screening. The selected research projects were identified from a number of sources: TRB’s Research in Progress (RIP) database, TRB’s Research Pays Off series, research projects identified by interviewees for the first two tasks, and research projects put forth by members of the panel or the study team. In selecting the nine transportation research projects or programs from this universe, we defined “research” as a product that could be used more than once by other persons for other applications, to distinguish it from a planning study, for instance. We also systematically excluded research that we felt might have resulted because of a response to a Request for Proposal. We wanted to focus on research that needed to be “sold” to a sponsoring or funding agency or to another audience for implementation. In addition to these two factors, the nine projects were selected to meet the four specific criteria identified above. Selected Cases A. Hard Science 1. Adaptive Control Software (ACS) Lite: A significant portion of traffic delays in metropolitan areas is caused by poor traffic signal timing. ACS Lite, a reduced-scale version of the Federal Highway Administration’s (FHWA) Adaptive Control Software (ACS), offers small and medium-size communities a low-cost traffic control system that operates in real time, adjusting signal timing to accommodate changing traffic patterns and ease traffic congestion. ACS Lite can be used with new signals or to retrofit existing traffic signals. (NANCY) Performing Organization: Turner-Fairbank Highway Research Center (FHWA), Siemens, Purdue University, and the University of Arizona Location: National Key Audiences for Communicating the Value of the Research:  Implementers in industry (to get them to be the deliverers)  Various state DOT and local operations professionals (by industry) Rationale: This was a market-ready innovation and was integrated by vendors (manufacturers) so that it was part of the signal timing packages. Delivery was through industry. This case study provided insight into how support for the research to develop the technology was obtained, and how the industry was “sold” on the value of the research product so that it is now the “promoter” of the innovation to small- and medium-size communities. It answered the question: how is industry communicating the value of this research product to these communities? 7 The audiences noted in the case study capsules represent our assumptions at this point. We may find that these will change after we find out more during the execution of the case study research.

NCHRP 20-78: Final Report Page 32 2. Development of a Tough Alloy Structural Steel: Using a high performance steel developed by Northwestern University on behalf of FHWA and US Navy, researchers modified its composition to increase its cold weather toughness and weldability to make it more suitable for highway bridge applications. This new alloy represents a major development in hot- rolled high performance steels that do not require quenching and tempering or other thermo- mechanical processing. The cost per ton is directly competitive with conventional weathering steel (ASTM A588). Northwestern collaborated with the Illinois Department of Transportation to have structural beams fabricated from this special steel, which were then used as main support members on a replacement railroad overpass. Constructability was good, no painting was required, and the bridge continues to perform well under periodic monitoring. Performing Organization: Northwestern University Location: Midwest Key Audiences for Communicating the Value of the Research:  Illinois DOT Executives and engineers Rationale: The case had to be made to IDOT to actually use this steel in a bridge. The case study examined how the university researchers found out that the department had a problem and what they did to convince the department that they had developed a solution to that problem. So Northwestern had to “sell” them on the value of the prior research. They did, and it has worked well. This steel was developed under UTC funding. IDOT’s installation provided needed matching. 3. Seismic Safety Retrofit Program (California Bridges): Caltrans, as a result of the 1989 Loma Prieta earthquake, initiated a major research program to improve the seismic safety of bridges in the state. The program focused on developing retrofit strategies to improve the performance of existing bridges, as well as improve the current design guidelines for new structures. This program continues today, albeit at a slightly reduced scale. The program involved a significant research component to identify the causes of earthquake damage and then to deploy possible solutions to those factors to the actual bridges. The research program was initiated in the early 1990s and focused on understanding the hazard (i.e., earthquake), as well as the structural response to the hazard. A ground motion research program (one research component) was a combined effort of Caltrans, a utility company and the state Energy Commission. This effort required the Legislature to enact a budget change to allow Caltrans to participate. The bridge structure solution set was designed to take that information and design specific fixes to the problems. Performing Organization: Caltrans and consortium of utility companies . Location: West Coast Key Audiences for Communicating the Value of the Research:  State Legislature  Public  DOT engineers.  Public utility companies Rationale: The cost of the research projects was large; an average of $5 million per year since 1989—the ground motion testing program was an additional $14 million. An aspect of

NCHRP 20-78: Final Report Page 33 this case study was to ascertain how the sponsors were able to sell “research” instead of just doing something more immediate. To implement the research, the state needed to defer other capital improvements. This situation required tough decisions among the executives within Caltrans. It turns out that the benefits from the research effort have been enormous; the understanding of the directionality of forces in an earthquake saved between $70 million and $100 million in construction costs on the new San Francisco-Oakland Bay Bridge. The 1994 Northridge Earthquake demonstrated that newly retrofitted structures could survive the design event. In addition to communicating the benefits of such a high cost research project, the significant commitment to seismic retrofit also required that the capital construction program be significantly curtailed (approximately one-half of the expected road building program in the early and mid-1990s was postponed). This represented two significant efforts—voter approval for bonds to do the construction program and working with local governments that received fewer new road projects in a state with congestion problems. 4. Fiber-Reinforced Polymer Bridge Deck: Virginia Department of Transportation (VDOT) tested the utility of using a fiber-reinforced polymer (FRP) composite cellular deck system to rehabilitate cast-iron thru-truss structures. Testing of the technology was done using a full- scale, two-bay section of the bridge that was constructed and tested in the Structures Laboratory at Virginia Tech. Test results showed that no cracks initiated in the joints under the service load, and no significant change in stiffness or strength of the joint occurred after 3 million cycles of fatigue loading. The proposed adhesive bonding technique was installed on the historic Hawthorne Street Bridge in Covington, Virginia, in 2006. Corrosion and other infrastructure damage had rendered the bridge unsafe for vehicle and pedestrian traffic, and it was closed to traffic. The bridge reopened after installation of the new FRP bridge deck that tripled its load limit because of the significantly reduced deck weight. Performing Organization: Virginia DOT and Virginia Tech Location: East Coast Key Audiences for Communicating the Value of the Research:  FHWA’s Innovative Bridge Research Construction (IBRC) Program (now the Innovative Bridge Research and Deployment [IBRD] program under SAFETEA-LU)  DOT engineers  Public (note: because a significant historic bridge was restored, the research and the opening received significant play in the news media, which demonstrated how new technologies can help save historic structures) Rationale: This research demonstrated the application of innovative technologies in the repair, replacement, rehabilitation, or new construction of bridges or other highway structures. VDOT needed to sell the value of its bridge research program to FHWA to receive the funding under the IBRC program, and then after implementation, it needed to market the value of such research to engineers within the department for implementation of the results.

NCHRP 20-78: Final Report Page 34 5. Eliminating Cross-Median Fatalities: Statewide Installation of Median Cable Barrier in Missouri: According to Missouri data, a motorist crossing the median is highly likely to collide with another vehicle, and the chances are high that the opposing vehicle will be a large truck. To address this issue the Missouri Department of Transportation (DOT) researched several options and decided to install a median cable barrier system on I-70 and on other Missouri Interstates. When the cable is struck, the posts yield and the cable deflects up to 12 feet, effectively catching and decelerating the vehicle and keeping it in the median. The installation of 179 miles of median cable barrier on the freeway has nearly eliminated cross- median roadway deaths. In 2006, only two cross-median fatalities occurred on Interstate 70, a staggering 92 percent decrease. Performing Organization: Missouri DOT Location: Midwest Key Audiences for Communicating the Value of the Research:  MoDOT executives  State DOT Engineers (MoDOT and other states)  Public  Media Rationale: This was a cost-effective safety improvement. This case study provided insight into how support for the research to analyze crash location data, search for solutions, prepare and disseminate the information within the agency/division, and get the go-ahead to start implementation was achieved. It was interesting to find out how the researchers were able to sell the idea within the Department to get funding for the study, what the role of research at the national level or in other states was, and how they communicated the value of the research to the public and media. In addition, FHWA promoted cable median barriers to other states—how are these being “sold” to other states and what is the reaction of the states to the sales program?

NCHRP 20-78: Final Report Page 35 B. Soft Science 6. Road User Fee Pilot Program: With the steady erosion of revenue from the state’s gas tax, the Oregon State Legislature created the Road User Fee Task Force (RUFTF) to examine various alternatives for replacing Oregon’s gas tax as the primary source of revenues for repairing, maintaining, and building Oregon’s roads. RUFTF, administered by Oregon DOT, identified mileage-based charging as a potential solution. Oregon DOT launched a mileage fee pilot project in the Portland area to test several key aspects of charging a per mile fee at the pump in lieu of paying the state gas tax. Based on the results of the pilot program, Oregon DOT will draft model legislation for the Oregon State Legislature to consider. Performing Organization: Oregon DOT and Oregon State University Location: Northwest Key Audiences for Communicating the Value of the Research:  State Legislators  Public  Media Rationale: This was a high-risk transportation research project that needed to be approved for research funding by the legislature. This case study provided insight into how a state DOT can communicate effectively with its legislature and then sell a research idea to the general public. How was the state legislature “sold” on the value of the idea and the pilot program? Implementation of research results needed to be “sold” to the legislature and the general public. This project had high policy and public acceptance risk, although there was some technology risk as well. 7. National Cooperative Freight Research Program: A National Cooperative Freight Research Program (NCFRP) was authorized in SAFETEA-LU. The NCFRP is managed by the National Academies, acting through the Transportation Research Board (TRB). The NCFRP Oversight Committee, the governing board for the program, met on December 14–15, 2006 and selected 10 projects for the Fiscal Year 2006 and 2007 programs. Performing Organization: Broad coalition of public-private interests Location: National Key Audiences for Communicating the Value of the Research:  Congress  National stakeholder groups  Various state DOTs  Private sector Rationale: Selling programs is harder than selling projects. The National Cooperative Freight Research Program was a successful result of a concerted communications effort involving public and private entities to communicate the “value” of a dedicated and formal freight research program.

NCHRP 20-78: Final Report Page 36 C. Possible Projects 8. Development of a Cold Region Rural Transportation Research Test Bed in Lewistown, Montana http://www.coe.montana.edu/wti/wti/display.php?id=267: The objective of this project was to improve transportation maintenance, operations and safety with cold-regions research through the collaboration of academia, industry and government. Northern tier states as well as many countries must address similar issues regarding the impacts of harsh winter conditions on operation and maintenance activities, and how these activities affect the environment, roadway infrastructure, and travelers’ safety. Oftentimes it becomes necessary to research innovative designs, maintenance practices and technology applications to address these challenges. However, conducting this type of research can create a public nuisance or safety hazard. Furthermore, and perhaps most importantly, transportation research can oftentimes be one-dimensional. Researchers worldwide recognize and appreciate the multidimensional aspects of this type of research, yet do not have the opportunity to simultaneously study them in a controlled environment. The Western Transportation Institute developed, along with a consortium of five western U.S. state DOTs, a research test bed to study rural transportation issues related to design, maintenance and operations in a colder climate using the runways, taxiways, and other underutilized assets at the Lewistown airport. Establishing a single research facility that has the capability to conduct a broad array of transportation research will save much needed resources over many years. Performing Organization: Western Transportation Institute, Montana State University, Bozeman Location: Mountain Region Key Audiences for Communicating the Value of the Research:  State DOTs in Montana, Idaho, Washington, Oregon and California (especially the Maintenance Offices)  U.S. Congress Rationale: This was programmatic—a long-term effort to conduct both basic and applied research. The communication approaches and messaging strategies to “sell” the value of the state-of-the-art research facility on a difficult topic were interesting; the project benefits were communicated differently to the maintenance offices (where technical benefits were important) and the Congressional office (where local job creation at an underutilized airport was important).

NCHRP 20-78: Final Report Page 37 9. Seattle Area Freeway and HOV Lane Performance Monitoring: The Washington State Department of Transportation sponsors an analysis of the operation of the freeway and high- occupancy vehicle facilities in the Seattle region conducted by the University of Washington Transportation Research Center. The project uses the same data used to produce the real-time travel time and speed Website maps and has also benefited from funding by the Transportation Northwest (TransNow) Regional University Transportation Center. The case study will focus on the research communication techniques and the connections between data and the messages used in conjunction with legislative and public audiences. Performing Organization: University of Washington Transportation Research Center, Seattle Location: Northwest Key Audiences for Communicating the Value of the Research:  State Legislators  Public Rationale: This long-running research project has been a leader in the development and use of archived travel time, speed, and volume data. The research products included a variety of reports and technical memoranda that are used to evaluate transportation improvements by researchers, DOT staff, and legislative staff. The project has also pioneered the development of a number of communication methods, particularly graphic elements, which have provided WSDOT and many other agencies across the country with ways to use data in discussions about the effect of various transportation elements with the state legislature and the public.

TRB’s NationalCooperative Highway Research Program (NCHRP) Web-Only Document 131: Communicating the Value of Transportation Research is the contractor’s final report on the research associated with NCHRP Report 610: Communicating the Value of Transportation Research .

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Comprehensive analysis of rule formalisms to represent clinical guidelines: Selection criteria and case study on antibiotic clinical guidelines

Affiliations.

• 1 AIKE Research Group (INTICO), Faculty of Computer Science, University of Murcia, Spain. Electronic address: [email protected].
• 2 AIKE Research Group (INTICO), Faculty of Computer Science, University of Murcia, Spain. Electronic address: [email protected].
• 3 AIKE Research Group (INTICO), Faculty of Computer Science, University of Murcia, Spain. Electronic address: [email protected].
• PMID: 31928849
• DOI: 10.1016/j.artmed.2019.101741

Background: The over-use of antibiotics in clinical domains is causing an alarming increase in bacterial resistance, thus endangering their effectiveness as regards the treatment of highly recurring severe infectious diseases. Whilst Clinical Guidelines (CGs) focus on the correct prescription of antibiotics in a narrative form, Clinical Decision Support Systems (CDSS) operationalize the knowledge contained in CGs in the form of rules at the point of care. Despite the efforts made to computerize CGs, there is still a gap between CGs and the myriad of rule technologies (based on different logic formalisms) that are available to implement CDSSs in real clinical settings.

Objective: To helpCDSS designers to determine the most suitable rule-based technology (medical-oriented rules, production rules and semantic web rules) with which to model knowledge from CGs for the prescription of antibiotics. We propose a framework of criteria for this purpose that is extensible to more generic CGs.

Materials and methods: Our proposal is based on the identification of core technical requirements extracted from both literature and the analysis of CGs for antibiotics, establishing three dimensions for analysis: language expressivity, interoperability and industrial aspects. We present a case study regarding the John Hopkins Hospital (JHH) Antibiotic Guidelines for Urinary Tract Infection (UTI), a highly recurring hospital acquired infection. We have adopted our framework of criteria in order to analyse and implement these CGs using various rule technologies: HL7 Arden Syntax, general-purpose Production Rules System (Drools), HL7 standard Rule Interchange Format (RIF), Semantic Web Rule Language (SWRL) and SParql Inference Notation (SPIN) rule extensions (implementing our own ontology for UTI).

Results: We have identified the main criteria required to attain a maintainable and cost-affordable computable knowledge representation for CGs. We have represented the JHH UTI CGs knowledge in a total of 12 Arden Syntax MLMs, 81 Drools rules and 154 ontology classes, properties and individuals. Our experiments confirm the relevance of the proposed set of criteria and show the level of compliance of the different rule technologies with the JHH UTI CGs knowledge representation.

Conclusions: The proposed framework of criteria may help clinical institutions to select the most suitable rule technology for the representation of CGs in general, and for the antibiotic prescription domain in particular, depicting the main aspects that lead to Computer Interpretable Guidelines (CIGs), such as Logic expressivity (Open/Closed World Assumption, Negation-As-Failure), Temporal Reasoning and Interoperability with existing HIS and clinical workflow. Future work will focus on providing clinicians with suggestions regarding new potential steps for CGs, considering process mining approaches and CGs Process Workflows, the use of HL7 FHIR for HIS interoperability and the representation of Knowledge-as- a-Service (KaaS).

Keywords: Arden; CDSS; CG; Drools; FHIR; HIS; KaaS; OWL; Ontology; RIF: SWRL; Rules; SHACL; SPIN; Semantic Web.

Copyright © 2019 Elsevier B.V. All rights reserved.

Publication types

• Research Support, Non-U.S. Gov't
• Anti-Bacterial Agents / therapeutic use
• Antimicrobial Stewardship / organization & administration*
• Antimicrobial Stewardship / standards
• Cross Infection / drug therapy
• Decision Support Systems, Clinical / organization & administration*
• Decision Support Systems, Clinical / standards
• Expert Systems*
• Organizational Case Studies
• Practice Guidelines as Topic / standards*
• Urinary Tract Infections / drug therapy
• Anti-Bacterial Agents

• Partnerships
• Contact CAP
• Advancing Measurement of Social Equity
• Harry Hatry Distinguished Performance Management Practice Award
• Joesph Wholey Distinguished Scholarship Awards
• CAP Organizational Leadership Award
• Emerging Leaders Award
• Purpose of CAP Case Studies
• Eligible Participants
• Potential Topic Areas for Case Studies
• The Selection Process

The Selection Criteria

• The Timetable
• 2015 & 2014 CAP Case Studies
• Case Study Dissemination
• Government Accountability Office Reports
• IBM Center for The Business of Government – Improving Government Performance
• Accountability and Performance Weekly

Case Study Selection Criteria

Listed below are the criteria to be used by CAP Board reviewers for their selection of CAP Case Studies for presentation at the upcoming ASPA Conference.

• Background – Does the case study contain enough recent history to provide information on how the agencies in the case studies analyzed their performance and related information, identified factors related to variations in program performance, and used the resulting information?

• Outcomes – Does the case study clearly address whether performance and related information are associated not only with resulting changes in program activities but also to be associated with changes in program outcomes?

• Communication – Does the case study provide evidence that author(s) have established effective contacts within the relevant organization(s) to support the case study, and its findings and conclusions?

• Multi-Level Application – To what extent are the case study findings expected to be useful to state and local governments as well as to federal agencies?

• Logic Model – Does the case study provide some description of the logic model supporting the program, including linkage between strategic goals and performance measures and targets, and if appropriate, budgets?

• Conclusion(s) – Does the case study clearly articulate the conclusion that follows from its analysis of the use of performance information, and if appropriate, identify any best practices identified as part of the analysis?

• Contribution to Research and Practice – To what extent does the case study provides insights, lessons learned, observations, or best practices that would be useful in advancing the research and practice related to performance, management, and evaluation?

Share this:

• Search for:

Welcome to ASPA-CAP

Welcome to the Center for Accountability and Performance website! CAP aims to improve the practice of public service by helping Public Administration and Nonprofit professionals acquire the knowledge, technical skills and resources necessary to successfully manage for results. CAP achieves its mission through education, training, advocacy, technical assistance, resource sharing, and research into best practices in Performance Management.

Please browse the website to learn more about CAP.

Recent Posts

• Accountability and Performance Update – May 6 – 12, 2024
• Accountability and Performance Update – Apr 29-May 5, 2024
• Accountability and Performance Update – Apr 22-28, 2024
• Accountability and Performance Update – Apr 15-21, 2024
• Accountability and Performance Update – Apr 8-14, 2024

Follow ASPA-CAP

• To learn more about the American Society for Public Administration, please visit ASPA Home

• Already have a WordPress.com account? Log in now.
• Subscribe Subscribed
• Copy shortlink
• Report this content
• View post in Reader
• Manage subscriptions
• Collapse this bar

• Open access
• Published: 05 May 2024

The learning curve in endoscopic transsphenoidal skull-base surgery: a systematic review

• Abdulraheem Alomari 1 ,
• Mazin Alsarraj 2 &
• Sarah Alqarni 3  

BMC Surgery volume  24 , Article number:  135 ( 2024 ) Cite this article

159 Accesses

Metrics details

The endoscopic endonasal transsphenoidal approach (EETA) has revolutionized skull-base surgery; however, it is associated with a steep learning curve (LC), necessitating additional attention from surgeons to ensure patient safety and surgical efficacy. The current literature is constrained by the small sample sizes of studies and their observational nature. This systematic review aims to evaluate the literature and identify strengths and weaknesses related to the assessment of EETA-LC.

A systematic review was conducted following the PRISMA guidelines. PubMed and Google Scholar were searched for clinical studies on EETA-LC using detailed search strategies, including pertinent keywords and Medical Subject Headings. The selection criteria included studies comparing the outcomes of skull-base surgeries involving pure EETA in the early and late stages of surgeons’ experience, studies that assessed the learning curve of at least one surgical parameter, and articles published in English.

The systematic review identified 34 studies encompassing 5,648 patients published between 2002 and 2022, focusing on the EETA learning curve. Most studies were retrospective cohort designs (88%). Various patient assortment methods were noted, including group-based and case-based analyses. Statistical analyses included descriptive and comparative methods, along with regression analyses and curve modeling techniques. Pituitary adenoma (PA) being the most studied pathology (82%). Among the evaluated variables, improvements in outcomes across variables like EC, OT, postoperative CSF leak, and GTR. Overcoming the initial EETA learning curve was associated with sustained outcome improvements, with a median estimated case requirement of 32, ranging from 9 to 120 cases. These findings underscore the complexity of EETA-LC assessment and the importance of sustained outcome improvement as a marker of proficiency.

Conclusions

The review highlights the complexity of assessing the learning curve in EETA and underscores the need for standardized reporting and prospective studies to enhance the reliability of findings and guide clinical practice effectively.

Peer Review reports

With the advent of endoscopic techniques, skull-base surgery has significantly advanced. The modern history of neuro-endoscopy began in the early 1900s with an innovation by Lespinasse and Dandy, involving intraventricular endoscopy to coagulate the choroid plexus for treating communicating hydrocephalus [ 1 ]. In 1963, Guiot first reported an endoscopic approach via the transsphenoidal route as an adjunct to procedures performed under microscopy [ 2 , 3 ]. In 1992, Jankowski et al. described a purely endoscopic approach for pituitary adenoma resection [ 1 ].

The advantages of endoscopy have encouraged skull-base surgeons to adopt this technique, which provides a panoramic view of critical anatomical landmarks and improved access to the corners and deep surgical areas while inducing only minor trauma to the nasal structures, thereby enhancing postoperative patient comfort [ 4 ]. Compared with procedures involving microscopy, the endoscopic approach results in a shorter operating time (OT), a reduced hospitalization period, a lower rate of complications, and a higher endocrinological cure rate [ 5 , 6 ]. Despite these benefits, the endoscopic approach is hindered by a two-dimensional view, instrument interference, difficulties in achieving homeostasis, and a steep learning curve (LC) [ 4 ].

Since its inception, pioneers in the field have recognized the steep LC associated with the endoscopic technique [ 7 ]. The safety and efficacy of the endoscopic endonasal transsphenoidal approach (EETA), as an alternative to the gold-standard microscopic technique, have been established. However, the steep LC associated with the endoscopic approach may affect short-term outcomes post-procedure [ 5 , 6 ]. Additionally, as the skull-base endoscopic technique constantly evolves and expands, a thorough understanding of the associated LC is critical.

The results of existing publications on the EETA-LC are challenging to interpret due to small sample sizes, observational study designs, and a lack of standardization in assessment methodologies. In this systematic review aims to elucidate the EETA-LC from the literature by addressing the following questions: How was EETA LC evaluated? Which set of variables was used to assess the LC? What is the influence of the LC on the examined variables?

A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines [ 8 ]. The review was registered on PROSPERO (CRD42023494731). We searched different databases for articles that assessed the learning curve of EETA without date restriction (PubMed, and Google Scholar). We used a particular equation for each database using a combination of the following keywords and Medical Subject Headings: (Endoscopy OR endoscopic skull base OR endoscopic endonasal transsphenoidal approach) AND (Skull Base Neoplasms OR Pituitary OR pituitary adenoma) AND (Learning Curve OR endoscopic learning curve OR surgical learning curve).

First, two authors (AA, MA) independently screened the titles and abstracts of articles in the databases for learning curve analysis of EETA, either for a single surgeon or a team, by directly comparing outcomes between early and late cases performed. The full texts of the relevant articles were reviewed. When there was a disagreement, the articles were thoroughly discussed before their inclusion in the review. The bibliographies of the selected studies were also screened for relevant citations, which turned up studies that were already selected from the database search.

Studies were included according to the following inclusion criteria: 1) Comparison of outcomes between initial and advanced experiences with the endonasal endoscopic transsphenoidal approach to treat skull-base pathology, defined as "early experience" and "late experience," respectively; 2) Assessment of at least one parameter based on early and late experiences; 3) Randomized controlled trials, prospective cohort studies, retrospective cohort studies, case–control studies, and case series studies were included; and 4) English-language publications.

The study’s exclusion criteria included the following: 1) Studies not performing learning curve analysis; 2) Studies comparing the outcomes of microscopic and endoscopic transsphenoidal approaches without providing separate data for the endoscopic approach; 3) Studies comparing the learning curve between two EETA techniques, using simulated models or questionnaire-based analysis; 4) Studies comparing the microscopic vs. endoscopic approach without separate data available specifically for the endoscopic arm. Additionally, case reports, reviews, animal studies, technical notes, comments, and correspondence were excluded.

Data collection and analysis

The following data were extracted directly from the articles: 1) author names; 2) the year of publication; 3) Time interval of performed procedures; 4) study design; 5) the sample size; 6) techniques used for learning curve analysis (methods used to assort the patients for the analysis); (conducting statistical analysis vs. simple comparison of outcomes); 7) the sample size in each study arm when group splitting performed (early experience vs. late experience); 8) detailed information about surgeon experience at the time of LC assessment (including or omitting the first few EETA cases); 9) single vs. multiple pathologies; 10) team vs. single-surgeon experiences; 11) evaluated set of variables; 12) Variables that improved with experience; and 13) the number of cases required to overcome the initial LC or other methods to identify overcoming the learning curve.

Study quality assessment and risk of bias

Two reviewers conducted a quality assessment and evaluated the risk of bias in the included articles. We utilized the Newcastle–Ottawa Scale (NOS) [ 9 ] and the GRADE system [ 10 ].

Heterogeneity Analysis: Due to substantial heterogeneity observed among the included studies, which encompassed variations in study design, included pathologies, and outcome measures, a formal meta-analysis was not feasible. Therefore, we opted for a qualitative synthesis instead of a formal meta-analysis. Heterogeneity analysis and sensitivity analyses were not explicitly conducted.

Based on the inclusion and exclusion criteria, a total of 34 studies were identified (6 articles excluded after reviewing the full articles), including 5,648 patients [ 7 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ] (Fig.  1 ). The included studies were published between 2002 and 2022, and the evaluated procedures were performed between 1990 and 2018. The majority of the included articles comprised retrospective cohort studies (88%), with two being prospective studies, and two articles presenting data from both prospective and retrospective study designs. Assessing a surgical learning curve involves various methods and techniques documented within the included articles. We observed various methods for patient assortment in conducting learning curve analyses across the literature, with group-based learning curve analysis noticeable in a significant proportion of articles (68%). Within these studies, there was an unclear rationale behind patient grouping. Nonetheless, patients were categorized into either equal group, segmented based on arbitrary time periods, or separated based on improvements in outcomes observed retrospectively after data analysis. Eleven articles (32%) utilize case-based analysis, where individual surgical cases serve as distinct data points, and their outcomes are monitored over time.

PRISMA flow diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses

* The bibliographies of the selected studies were also screened for relevant citations which turned up studies already included from databases search

Our systematic review encompasses a wide range of statistical tests employed in the included studies to analyze various data types and address multifaceted research inquiries. The primary statistical methodologies utilized encompass descriptive statistical analysis, which includes metrics such as mean, median, frequency, and standard deviation, along with comparative statistical analysis, which includes techniques such as Chi-square analysis, analysis of variance (ANOVA), and t-tests. Descriptive statistical analysis alone was evident in 10 articles (29%), whereas comparative statistical analysis was present in 24 articles (71%). Noteworthy examples include Leach et al. [ 16 ], who conducted analysis of variance (ANOVA) with post hoc Bonferroni tests for parametric data, Chi-Square Test, or Mann–Whitney tests for nonparametric data, and regression analysis to explore the relationship between surgical duration and relevant factors. Smeth et al. [ 17 ] undertook analyses using chi-square, Fisher exact, Student t-test, Mann–Whitney U test, and analysis of variance, aligning with their examination of categorical and continuous variables across distinct groups. Similarly, Sonnenburg et al. [ 12 ] applied a one-way ANOVA to discern variations between groups, highlighting the importance of understanding differences in means across categorical variables or treatment cohorts.

Regression analyses, scatterplots, McNemar tests, ROC curve analysis, and logistic regression models were integral across various studies, serving multiple purposes. Regression analyses, such as linear regression models, facilitated the exploration of intricate relationships among variables like age, tumor size, and surgical duration, identifying potential risk factors in surgical contexts [ 22 ]. Scatterplots visually depicted these relationships, offering intuitive insights into temporal variations, notably in the examination of surgery date versus duration [ 22 ]. McNemar tests were instrumental in evaluating changes in hormone levels, crucial for understanding postoperative outcomes and hormonal dynamics [ 37 ]. Additionally, ROC curve analysis provided a robust method for determining the level of surgical experience necessary to achieve gross total resection (GTR), offering actionable insights into surgical proficiency and patient outcomes [ 37 ]. Binary logistic regression models were utilized to identify prognostic factors contributing to the attainment of Gross Total Resection (GTR), hormonal recuperation, and visual restoration. For instance, variables such as surgical experience (≤ 100 vs. > 100 cases) were examined within this analytical framework [ 37 ].

In our examination of the included articles, we noted a lack of thorough description regarding the experience of surgeons or surgical teams with the endoscopic endonasal transsphenoidal approach (EETA), the extent of the approach undertaken, and the level of involvement of individual surgeons or surgical teams during procedures. Thirteen articles (38%) reported including the initial cases of EETA, which may indicate a lack of prior experience with the approach. Additionally, seven articles (21%) detailed the experience of a single surgeon, while the majority (79%) evaluated team experiences. There was a wide range of pathologies included in all the studies. Twenty articles (59%) focused on a single pathology, while fourteen studies (41%) examined multiple pathologies. Pituitary adenoma (PA) was the most frequently reported pathology (82%), followed by craniopharyngioma (CP) (44%). Three studies assessed the learning curve of cerebrospinal fluid (CSF) leak repair following treatment of multiple pathologies. Descriptions of the surgical approach, particularly distinguishing between simple and extended techniques, were notably lacking across all articles. However, seventeen articles (50%) did mention pathologies that often require an extended approach, such as meningioma, chordoma, and CP. A number of studies have investigated the variations in tumor type and size among the examined groups, particularly between early and late groups. Notably, findings from studies such as [ 7 , 16 , 17 , 22 , 23 , 26 , 38 ] indicated that no statistical differences were observed between these groups. The characteristics of the included studies [ 7 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ] are summarized in Table  1 .

The EETA-LC was evaluated based on a diverse set of variables. The most frequently analyzed variables were postoperative cerebrospinal fluid (CSF) leak in 28 articles (82%) [ 7 , 12 , 13 , 15 , 16 , 17 , 19 , 20 , 21 , 22 , 23 , 25 , 27 , 28 , 29 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ], gross total resection (GTR) in 21 articles (62%) [ 7 , 13 , 14 , 16 , 19 , 21 , 22 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 36 , 37 , 38 , 39 , 40 ], post operative diabetes insipidus (DI) in 15 articles (44%) [ 12 , 13 , 16 , 17 , 19 , 21 , 22 , 29 , 30 , 31 , 32 , 34 , 36 , 37 , 41 ], operative time (OT) in 12 articles (35%) [ 7 , 13 , 14 , 16 , 17 , 22 , 29 , 32 , 34 , 35 , 36 , 38 ] and visual improvement in 12 articles (35%) [ 13 , 14 , 16 , 21 , 22 , 28 , 31 , 32 , 34 , 36 , 37 , 41 ]. (Fig.  2 ).

Frequency at which certain variables were evaluated in the literature to assess the EETA learning curve. EETA, endoscopic endonasal transsphenoidal approach; post-op, postoperative; CSF, cerebrospinal fluid; GTR, gross total resection; DI, diabetes insipidus; LOS, length of stay; IOP, intraoperative; ICA, internal carotid artery; SIADH, syndrome of inappropriate antidiuretic hormone secretion; LD, lumbar drain; CNS, central nervous system; CN, cranial nerve; EBL, estimated blood loss; DVT, deep vein thrombosis

In all the studies included, improvements were observed between early and late-experience stages [ 7 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ]. Among the evaluated variables, the following improvements were noted: the endocrinological cure rate (EC) showed improvement in all 7 articles out of 7 evaluated [ 13 , 16 , 18 , 21 , 24 , 30 , 33 ], operative time (OT) improved in 11 out of 12 articles (91%) [ 13 , 14 , 16 , 17 , 22 , 29 , 32 , 34 , 35 , 36 , 38 ], postoperative cerebrospinal fluid leak (CSF) improved in 23 out of 28 articles (82%) [ 12 , 15 , 17 , 19 , 20 , 22 , 23 , 25 , 27 , 28 , 29 , 31 , 32 , 33 , 34 , 35 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ], visual improvement was observed in 9 out of 12 articles (75%) [ 13 , 14 , 16 , 22 , 28 , 31 , 34 , 37 , 41 ], gross total resection (GTR) improved in 14 out of 21 articles (67%) [ 7 , 13 , 14 , 19 , 21 , 22 , 26 , 27 , 28 , 29 , 30 , 38 , 39 , 40 ], hospital length of stay (LOS) decreased in five out of 10 studies (50%) [ 11 , 12 , 16 , 17 , 22 ], and postoperative diabetes insipidus (DI) decreased in 7 out of 15 articles (47%) [ 3 , 14 , 16 , 17 , 21 , 22 , 33 ] (Fig.  3 ).

Proportion of main improved variables with experiences. EC, Endocrinological cure; OT, Operative time; post-op: postoperative; CSF, cerebrospinal fluid; GTR, gross total resection; hLOS, hospital length of stay; DI, diabetes insipidus

Moreover, 12 articles (35%) reported both significant and non-significant improvements in outcomes [ 7 , 13 , 14 , 16 , 17 , 21 , 22 , 31 , 32 , 34 , 38 , 41 ]. In 10 studies (29%), solely a trend of improvement was observed [ 11 , 15 , 19 , 20 , 23 , 26 , 27 , 29 , 30 , 40 ], while 8 articles (23%) reported solely significant improvements [ 18 , 24 , 25 , 35 , 36 , 37 , 42 , 43 ]. However, in four studies, despite observing a tendency towards better outcomes, no statistical disparities were identified among all assessed variables [ 12 , 28 , 33 , 39 ]. None of the included studies reported a deterioration in any of the assessed outcomes over time, except for one study where a significant decline in GTR was observed in the late group [ 33 ]. This decline was attributed to the inclusion of more invasive and complex tumors in the late group. Nevertheless, Younus et al. documented ongoing improvement in GTR even after surpassing the initial learning curve [ 7 ].

In this systematic review, the primary technique employed to determine the transition point indicating the overcoming of the initial learning curve involved observing sustained and consistent improvement in outcomes over time. In almost half of the included articles, overcoming the initial learning curve (observing improvement of outcomes) was linked to the number of cases performed. Out of the 34 analyzed studies, 16 (47%) estimated the number of cases needed to overcome the initial learning curve of EETA. Reported cases ranged widely from 9 to 120, with a mode of 50. Considering both the median and the Interquartile Range (IQR) provides a comprehensive understanding of the reported case distribution and central tendency for overcoming the initial EETA learning curve. The median number of cases needed is 32, with an IQR of 20. These numbers are estimates and require careful interpretation [ 16 , 17 , 20 , 21 , 22 , 23 , 24 , 25 , 29 , 31 , 32 , 33 , 35 , 36 , 37 , 38 , 42 ].

Regarding the quality of included studies, the NOS quality assessment scale was used. 21 studies graded as fair quality while the remaining 13 articles rated as poor quality [ 9 ]. The risk of bias was evaluated according to the GRADE system. All included studies are observational cohort study and graded either as low or very low grade [ 10 ]. This reflects the great heterogeneity and high risk of bias due to the study design of the current EETA-LC literature.

Endoscopic techniques have drastically improved skull-base surgery. Unlike procedures involving a microscope, many neurosurgeons have acquired experience in endoscopic techniques later in their careers, and the level of exposure to these techniques during training years has varied among surgeons. The LC is a critical factor in the acquisition of new surgical skills. Understanding the link between the EETA-LC and surgical outcomes will enable surgeons to better understand what to expect and what measures to apply as those surgical skills develop. Many studies in other surgical domains have reported on the LC during the acquisition of new surgical techniques [ 44 , 45 , 46 , 47 ]. Most minimally invasive surgeries are associated with a challenging LC, and EETA is no exception [ 7 , 46 ].

The concept of the LC was first established in the field of aircraft manufacturing and refers to an improvement in performance over time [ 48 ]. Smith et al. [ 17 ] have defined it as the number of procedures that must be performed for the outcomes to approach a long-term mean rate. Typically, an LC is characterized by an S-shaped curve with three stages: an early phase, during which new skill sets are acquired; a middle phase, in which the speed of learning rapidly increases; and an expert phase in which the performance reaches a plateau [ 49 ]. However, other curves have been proposed that involve a dip in the LC following the initial acceleration of the learning rate; this occurs especially with handling more challenging cases. Another potential decline may emerge after a long period of experience. Despite having reached a plateau in the learning curve after an extended period, declines in manual dexterity, eyesight, memory, and cognition may overshadow the benefits of accumulated experience, leading to diminished performance levels [ 50 ].

The absence of consensus on the best applicable methods to describe and assess the learning curve may explain the diversity of analysis methods observed in this systematic review. In their large systematic review regarding learning curve assessment in healthcare technologies, Ramsay et al. [ 51 ] reported that group splitting was the most frequent method. They defined group splitting as dividing the data by experience levels and conducting testing on discrete groups, often halves or thirds. The statistical methods applied included t-tests, chi-squared tests, Mann–Whitney U tests, and simple ANOVA.

In our review, we reached a similar conclusion. We observed that a substantial portion of articles (68%) utilized group-based learning curve analysis [ 7 , 11 , 12 , 13 , 16 , 17 , 19 , 21 , 22 , 23 , 26 , 27 , 28 , 30 , 31 , 32 , 33 , 34 , 36 , 37 , 38 , 42 , 43 ]. Additionally, we similarly noted that papers frequently lacked explanations for the selection of cut points, raising concerns about potential bias resulting from data-dependent splitting. It is important to acknowledge that this method of group categorization has inherent drawbacks, including challenges related to small sample sizes, the use of arbitrary cutoff points, and the inability to eliminate all potential confounding variables [ 52 ].

Descriptive analysis was found in 10 articles (29%) within this review [ 11 , 15 , 19 , 20 , 23 , 26 , 27 , 29 , 30 , 40 ]. While providing an initial grasp of data distribution and characteristics, descriptive analysis may fall short in capturing the intricate dynamics of the learning curve over time or the factors affecting its impact [ 51 ]. Alternatively, conducting rigorous statistical analyses afterward offers better insight and interpretation of the results. This approach aims to mitigate the influence of confounding factors on outcome assessments over time [ 51 , 52 ].

In our review, 24 articles (71%) conducted a wide variety of statistical analyses [ 7 , 12 , 13 , 14 , 16 , 17 , 18 , 21 , 22 , 24 , 25 , 28 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 41 , 42 , 43 ], including but not limited to the following tests: Chi-square Test, Fischer exact test, Student's t-test, Analysis of Variance (ANOVA), Mann–Whitney U Test, McNemar tests, Multivariate linear regression model, Cumulative Sum (CUSUM), and ROC Curve Analysis [ 13 , 16 , 22 , 32 , 37 , 38 , 39 ]. Four studies indicated that there was no statistically significant difference observed among the variables under evaluation. The lack of significance was attributed to several factors including small sample sizes, meticulous case selection, involvement of an otolaryngology team throughout the procedure, an increase in the number of invasive tumors in the late-experience study group, previous surgical experience, intensive training, level of supervision, and gradual inclusion of residents [ 12 , 28 , 33 , 39 ]. These efforts should be regarded as beneficial strategies aimed at reducing the steepness of the EETA learning curve.

To obtain more accurate results, it is crucial to eliminate confounding factors, such as the level of supervision, prior experience, the heterogeneity of cases being treated, and their complexity when evaluating the LC. Thus, it is essential to incorporate multivariate logistic regression analysis to mitigate the impact of these potential confounding factors [ 51 ]. Chi et al. [ 22 ] divided their patients into equal groups of 40 cases each. They then compared potential confounding variables to minimize their influence on learning curve assessment. This comparison includes demographic and clinical factors between the two groups, such as sex distribution, mean age, tumor size (microadenomas vs. macroadenomas), visual field defects, and tumor types (non-functioning, functioning adenomas, etc.). By conducting these comparisons, the researchers sought to identify discrepancies in demographic and clinical features between the groups.

The description of a surgeon's extensive prior experience is crucial for accurately quantifying the assessment of the learning curve, a point reported to be neglected during the assessment in various types of learning assessments related to healthcare procedures [ 49 ]. In our review, we observed the same conclusion in all included studies. However, the inclusion of the initial first few cases was mentioned in 13 (38%) articles, which might be used as a surrogate for no prior experience with EETA. Furthermore, five articles did not include the initial few cases. Among these, four studies examined the learning curve of more complex cases such as meningioma, craniopharyngioma, and growth hormone pituitary adenoma, employing an extended approach. Conversely, Younus et al. [ 7 ] deliberately excluded these cases to assess various stages of the learning curve.

Assessing multiple pathologies with varying complexities could significantly impact learning curve assessments. In our review, 59% of articles focused on a single pathology, while 41% explored multiple pathologies. Pituitary adenoma (PA) was the most evaluated (82%), followed by craniopharyngioma (CP) (44%). Controlling confounding variables like tumor type and size may yield more reliable results. Some studies used statistical analyses to compare early and late cases, while others relied on descriptive analyses. Shou et al. noted a drop in GTR over time due to late involvement of complex cases [ 33 ]. Conversely, studies analyzing tumor size and type found GTR improvement with experience [ 7 , 23 ]. Thorough multivariable analysis of confounding factors is crucial for representative LC analysis.

The LC is often assessed based on two main categories of variables: those related to the surgical procedure (OT, estimated blood loss, and extent of resection) and those related to patient outcomes (duration of hospitalization, the incidence of complications, and the mortality rate) [ 50 ]. In this systematic review, OT was one of the most frequent parameters that significantly reduced as one gained experience. Although OT is commonly utilized as an outcome measure, it is only a surrogate means of evaluating the LC and may not always accurately represent patient outcomes [ 52 ]. Another point to consider is the lack of standardized variables for assessing the LC, and the included studies evaluated more than 45 distinct variables. Khan et al. highlighted the importance of using consistent variable definitions across studies to derive accurate conclusions from aggregated LC data [ 52 ].

A dynamic relationship exists between surgical outcomes and the LC, and each phase of the LC influences a distinct set of variables differently. One study, which included data from 1,000 EETA cases after purposely eliminating the first 200 cases, showed that variables such as GTR and the endocrinological cure rate continued to improve after the first 200 cases, whereas other parameters remained unchanged. Authors concluded that some variables will continue to improve after passing the initial LC phase [ 7 ]. Determining the precise number of cases needed to surpass the initial learning curve (LC) has proven challenging. Shikary et al. observed a notable decrease in postoperative CSF leaks after 100 surgeries, while a reduction in operative time was evident after 120 cases [ 35 ]. However, specifying a definitive number to overcome the learning curve of the Endoscopic Endonasal Transsphenoidal Approach (EETA) remains challenging due to individual variability, diverse pathologies, and evolving surgical techniques.

Assessing the learning curve of the Endoscopic Endonasal Transsphenoidal Approach (EETA-LC) faces notable challenges due to its intricate techniques and the wide array of pathologies it addresses. The diversity across specialties makes standardizing studies difficult. To understand the dynamic learning process in EETA-LC, influenced by individual surgeon skill, patient nuances, and procedural complexities, longitudinal studies and advanced analytical methods are essential. Moreover, the complexity of statistical analysis adds another layer of challenge, highlighting the necessity for interdisciplinary collaboration and innovative methodologies.

To address the current limitations in the literature regarding the EETA LC, we propose several key strategies for future studies. Firstly, we advocate for multicenter collaboration, coupled with standardized processes, to comprehensively assess the EETA LC. This collaborative approach will facilitate the aggregation of data from diverse surgical settings, enhancing the generalizability of findings and minimizing bias. Furthermore, rigorous documentation of the previous and current experience of involved surgeons is paramount. We suggest categorizing surgeons based on their levels of experience to accurately elucidate the impact of proficiency on surgical outcomes. Secondly, given the wide variety of complexities of skull base pathologies encountered, we recommend further categorization of cases based on their levels of complexity. This stratification will enable a more nuanced analysis of the learning curve across different levels of surgical challenge. Thirdly, standardization of outcome measures used to assess the learning curve is imperative, with specific definitions provided for each outcome. This ensures consistency and comparability across studies, facilitating meaningful interpretation of results. Finally, conducting prospective study designs with sufficient follow-up periods, along with rigorous multivariate statistical analyses among these categorized groups, is essential to mitigate the influence of confounding variables and strengthen the validity of findings. Implementing these strategies will help future studies to overcome the current limitations in the literature, leading to a deeper understanding of the EETA learning curve and ultimately improving patient outcomes.

This systematic review identified 34 studies that reported a relationship between improvements in surgical outcomes and a surgeon’s level of experience with EETA. There is notable significant heterogeneity in the current literature on EETA-LC regarding the techniques used to assess the LC, variables assessed, types of pathology included, and insufficient reporting of the surgeon or team's current and previous experience with EETA. The main variables improved with experience were EC, postoperative CSF leak, OT, GTR visual improvement, and hospital LOS. Future studies with multicenter collaboration and standardized processes for assessing the EETA LC will enhance generalizability and minimize bias. Rigorous documentation of surgeons' experience levels, categorization of cases by complexity, and standardized outcome measures are essential. Additionally, rigorous statistical analyses will strengthen validity and mitigate confounding variables. Implementing these strategies will deepen our understanding of the EETA learning curve, ultimately leading to improved patient outcomes.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Endoscopic endonasal transsphenoidal approach

• Learning curve

Cerebrospinal fluid

Diabetes insipidus

Gross total resection

Cappabianca P, de Divitiis E. Endoscopy and transsphenoidal surgery. Neurosurgery. 2004;54:1043–50.

Article   PubMed   Google Scholar  

Gandhi CD, Post KD. Historical movements in transsphenoidal surgery. Neurosurg Focus. 2001;11:1–4.

Article   Google Scholar  

Gandhi CD, Christiano LD, Eloy JA, Prestigiacomo CJ, Post KD. The historical evolution of transsphenoidal surgery: facilitation by technological advances. Neurosurg Focus. 2009;27:E8.

de Divitiis E. Endoscopic transsphenoidal surgery: stone-in-the-pond effect. Neurosurgery. 2006;59:512–20.

Rotenberg B, Tam S, Ryu WHA, Duggal N. Microscopic versus endoscopic pituitary surgery: a systematic review. Laryngoscope. 2010;120:1292–7.

Tabaee A, Anand VK, Barrón Y, Hiltzik DH, Brown SM, Kacker A, et al. Endoscopic pituitary surgery: a systematic review and meta-analysis. J Neurosurg. 2009;111:545–54.

Younus I, Gerges MM, Uribe-Cardenas R, Morgenstern PF, Eljalby M, Tabaee A, et al. How long is the tail end of the learning curve? Results from 1000 consecutive endoscopic endonasal skull base cases following the initial 200 cases. J Neurosurg. 2020;134:750–60.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88:105906.

Wells G, Shea B, O'Connell D, Peterson J, Welch V, Losos M, Tugwell P. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2013. Retrieved from http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp .

Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401–6. https://doi.org/10.1016/j.jclinepi.2010.07.015 .

Cappabianca P, Cavallo L, Colao A, Del Basso De Caro M, Esposito F, Cirillo S, et al. Endoscopic endonasal transsphenoidal approach: outcome analysis of 100 consecutive procedures. Minim Invasive Neurosurg. 2002;45:193–200.

Article   CAS   PubMed   Google Scholar  

Sonnenburg RE, White D, Ewend MG, Senior B. The learning curve in minimally invasive pituitary surgery. Am J Rhinol. 2004;18:259–63.

Kenan K, İhsan A, Dilek O, Burak C, Gurkan K, Savas C. The learning curve in endoscopic pituitary surgery and our experience. Neurosurg Rev. 2006;29:298–305.

Yano S, Kawano T, Kudo M, Makino K, Nakamura H, Kai Y, et al. Endoscopic endonasal transsphenoidal approach through the bilateral nostrils for pituitary adenomas. Neurol Med Chir (Tokyo). 2009;49:1–7.

Gondim JA, Schops M, de Almeida JPC, de Albuquerque LAF, Gomes E, Ferraz T, et al. Endoscopic endonasal transsphenoidal surgery: surgical results of 228 pituitary adenomas treated in a pituitary center. Pituitary. 2010;13:68–77.

Leach P, Abou-Zeid AH, Kearney T, Davis J, Trainer PJ, Gnanalingham KK. Endoscopic transsphenoidal pituitary surgery: evidence of an operative learning curve. Neurosurgery. 2010;67:1205–12.

Smith SJ, Eralil G, Woon K, Sama A, Dow G, Robertson I. Light at the end of the tunnel: the learning curve associated with endoscopic transsphenoidal skull base surgery. Skull Base. 2010;20:69–74.

Article   PubMed   PubMed Central   Google Scholar  

Wagenmakers MAE, Netea-Maier RT, van Lindert EJ, Pieters GF, Grotenhuis AJ, Hermus AR. Results of endoscopic transsphenoidal pituitary surgery in 40 patients with a growth hormone-secreting macroadenoma. Acta Neurochir (Wien). 2011;153:1391–9.

Kumar S, Darr A, Hobbs C, Carlin W. Endoscopic, endonasal, trans-sphenoidal hypophysectomy: retrospective analysis of 171 procedures. J Laryngol Otol. 2012;126:1033–40.

Snyderman CH, Pant H, Kassam AB, Carrau RL, Prevedello DM, Gardner PA. The learning curve for endonasal surgery of the cranial base: A systematic approach to training. In: Kassam AB, Gardner PA, editors. Endoscopic approaches to the skull base. Ettlingen: Karger Publishers; 2012. p. 222–31.

Chapter   Google Scholar  

Bokhari AR, Davies MA, Diamond T. Endoscopic transsphenoidal pituitary surgery: a single surgeon experience and the learning curve. Br J Neurosurg. 2013;27:449.

Chi F, Wang Y, Lin Y, Ge J, Qiu Y, Guo L. A learning curve of endoscopic transsphenoidal surgery for pituitary adenoma. J Craniofac Surg. 2013;24:2064–7.

de los Santos G, Fragola C, Del Castillo R, Rodríguez V, D’oleo C, Reyes P. Endoscopic approaches to pituitary lesions: difficulties and challenges. Acta Otorrinolaringol Esp. 2013;64(258):64.

Google Scholar  

Hazer DB, Işık S, Berker D, Güler S, Gürlek A, Yücel T, et al. Treatment of acromegaly by endoscopic transsphenoidal surgery: surgical experience in 214 cases and cure rates according to current consensus criteria. J Neurosurg. 2013;119:1467–77.

Jakimovski D, Bonci G, Attia M, Shao H, Hofstetter C, Tsiouris AJ, et al. Incidence and significance of intraoperative cerebrospinal fluid leak in endoscopic pituitary surgery using intrathecal fluorescein. World Neurosurg. 2014;82:e513–23.

Koutourousiou M, Fernandez-Miranda JC, Wang EW, Snyderman CH, Gardner PA. Endoscopic endonasal surgery for olfactory groove meningiomas: outcomes and limitations in 50 patients. Neurosurg Focus. 2014;37:E8.

Mascarenhas L, Moshel YA, Bayad F, Szentirmai O, Salek AA, Leng LZ, et al. The transplanum transtuberculum approaches for suprasellar and sellar-suprasellar lesions: avoidance of cerebrospinal fluid leak and lessons learned. World Neurosurg. 2014;82:186–95.

Ottenhausen M, Banu MA, Placantonakis DG, Tsiouris AJ, Khan OH, Anand VK, et al. Endoscopic endonasal resection of suprasellar meningiomas: the importance of case selection and experience in determining extent of resection, visual improvement, and complications. World Neurosurg. 2014;82:442–9.

Ananth G, Hosmath AV, Varadaraju DN, Patil SR, Usman MM, Patil RP, et al. Learning curve in endoscopic transnasal sellar region surgery. J Evid Based Med Healthc. 2016;3:3166–72.

Jang JH, Kim KH, Lee YM, Kim JS, Kim YZ. Surgical results of pure endoscopic endonasal transsphenoidal surgery for 331 pituitary adenomas: a 15-year experience from a single institution. World Neurosurg. 2016;96:545–55.

Kshettry VR, Do H, Elshazly K, Farrell CJ, Nyquist G, Rosen M, et al. The learning curve in endoscopic endonasal resection of craniopharyngiomas. Neurosurg Focus. 2016;41:E9.

Qureshi T, Chaus F, Fogg L, Dasgupta M, Straus D, Byrne RW. Learning curve for the transsphenoidal endoscopic endonasal approach to pituitary tumors. Br J Neurosurg. 2016;30:637–42.

Shou X, Shen M, Zhang Q, Zhang Y, He W, Ma Z, et al. Endoscopic endonasal pituitary adenomas surgery: the surgical experience of 178 consecutive patients and learning curve of two neurosurgeons. BMC Neurol. 2016;16:1–8.

Ding H, Gu Y, Zhang X, Xie T, Liu T, Hu F, et al. Learning curve for the endoscopic endonasal approach for suprasellar craniopharyngiomas. J Clin Neurosci. 2017;42:209–16.

Shikary T, Andaluz N, Meinzen-Derr J, Edwards C, Theodosopoulos P, Zimmer LA. Operative learning curve after transition to endoscopic transsphenoidal pituitary surgery. World Neurosurg. 2017;102:608–12.

Eseonu CI, ReFaey K, Pamias-Portalatin E, Asensio J, Garcia O, Boahene KD, et al. Three-hand endoscopic endonasal transsphenoidal surgery: experience with an anatomy-preserving mononostril approach technique. Oper Neurosurg (Hagerstown). 2018;14:158–65.

Kim JH, Lee JH, Lee JH, Hong AR, Kim YJ, Kim YH. Endoscopic transsphenoidal surgery outcomes in 331 nonfunctioning pituitary adenoma cases after a single surgeon learning curve. World Neurosurg. 2018;109:e409–16.

Lofrese G, Vigo V, Rigante M, Grieco DL, Maresca M, Anile C, et al. Learning curve of endoscopic pituitary surgery: experience of a neurosurgery/ENT collaboration. J Clin Neurosci. 2018;47:299–303.

Robins JM, Alavi SA, Tyagi AK, Nix PA, Wilson TM, Phillips NI. The learning curve for endoscopic trans-sphenoidal resection of pituitary macroadenomas. A single institution experience, Leeds, UK. Acta Neurochir (Wien). 2018;160:39–47.

Algattas H, Setty P, Goldschmidt E, Wang EW, Tyler-Kabara EC, Snyderman CH, et al. Endoscopic endonasal approach for craniopharyngiomas with intraventricular extension: case series, long-term outcomes, and review. World Neurosurg. 2020;144:e447–59.

Soliman MA, Eaton S, Quint E, Alkhamees AF, Shahab S, O’Connor A, et al. Challenges, learning curve, and safety of endoscopic endonasal surgery of sellar-suprasellar lesions in a community hospital. World Neurosurg. 2020;138:e940–54.

Nix P, Alavi SA, Tyagi A, Phillips N. Endoscopic repair of the anterior skull base-is there a learning curve? Br J Neurosurg. 2018;32:407–11.

Park W, Nam D-H, Kong D-S, Lee KE, Park SI, Kim HY, et al. Learning curve and technical nuances of endoscopic skull base reconstruction with nasoseptal flap to control high-flow cerebrospinal fluid leakage: reconstruction after endoscopic skull base surgery other than pituitary surgery. Eur Arch Otorhinolaryngol. 2022;279:1335–40.

Lubowitz JH, Sahasrabudhe A, Appleby D. Minimally invasive surgery in total knee arthroplasty: the learning curve. Orthopedics. 2007;30:80.

PubMed   Google Scholar  

Hoppe DJ, Simunovic N, Bhandari M, Safran MR, Larson CM, Ayeni OR. The learning curve for hip arthroscopy: a systematic review. Arthroscopy. 2014;30:389–97.

Sclafani JA, Kim CW. Complications associated with the initial learning curve of minimally invasive spine surgery: a systematic review. Clin Orthop Relat Res. 2014;472:1711–7.

Pernar LI, Robertson FC, Tavakkoli A, Sheu EG, Brooks DC, Smink DS. An appraisal of the learning curve in robotic general surgery. Surg Endosc. 2017;31:4583–96.

Wright TP. Factors affecting the cost of airplanes. J Aeronaut Sci. 1936;3:122–8.

Cook JA, Ramsay CR, Fayers P. Using the literature to quantify the learning curve: a case study. Int J Technol Assess Health Care. 2007;23:255–60.

Hopper A, Jamison M, Lewis W. Learning curves in surgical practice. Postgrad Med J. 2007;83:777–9.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Monk AF, Russell IT. Assessment of the learning curve in health technologies: a systematic review. Int J Technol Assess Health Care. 2000;16:1095–108.

Khan N, Abboudi H, Khan MS, Dasgupta P, Ahmed K. Measuring the surgical ‘learning curve’: methods, variables and competency. BJU Int. 2014;113:504–8.

Download references

Acknowledgements

Not applicable

No funding was received for this study.

Author information

Authors and affiliations.

Neurosurgery Department, East Jeddah Hospital, 2277 King Abdullah Rd, Al Sulaymaniyah, 22253, Jeddah, Saudi Arabia

Abdulraheem Alomari

Otolaryngology and Head and Neck Surgery Department, King Abdullah Medical Complex, Prince Nayef Street, Northern Abhor, 23816, Jeddah, Saudi Arabia

Mazin Alsarraj

Neurosurgery Department, King Abdulaziz Medical City, 21423, Jeddah, Saudi Arabia

Sarah Alqarni

You can also search for this author in PubMed   Google Scholar

Contributions

AA: Acquisition of Data, Analysis and Interpretation of Data, Drafting the Article, Critically Revising the Article, Drafting the Article, Reviewed submitted version of manuscript, Approved the final version of the manuscript on behalf of all authors. Study supervision; MA: Conception and Design, Analysis and Interpretation of Data, Reviewed submitted version of manuscript, Analysis and Interpretation of Data; SA: Acquisition of Data, Conception and Design, Critically Revising the Article, Analysis and Interpretation of Data, Administrative / technical / material support.

Corresponding author

Correspondence to Abdulraheem Alomari .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Alomari, A., Alsarraj, M. & Alqarni, S. The learning curve in endoscopic transsphenoidal skull-base surgery: a systematic review. BMC Surg 24 , 135 (2024). https://doi.org/10.1186/s12893-024-02418-y

Download citation

Received : 21 September 2023

Accepted : 20 April 2024

Published : 05 May 2024

DOI : https://doi.org/10.1186/s12893-024-02418-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Endoscopic skull base
• Transsphenoidal surgery

BMC Surgery

ISSN: 1471-2482

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Evaluation of online job portals for HR recruitment selection using AHP in two wheeler automotive industry: a case study

• ORIGINAL ARTICLE
• Published: 12 May 2024

Cite this article

• S. M. Vadivel   ORCID: orcid.org/0000-0002-5287-3693 1 &
• Rohan Sunny   ORCID: orcid.org/0009-0002-2347-3081 2  

Automotive companies are booming worldwide in the economy. In order to sustain in the highly competitive world, every organization tries to create itself a trademark in the market. In our research, we looked at how two wheelers automotive company's selection enhances an organizational performance, which ensures the company's future growth. In today's fast-paced, globally integrated world, human resources are one of the most important production variables. It is critical to preserve and improve economic competitiveness by properly selecting and developing these resources. The main aim of this study is to identify the best online job portal website for recruitment at Two Wheeler Company and to suggest an HR strategy which resonates company’s values and culture. In this study, we have selected 6 criteria and 6 online popular job portals for recruitment with a sample of 15 candidates have been selected. Findings reveal that, AHP method has significant results on the selection of best employer, which helps HR Manager to finalize the decision making process/strategies. Towards the managerial implications section, the researcher aims to design an functional and effective HR strategy that can grasp, engage and retain the top talent in the organization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Availability of data and material

'Not applicable' in this section.

Abbreviations

Analytic hierarchy process

Artificial intelligence

Analysis of variance

Chief Human Resources Officer

Consistency index

Curriculum vitae

Consistency ratio

Decision making

Faculty Development Programme

Hierarchical linear modelling

Human resources

Research and Development

Randomized index

Structural equation modelling

Search engine optimization

Triple bottom line

Technique for order preference by similarity

Maximum Eigen value

The normalized value of ith criterion for the jth alternative

The normalized value of jth criterion for the ith alternative

The number of alternatives for a certain MCDM problem

The number of criteria for a certain MCDM problem

Avinash Kapse S, Vishal Patil S, Nikhil PV (2021) E-Recruitment. Int J Eng Adv Technol (IJEAT) 1(4):82–86

Google Scholar  

Chahar B, Jain SR, Hatwal V (2021) Mediating role of employee motivation for training, commitment, retention, and performance in higher education institutions. Probl Perspect Manag 19(3):95

Chauhan P (2019) Impact of training and development programs on motivation of employees in “A” graded commercial bank of Nepal. Int J Res Anal Rev 6(3):850–857

Elsafty A, Oraby M (2022) The impact of training on employee retention: an empirical research on the Private Sector in Egypt. Int J Bus Manage 17(5):58–74

Article   Google Scholar  

Habibie M, Mustika I (2020) The effect of training on work motivation and its impact on employee performance (Case Study at BPJS Ketenagakerjaan Headquarters). Int J Innovat Sci Res Technol 5(7):51–57

Haryono S, Supardi S, Udin U (2020) The effect of training and job promotion on work motivation and its implications on job performance: evidence from Indonesia. Manage Sci Lett 10(9):2107–2112

Horodyski P (2023a) Recruiter’s perception of artificial intelligence (AI)-based tools in recruitment. Comp Human Behav Reports 10:100298

Horodyski P (2023b) Applicants’ perception of artificial intelligence in the recruitment process. Comp Human Behav Reports 11:100303

Jack Walker H, Feild HS, Giles WF, Bernerth JB, Short JC (2011) ‘so what do you think of the organization? a contextual priming explanation for recruitment web site characteristics as antecedents of job seekers’ organizational image perceptions. Organ Behav Human Decis Process 2(2011):165–178

Khan N (2018) Does training & employee commitment predict employee retention. In: International Conference on Management and Information Systems (Vol. 21, pp. 120–124)

Lee I (2005) The evolution of E-Recruiting. A content analysis of Fortune 100 Career Websites. J Electronic Commerce Organ 3(3):57–68

Lievens F, Harris MM (2003) Research on Internet recruiting and testing: current status and future directions. Int Rev Ind Organ Psychol 16:131–165

Lin CY, Huang CK (2021) Employee turnover intentions and job performance from a planned change: the effects of an organizational learning culture and job satisfaction. Int J Manpow 42(3):409–423

Martins D, Diaconescu LM (2014) Expatriates recruitment and selection for long-term international assignments in Portuguese companies. Tékhne 12:48–57

RoyChowdhury T, Srimannarayana M (2013) Applicants’ perceptions on online recruitment procedures. Manage Labour Stud 38(3):185–199

Ryu G, Moon SG (2019) The effect of actual workplace learning on job satisfaction and organizational commitment: The moderating role of intrinsic learning motive. J Workplace Learn 31(8):481–497

Saaty TL (1990) How to make a decision: the analytic hierarchy process. Eur J Oper Res 48(1):9–26

Sengazhani Murugesan V, Sequeira AH, Shetty DS, Jauhar SK (2020) Enhancement of mail operational performance of India post facility layout using AHP. Int J Syst Assur Eng Manage 11(2):261–273

Sigalingging H, Pakpahan ME (2021) The Effect of training and work environment on employee performance with motivation as an intervening variable At PT Intraco Agroindustry. South East Asia Journal of Contemporary Business. Econ Law 24(6):130–139.

Sharawat K, Dubey SK (2018) An approach to vendor selection on usability basis by AHP and fuzzy topsis method. In: Soft computing: theories and applications: proceedings of SoCTA 2016, vol 2. Springer, Singapore, pp 595–604

Steil AV, de Cuffa D, Iwaya GH, Pacheco RCDS (2020) Perceived learning opportunities, behavioral intentions and employee retention in technology organizations. J Work Learn 32(2):147–159

Sugiarti E (2022) The influence of training, work environment and career development on work motivation that has an impact on employee performance at PT. Suryamas Elsindo Primatama In West Jakarta. Int J Artif Intell Res 6(1.2).

Sumrit D (2020) Supplier selection for vendor-managed inventory in healthcare using fuzzy multi-criteria decision-making approach. Decis Sci Lett 9(2):233–256

Thompson LF, Braddy PW, Wuensch KL (2008) E-recruitment and the benefits of organizational web appeal. Comp Human Behav 24(5):2384–2398

Turan FK, Scala NM, Sacre MB, Needy KL (2009) An Analytic Network Process (ANP) approach to the project portfolio management for Organizational Sustainability. In: Proceedings of the Industrial Engineering Research Conference. Institute of Industrial Engineers.

Wadhawan S, Sinha S (2018) Factors Influencing Young Job Seekers Perception towards Job Portals. AIMS Int J Manage 12(3).

Weerarathna RS, Somawardana WSD (2021) Impact on training and employee motivation in an electricity company. Future Work, 497.

Zusman R, Landis R (2002) Applicant preferences for Web-based versus traditional job postings. Comput Hum Behav 18(3):285–329

Download references

Acknowledgements

The authors would like to express their gratitude to two wheeler Automotive Industries in Chennai, Tamil Nadu, India, for their invaluable assistance and cooperation. We greatly acknowledge Ms. Ruchi Mishra, Research scholar from NIT Karnataka, for editing this manuscript in better form.

There is no funding provided in this research.

Author information

Authors and affiliations.

Operations Management Division, Vellore Institute of Technology Chennai, Vandalur-Kelambakkam Road, Chennai, 600127, India

S. M. Vadivel

Vellore Institute of Technology Chennai, Vandalur-Kelambakkam Road, Chennai, 600127, India

Rohan Sunny

You can also search for this author in PubMed   Google Scholar

Contributions

S M Vadivel: Methodology, Writing—review & editing, Supervision. Rohan Sunny: Data Curation, Writing—original draft preparation.

Corresponding author

Correspondence to S. M. Vadivel .

Ethics declarations

Conflict of interests.

The authors declare that they have no competing interests.

Ethics approval and consent to participate

This manuscript has a research study involves human participants (Interview Candidates) for studying job portal evaluations in Indian two wheeler company running in Chennai, Tamil Nadu.

Consent for publication

‘ Not applicable’ in this section.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Vadivel, S.M., Sunny, R. Evaluation of online job portals for HR recruitment selection using AHP in two wheeler automotive industry: a case study. Int J Syst Assur Eng Manag (2024). https://doi.org/10.1007/s13198-024-02358-z

Download citation

Received : 04 December 2023

Revised : 07 April 2024

Accepted : 25 April 2024

Published : 12 May 2024

DOI : https://doi.org/10.1007/s13198-024-02358-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Analytical hierarchy process (AHP)
• Online job portals
• Automobile two wheeler industries
• Artificial intelligence (AI)
• HR recruitment
• Find a journal
• Publish with us
• Track your research