hypothesis for retrospective study

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Retrospective Study: Definition & Examples

By Jim Frost 1 Comment

What is a Retrospective Study?

A retrospective study an experimental design that looks back in time and assesses events that have already occurred. The researchers already know the outcome for each subject when the project starts. Instead of recording data going forward as events happen, these studies use participant recollection and data that were previously recorded for reasons not relating to the project. These studies typically don’t follow patients into the future.

In retrospective designs, the researchers collect their data using existing records. Consequently, they can complete their assessment more quickly and inexpensively than a prospective study that must follow subjects over time and record the data under carefully controlled conditions. However, the data that a retrospective study uses might not have been measured consistently or accurately because they weren’t explicitly designed to be part of a study.

The statistical analysis for a retrospective study is frequently the same as for prospective designs (looking forward). The main difference is that the project occurs after the outcomes are known rather than how researchers analyze the data.

Statisticians consider retrospective designs to be inferior to prospective methods because they tend to introduce more bias and confounding. Retrospective studies are observational studies by necessity because they assess past events and it is impossible to perform a randomized, controlled experiment with them. However, they can be quicker and cheaper to complete, making them a good choice for preliminary research.  Findings from a retrospective study can help inform a prospective experimental design. Learn more about Experimental Designs .

Retrospective Study Designs

Retrospective studies use various designs. While these designs differ in detail, they all tend to compare subjects with and without a condition and determine how they differ. Using the usual hypothesis tests, researchers can determine whether there are statistically significant relationships between subject variables (risk factors , personal characteristics, etc.) and the outcome of interest.

Cohort and case-control studies are standard retrospective designs. Let’s learn more about them!

Retrospective Cohort Study

This study design compares groups of subjects who are similar overall but differ in a particular characteristic, such as exposure to a risk factor. Because it is a retrospective study, the researchers find individuals where the outcomes are known when the project starts. Retrospective cohort studies frequently determine whether exposure to risk and protective factors affects an outcome. These are longitudinal studies that use existing datasets to look back at events that have already occurred. Learn more about Longitudinal Studies: Overview, Examples & Benefits .

In these projects, researchers use databases and medical records to identify patients and gather information about them. They can also ask subjects to recall their exposure over time. Then the researchers analyze the data to determine whether the risk factor correlates with the outcome of interest.

Suppose researchers hypothesize that exposure to a chemical increases skin cancer and conduct a retrospective cohort study. In that case, they can form a cohort based on a group commonly exposed to that chemical (e.g., a particular job). Then they access medical databases and records to collect their data. After identifying their subjects and obtaining the medical information, they can immediately analyze the data, comparing the outcomes for those with and without exposure.

Learn more about Cohort Studies .

Case-Control Studies

Case-control designs are generally retrospective studies. Like their cohort counterparts, case-control studies compare two groups of people, those with and without a condition. These designs both assess risk and protective factors.

Retrospective cohort and case-control studies are similar but generally have differing goals. Cohort designs typically assess known risk factors and how they affect outcomes at different times. Case-control studies evaluate a particular incident, and it is an exploratory design to identify potential risk factors.

For example, a case-control assessment might evaluate an episode of severe illness occurring after a company picnic to identify potential food culprits.

Learn more about Case-Control Studies .

Advantages of a Retrospective Study

A retrospective study tends to have the following advantages compared to a prospective design:

Cheaper : You don’t need a lab or equipment to measure information. Others did that for you!

Faster : The events have already occurred in a retrospective study—no need to wait for them to happen and then look for the differences between the groups.

Great for rare diseases : You can specifically look through a database for individuals with a rare disease or condition. In a prospective experiment, you need an immense sample size and hope enough of the rare outcomes occur for you to analyze.

Disadvantages of a Retrospective Study

Unfortunately, they tend to have the following disadvantages relating to a greater propensity for inaccuracies, inconsistencies, lack of controlled conditions, and bias:

• A retrospective study uses data measured for other purposes.
• Different people, procedures, and equipment might have recorded the data, leading to inconsistencies.
• Measurements might have occurred under differing conditions.
• Control variables might not be measured, leading to confounding.
• Recall bias.

Dean R Hess, Retrospective Studies and Chart Reviews , Respiratory Care , October 2004, 49 (10) 1171-1174.

Share this:

Reader Interactions

November 7, 2022 at 8:26 am

Coincidentally, I just read this Israeli retrospective cohort study regarding the incidence of myocarditis and pericarditis in unvaxxed post-COVID-19 patients: https://pubmed.ncbi.nlm.nih.gov/35456309/

Good news for a change.

Comments and Questions Cancel reply

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Retrospective Cohort Study? | Definition & Examples

What Is a Retrospective Cohort Study? | Definition & Examples

Published on February 10, 2023 by Tegan George . Revised on June 22, 2023.

A retrospective cohort study is a type of observational study that focuses on individuals who have an exposure to a disease or risk factor in common. Retrospective cohort studies analyze the health outcomes over a period of time to form connections and assess the risk of a given outcome associated with a given exposure.

It is crucial to note that in order to be considered a retrospective cohort study, your participants must already possess the disease or health outcome being studied.

Table of contents

When to use a retrospective cohort study, examples of retrospective cohort studies, advantages and disadvantages of retrospective cohort studies, other interesting articles, frequently asked questions.

Retrospective cohort studies are a type of observational study . They are often used in fields related to medicine to study the effect of exposures on health outcomes. While most observational studies are qualitative in nature, retrospective cohort studies are often quantitative , as they use preexisting secondary research data. They can be used to conduct both exploratory research and explanatory research .

Retrospective cohort studies are often used as an intermediate step between a weaker preliminary study and a prospective cohort study , as the results gleaned from a retrospective cohort study strengthen assumptions behind a future prospective cohort study.

A retrospective cohort study could be a good fit for your research if:

• A prospective cohort study is not (yet) feasible for the variables you are investigating.
• You need to quickly examine the effect of an exposure, outbreak, or treatment on an outcome.
• You are seeking to investigate an early-stage or potential association between your variables of interest.

Retrospective cohort studies use secondary research data, such as existing medical records or databases, to identify a group of people with an exposure or risk factor in common. They then look back in time to observe how the health outcomes developed. Case-control studies rely on primary research , comparing a group of participants with a condition of interest to a group lacking that condition in real time.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Retrospective cohort studies are common in fields like medicine, epidemiology, and healthcare.

You collect data from participants’ exposure to organophosphates, focusing on variables like the timing and duration of exposure, and analyze the health effects of the exposure. Example: Healthcare retrospective cohort study You are examining the relationship between tanning bed use and the incidence of skin cancer diagnoses.

Retrospective cohort studies can be a good fit for many research projects, but they have their share of advantages and disadvantages.

Advantages of retrospective cohort studies

• Retrospective cohort studies are a great choice if you have any ethical considerations or concerns about your participants that prevent you from pursuing a traditional experimental design .
• Retrospective cohort studies are quite efficient in terms of time and budget. They require fewer subjects than other research methods and use preexisting secondary research data to analyze them.
• Retrospective cohort studies are particularly useful when studying rare or unusual exposures, as well as diseases with a long latency or incubation period where prospective cohort studies cannot yet form conclusions.

Disadvantages of retrospective cohort studies

• Like many observational studies, retrospective cohort studies are at high risk for many research biases . They are particularly at risk for recall bias and observer bias due to their reliance on memory and self-reported data.
• Retrospective cohort studies are not a particularly strong standalone method, as they can never establish causality . This leads to low internal validity and external validity .
• As most patients will have had a range of healthcare professionals involved in their care over their lifetime, there is significant variability in the measurement of risk factors and outcomes. This leads to issues with reliability and credibility of data collected.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

The primary difference between a retrospective cohort study and a prospective cohort study is the timing of the data collection and the direction of the study.

A retrospective cohort study looks back in time. It uses preexisting secondary research data to examine the relationship between an exposure and an outcome. Data is collected after the outcome you’re studying has already occurred.

Alternatively, a prospective cohort study follows a group of individuals over time. It collects data on both the exposure and the outcome of interest as they are occurring. Data is collected before the outcome of interest has occurred.

Retrospective cohort studies are at high risk for research biases like recall bias . Whenever individuals are asked to recall past events or exposures, recall bias can occur. This is because individuals with a certain disease or health outcome of interest are more likely to remember and/or report past exposures differently to individuals without that outcome. This can result in an overestimation or underestimation of the true relationship between variables and affect your research.

No, retrospective cohort studies cannot establish causality on their own.

Like other types of observational studies , retrospective cohort studies can suggest associations between an exposure and a health outcome. They cannot prove without a doubt, however, that the exposure studied causes the health outcome.

In particular, retrospective cohort studies suffer from challenges arising from the timing of data collection , research biases like recall bias , and how variables are selected. These lead to low internal validity and the inability to determine causality.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). What Is a Retrospective Cohort Study? | Definition & Examples. Scribbr. Retrieved August 12, 2024, from https://www.scribbr.com/methodology/retrospective-cohort-study/

Is this article helpful?

Tegan George

Other students also liked, what is a case-control study | definition & examples, what is an observational study | guide & examples, what is recall bias | definition & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

About Stanford GSB

• The Leadership
• Dean’s Updates
• School News & History
• Commencement
• Business, Government & Society
• Centers & Institutes
• Center for Entrepreneurial Studies
• Center for Social Innovation
• Stanford Seed

About the Experience

• Learning at Stanford GSB
• Experiential Learning
• Guest Speakers
• Entrepreneurship
• Social Innovation
• Communication
• Life at Stanford GSB
• Collaborative Environment
• Activities & Organizations
• Student Services
• Housing Options
• International Students

Full-Time Degree Programs

• Why Stanford MBA
• Academic Experience
• Financial Aid
• Why Stanford MSx
• Research Fellows Program
• See All Programs

Non-Degree & Certificate Programs

• Executive Education
• Stanford Executive Program
• Programs for Organizations
• The Difference
• Online Programs
• Stanford LEAD
• Seed Transformation Program
• Aspire Program
• Seed Spark Program
• Faculty Profiles
• Academic Areas
• Awards & Honors
• Conferences

Faculty Research

• Publications
• Working Papers
• Case Studies

Research Hub

• Research Labs & Initiatives
• Business Library
• Data, Analytics & Research Computing
• Behavioral Lab

Research Labs

• Cities, Housing & Society Lab
• Golub Capital Social Impact Lab

Research Initiatives

• Corporate Governance Research Initiative
• Corporations and Society Initiative
• Policy and Innovation Initiative
• Rapid Decarbonization Initiative
• Stanford Latino Entrepreneurship Initiative
• Value Chain Innovation Initiative
• Venture Capital Initiative
• Career & Success
• Climate & Sustainability
• Corporate Governance
• Culture & Society
• Finance & Investing
• Government & Politics
• Leadership & Management
• Markets and Trade
• Operations & Logistics
• Opportunity & Access
• Technology & AI
• Opinion & Analysis
• Email Newsletter

Welcome, Alumni

• Communities
• Digital Communities & Tools
• Regional Chapters
• Women’s Programs
• Identity Chapters
• Find Your Reunion
• Career Resources
• Job Search Resources
• Career & Life Transitions
• Programs & Webinars
• Career Video Library
• Alumni Education
• Research Resources
• Volunteering
• Alumni News
• Class Notes
• Alumni Voices
• Contact Alumni Relations
• Upcoming Events

Admission Events & Information Sessions

• MBA Program
• MSx Program
• PhD Program
• Alumni Events
• All Other Events
• Operations, Information & Technology
• Organizational Behavior
• Political Economy
• Classical Liberalism
• The Eddie Lunch
• Accounting Summer Camp
• California Econometrics Conference
• California Quantitative Marketing PhD Conference
• California School Conference
• China India Insights Conference
• Homo economicus, Evolving
• Political Economics (2023–24)
• Scaling Geologic Storage of CO2 (2023–24)
• A Resilient Pacific: Building Connections, Envisioning Solutions
• Adaptation and Innovation
• Changing Climate
• Civil Society
• Climate Impact Summit
• Climate Science
• Corporate Carbon Disclosures
• Earth’s Seafloor
• Environmental Justice
• Operations and Information Technology
• Organizations
• Sustainability Reporting and Control
• Taking the Pulse of the Planet
• Urban Infrastructure
• Watershed Restoration
• Junior Faculty Workshop on Financial Regulation and Banking
• Ken Singleton Celebration
• Marketing Camp
• Quantitative Marketing PhD Alumni Conference
• Presentations
• Theory and Inference in Accounting Research
• Stanford Closer Look Series
• Quick Guides
• Core Concepts
• Journal Articles
• Glossary of Terms
• Faculty & Staff
• Researchers & Students
• Research Approach
• Charitable Giving
• Financial Health
• Government Services
• Workers & Careers
• Short Course
• Adaptive & Iterative Experimentation
• Incentive Design
• Social Sciences & Behavioral Nudges
• Bandit Experiment Application
• Conferences & Events
• Get Involved
• Reading Materials
• Teaching & Curriculum
• Energy Entrepreneurship
• Faculty & Affiliates
• SOLE Report
• Responsible Supply Chains
• Current Study Usage
• Pre-Registration Information
• Participate in a Study

A How-To Guide for Conducting Retrospective Analyses: Example COVID-19 Study

In the urgent setting of the COVID-19 pandemic, treatment hypotheses abound, each of which requires careful evaluation. A randomized controlled trial generally provides the strongest possible evaluation of a treatment, but the efficiency and effectiveness of the trial depend on the existing evidence supporting the treatment. The researcher must therefore compile a body of evidence justifying the use of time and resources to further investigate a treatment hypothesis in a trial. An observational study can help provide this evidence, but the lack of randomized exposure and the researcher’s inability to control treatment administration and data collection introduce significant challenges for non-experimental studies. A proper analysis of observational health care data thus requires an extensive background in a diverse set of topics ranging from epidemiology and causal analysis to relevant medical specialties and data sources. Here we provide 10 rules that serve as an end-to-end introduction to retrospective analyses of observational health care data. A running example of a COVID-19 study presents a practical implementation of each rule in the context of a specific treatment hypothesis. When carefully designed and properly executed, a retrospective analysis framed around these rules will inform the decisions of whether and how to investigate a treatment hypothesis in a randomized controlled trial.

• See the Current DEI Report
• Supporting Data
• Research & Insights
• Share Your Thoughts
• Search Fund Primer
• Affiliated Faculty
• Faculty Advisors
• Louis W. Foster Resource Center
• Defining Social Innovation
• Impact Compass
• Global Health Innovation Insights
• Faculty Affiliates
• Student Awards & Certificates
• Changemakers
• Dean Jonathan Levin
• Dean Garth Saloner
• Dean Robert Joss
• Dean Michael Spence
• Dean Robert Jaedicke
• Dean Rene McPherson
• Dean Arjay Miller
• Dean Ernest Arbuckle
• Dean Jacob Hugh Jackson
• Dean Willard Hotchkiss
• Faculty in Memoriam
• Stanford GSB Firsts
• Class of 2024 Candidates
• Certificate & Award Recipients
• Dean’s Remarks
• Keynote Address
• Teaching Approach
• Analysis and Measurement of Impact
• The Corporate Entrepreneur: Startup in a Grown-Up Enterprise
• Data-Driven Impact
• Designing Experiments for Impact
• Digital Marketing
• The Founder’s Right Hand
• Marketing for Measurable Change
• Product Management
• Public Policy Lab: Financial Challenges Facing US Cities
• Public Policy Lab: Homelessness in California
• Lab Features
• Curricular Integration
• View From The Top
• Formation of New Ventures
• Managing Growing Enterprises
• Startup Garage
• Explore Beyond the Classroom
• Stanford Venture Studio
• Summer Program
• Workshops & Events
• The Five Lenses of Entrepreneurship
• Leadership Labs
• Executive Challenge
• Arbuckle Leadership Fellows Program
• Selection Process
• Training Schedule
• Time Commitment
• Learning Expectations
• Post-Training Opportunities
• Who Should Apply
• Introductory T-Groups
• Leadership for Society Program
• Certificate
• 2024 Awardees
• 2023 Awardees
• 2022 Awardees
• 2021 Awardees
• 2020 Awardees
• 2019 Awardees
• 2018 Awardees
• Social Management Immersion Fund
• Stanford Impact Founder Fellowships
• Stanford Impact Leader Prizes
• Social Entrepreneurship
• Stanford GSB Impact Fund
• Economic Development
• Energy & Environment
• Stanford GSB Residences
• Environmental Leadership
• Stanford GSB Artwork
• A Closer Look
• California & the Bay Area
• Voices of Stanford GSB
• Business & Beneficial Technology
• Business & Sustainability
• Business & Free Markets
• Business, Government, and Society Forum
• Second Year
• Global Experiences
• JD/MBA Joint Degree
• MA Education/MBA Joint Degree
• MD/MBA Dual Degree
• MPP/MBA Joint Degree
• MS Computer Science/MBA Joint Degree
• MS Electrical Engineering/MBA Joint Degree
• MS Environment and Resources (E-IPER)/MBA Joint Degree
• Academic Calendar
• Clubs & Activities
• LGBTQ+ Students
• Military Veterans
• Minorities & People of Color
• Partners & Families
• Students with Disabilities
• Student Support
• Residential Life
• Student Voices
• MBA Alumni Voices
• A Week in the Life
• Career Support
• Employment Outcomes
• Cost of Attendance
• Knight-Hennessy Scholars Program
• Yellow Ribbon Program
• BOLD Fellows Fund
• Application Process
• Loan Forgiveness
• Contact the Financial Aid Office
• Evaluation Criteria
• GMAT & GRE
• English Language Proficiency
• Personal Information, Activities & Awards
• Professional Experience
• Letters of Recommendation
• Optional Short Answer Questions
• Application Fee
• Reapplication
• Deferred Enrollment
• Joint & Dual Degrees
• Entering Class Profile
• Event Schedule
• Ambassadors
• New & Noteworthy
• Ask a Question
• See Why Stanford MSx
• Is MSx Right for You?
• MSx Stories
• Leadership Development
• How You Will Learn
• Admission Events
• Personal Information
• GMAT, GRE & EA
• English Proficiency Tests
• Career Change
• Career Advancement
• Daycare, Schools & Camps
• U.S. Citizens and Permanent Residents
• Requirements
• Requirements: Behavioral
• Requirements: Quantitative
• Requirements: Macro
• Requirements: Micro
• Annual Evaluations
• Field Examination
• Research Activities
• Research Papers
• Dissertation
• Oral Examination
• Current Students
• Education & CV
• International Applicants
• Statement of Purpose
• Reapplicants
• Application Fee Waiver
• Deadline & Decisions
• Job Market Candidates
• Academic Placements
• Stay in Touch
• Faculty Mentors
• Current Fellows
• Standard Track
• Fellowship & Benefits
• Group Enrollment
• Program Formats
• Developing a Program
• Diversity & Inclusion
• Strategic Transformation
• Program Experience
• Contact Client Services
• Campus Experience
• Live Online Experience
• Silicon Valley & Bay Area
• Digital Credentials
• Faculty Spotlights
• Participant Spotlights
• Eligibility
• International Participants
• Stanford Ignite
• Frequently Asked Questions
• Founding Donors
• Location Information
• Participant Profile
• Network Membership
• Program Impact
• Collaborators
• Entrepreneur Profiles
• Company Spotlights
• Seed Transformation Network
• Responsibilities
• Current Coaches
• How to Apply
• Meet the Consultants
• Meet the Interns
• Intern Profiles
• Collaborate
• Research Library
• News & Insights
• Program Contacts
• Databases & Datasets
• Research Guides
• Consultations
• Research Workshops
• Career Research
• Research Data Services
• Course Reserves
• Course Research Guides
• Material Loan Periods
• Fines & Other Charges
• Document Delivery
• Interlibrary Loan
• Equipment Checkout
• Print & Scan
• MBA & MSx Students
• PhD Students
• Other Stanford Students
• Faculty Assistants
• Research Assistants
• Stanford GSB Alumni
• Telling Our Story
• Staff Directory
• Site Registration
• Alumni Directory
• Alumni Email
• Privacy Settings & My Profile
• Success Stories
• The Story of Circles
• Support Women’s Circles
• Stanford Women on Boards Initiative
• Alumnae Spotlights
• Insights & Research
• Industry & Professional
• Entrepreneurial Commitment Group
• Recent Alumni
• Half-Century Club
• Fall Reunions
• Spring Reunions
• MBA 25th Reunion
• Half-Century Club Reunion
• Faculty Lectures
• Ernest C. Arbuckle Award
• Alison Elliott Exceptional Achievement Award
• ENCORE Award
• Excellence in Leadership Award
• John W. Gardner Volunteer Leadership Award
• Robert K. Jaedicke Faculty Award
• Jack McDonald Military Service Appreciation Award
• Jerry I. Porras Latino Leadership Award
• Tapestry Award
• Student & Alumni Events
• Executive Recruiters
• Interviewing
• Land the Perfect Job with LinkedIn
• Negotiating
• Elevator Pitch
• Email Best Practices
• Resumes & Cover Letters
• Self-Assessment
• Whitney Birdwell Ball
• Margaret Brooks
• Bryn Panee Burkhart
• Margaret Chan
• Ricki Frankel
• Peter Gandolfo
• Cindy W. Greig
• Natalie Guillen
• Carly Janson
• Sloan Klein
• Sherri Appel Lassila
• Stuart Meyer
• Tanisha Parrish
• Virginia Roberson
• Philippe Taieb
• Michael Takagawa
• Terra Winston
• Johanna Wise
• Debbie Wolter
• Rebecca Zucker
• Complimentary Coaching
• Changing Careers
• Work-Life Integration
• Career Breaks
• Flexible Work
• Encore Careers
• Join a Board
• D&B Hoovers
• Data Axle (ReferenceUSA)
• EBSCO Business Source
• Global Newsstream
• Market Share Reporter
• ProQuest One Business
• RKMA Market Research Handbook Series
• Student Clubs
• Entrepreneurial Students
• Stanford GSB Trust
• Alumni Community
• How to Volunteer
• Springboard Sessions
• Consulting Projects
• 2020 – 2029
• 2010 – 2019
• 2000 – 2009
• 1990 – 1999
• 1980 – 1989
• 1970 – 1979
• 1960 – 1969
• 1950 – 1959
• 1940 – 1949
• Service Areas
• ACT History
• ACT Awards Celebration
• ACT Governance Structure
• Building Leadership for ACT
• Individual Leadership Positions
• Leadership Role Overview
• Purpose of the ACT Management Board
• Contact ACT
• Business & Nonprofit Communities
• Reunion Volunteers
• Ways to Give
• Fiscal Year Report
• Business School Fund Leadership Council
• Planned Giving Options
• Planned Giving Benefits
• Planned Gifts and Reunions
• Legacy Partners
• Giving News & Stories
• Giving Deadlines
• Development Staff
• Submit Class Notes
• Class Secretaries
• Board of Directors
• Health Care
• Sustainability
• Class Takeaways
• All Else Equal: Making Better Decisions
• If/Then: Business, Leadership, Society
• Grit & Growth
• Think Fast, Talk Smart
• Spring 2022
• Spring 2021
• Autumn 2020
• Summer 2020
• Winter 2020
• In the Media
• For Journalists
• DCI Fellows
• Other Auditors
• Academic Calendar & Deadlines
• Course Materials
• Entrepreneurial Resources
• Campus Drive Grove
• Campus Drive Lawn
• CEMEX Auditorium
• King Community Court
• Seawell Family Boardroom
• Stanford GSB Bowl
• Stanford Investors Common
• Town Square
• Vidalakis Courtyard
• Vidalakis Dining Hall
• Catering Services
• Policies & Guidelines
• Reservations
• Contact Faculty Recruiting
• Lecturer Positions
• Postdoctoral Positions
• Accommodations
• CMC-Managed Interviews
• Recruiter-Managed Interviews
• Virtual Interviews
• Campus & Virtual
• Search for Candidates
• Think Globally
• Recruiting Calendar
• Recruiting Policies
• Full-Time Employment
• Summer Employment
• Entrepreneurial Summer Program
• Global Management Immersion Experience
• Social-Purpose Summer Internships
• Process Overview
• Project Types
• Client Eligibility Criteria
• Client Screening
• ACT Leadership
• Social Innovation & Nonprofit Management Resources
• Develop Your Organization’s Talent
• Centers & Initiatives
• Student Fellowships

• Molecular Biology

A how-to guide for conducting retrospective analyses: example COVID-19 study

• September 2020
• This person is not on ResearchGate, or hasn't claimed this research yet.

• University of Michigan

• Johns Hopkins University

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Elizabeth A. Stuart

• Daniel F McAuley
• Michael Brown
• Jessica Manson

• Xabier García-Albéniz
• Roger Logan
• Miguel A Hernán
• Verena Staedtke

• J ECONOMETRICS

• Anna C. Greene
• Deepashree Venkatesh Prasad

• Charles Piller
• John Travis

• Yuani M. Roman

• Michael Powell
• Chetan Bettegowda

• Jonathan Kimmelman
• Susan Carleton Athey

• Stefan Wager
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Retrospective Cohort Study: Definition & Examples

Julia Simkus

Editor at Simply Psychology

BA (Hons) Psychology, Princeton University

Julia Simkus is a graduate of Princeton University with a Bachelor of Arts in Psychology. She is currently studying for a Master's Degree in Counseling for Mental Health and Wellness in September 2023. Julia's research has been published in peer reviewed journals.

Learn about our Editorial Process

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

A retrospective study, sometimes called a historical cohort study, is a type of longitudinal study in which researchers look back to a certain point to analyze a particular group of subjects who have already experienced an outcome of interest.

In a retrospective cohort study, the researcher identifies a group of individuals who have been exposed to a certain factor and a group who have not been exposed (the cohorts), and then looks back in time to see how the rate of a certain outcome (like the development of a disease) differs between the two groups.

For example, a researcher might identify a group of people who smoked and a group who never smoked, and then look back at medical records to see how the rate of lung cancer differs between the two groups.

This type of study is beneficial for medical researchers, specifically in epidemiology, as scientists can use existing data to understand potential risk factors or causes of disease.

Researchers in retrospective studies will identify a cohort of subjects before they have developed a disease and then use existing data, such as medical records, to discover any patterns and examine exposures to suspected risks.

In cohort studies , one group of participants must share a common exposure factor, and this group is compared to another group of participants who do not share the exposure to that factor.

For example, men over age 60 who exercise daily could be compared to men over age 60 who do not exercise daily (control) to study the prevalence of diabetes in men over 60.

Researchers collect data from existing records to study a relationship and determine the influence of a particular factor (i.e., daily exercise) on a particular outcome (i.e., diabetes) and to analyze the relative risk of the cohort compared to the control group.

Feasibility

Estimating the relative risk of a population tends to be easier with retrospective studies than prospective studies. Retrospective studies are conducted on a smaller scale than prospective studies.

Because researchers study groups of people before they develop an illness, they can discover potential cause-and-effect relationships between certain behaviors and the development of a disease.

Inexpensive and less time-consuming

Retrospective studies tend to be cheaper and quicker than prospective studies as the data already exists, and researchers do not need to recruit participants.

Beneficial for rare diseases

Researchers in retrospective studies can address rare diseases easier than in prospective studies because, in prospective studies, researchers would need to recruit extremely large cohorts.

Limitations

Bias and confounding variables.

Most sources of error in retrospective studies are due to confounding and bias. These errors are more common in retrospective studies than in prospective studies, so a retrospective study design should not be used when a prospective design is possible.

Recall bias

Participants might not be able to remember if they were exposed or when they were exposed, or they might omit other details that are important for the study.

Missing data

Because researchers are using already existing data, they rely on others for accurate recordkeeping, and important information may not have been collected in the first place.

• Investigation of risk factors for breast cancer (Press & Pharoah, 2010).
• Characteristics of trafficked adults and children with severe mental illness (Oram et al., 2015).
• Activated injectable vitamin D and hemodialysis survival (Teng et al., 2005).
• Reporting critical incidents in a tertiary hospital Munting et al., 2015).
• Reporting critical incidents in a tertiary hospital (Munting et al., 2015).
• Association between blood eosinophil count and risk of readmission for patients with asthma (Kerkhof et al., 2018).
• Risk factors for mental disorders in women survivors of human trafficking (Abas et al., 2013).

Frequently Asked Questions

1. what is the difference between case-control and retrospective cohort studies.

Case-control studies are usually, but not exclusively, retrospective. Case-control studies are performed on individuals who already have a disease, and researchers compare them with other individuals who share similar characteristics but do not have the disease.

In a retrospective cohort study, on the other hand, researchers examine a group before any of the subjects have developed the disease. Then they examine any factors that differed between the individuals who developed the condition and those who did not.

More simply, the outcome is measured before the exposure in case-control studies, whereas the outcome is measured after exposure in cohort studies.

2. Is a retrospective study experimental?

No, retrospective cohort studies are observational. Researchers analyze a group of subjects without manipulating any variables or interfering with their environment.

Researchers use existing data to investigate the target population, so no experimentation is necessary. Retrospective cohort studies examine cause-and-effect relationships between a disease and an outcome. However, they do not explain why the factors that affect these relationships exist.

Experimental studies are required to determine why a certain factor is associated with a particular outcome.

Abas, M., Ostrovschi, N.V., Prince, M, et al. (2013). Risk factors for mental disorders in women survivors of human trafficking: a historical cohort study. BMC Psychiatry 13, 204. https://doi.org/10.1186/1471-244X-13-204.

Hess, D.R. (2004) Retrospective studies and chart reviews. Respir Care. 49(10):1171-4. PMID: 15447798.

Kerkhof, M., Tran, T.N., Van den Berge, M., Brusselle, G.G., Gopalan, G., Jones, R.C.M., et al. (2018). Association between blood eosinophil count and risk of readmission for patients with asthma: Historical cohort study. 13(7): e0201143.

Munting, K.E, et al. (2015). Reporting critical incidents in a tertiary hospital: a historical cohort study of 110,310 procedures. Can J Anesth/J Can Anesth 62, 1248–1258. https://doi.org/10.1007/s12630-015-0492-y

Oram, S., Khondoker, M.R., Abas, M.A., Broadbent, M.T., & Howard, L.M. (2015). Characteristics of trafficked adults and children with severe mental illness: a historical cohort study. The Lancet. Psychiatry, 2 12, 1084-91.

Press, D. J., & Pharoah, P. (2010). Risk factors for breast cancer: a reanalysis of two case-control studies from 1926 and 1931. Epidemiology (Cambridge, Mass.), 21(4), 566–572. https://doi.org/10.1097/EDE.0b013e3181e08eb3

Ranganathan, P., & Aggarwal, R. (2018). Study designs: Part 1 – An overview and classification. Perspectives in clinical research, 9(4), 184–186.

Song, J. W., & Chung, K. C. (2010). Observational studies: cohort and case-control studies. Plastic and reconstructive surgery, 126(6), 2234–2242. https://doi.org/10.1097/PRS.0b013e3181f44abc.

Teng, M., Wolf, M., Ofsthun, M. N., Lazarus, J. M., Hernán, M. A., Camargo, C. A., Jr, & Thadhani, R. (2005). Activated injectable vitamin D and hemodialysis survival: a historical cohort study. Journal of the American Society of Nephrology: JASN, 16(4), 1115–1125.

Further Information

• Cohort Effect? Definition and Examples
• Barrett, D., & Noble, H. (2019). What are cohort studies?. Evidence-based nursing, 22(4), 95-96.
• Hess, D. R. (2004). Retrospective studies and chart reviews. Respiratory care, 49(10), 1171-1174.
• Euser, A. M., Zoccali, C., Jager, K. J., & Dekker, F. W. (2009). Cohort studies: prospective versus retrospective. Nephron Clinical Practice, 113(3), c214-c217.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Retrospective studies - utility and caveats

Affiliation.

• 1 Yashoda Hospitals, Hyderabad, India.
• PMID: 33469615
• DOI: 10.4997/JRCPE.2020.409

A thorough understanding of the pros and cons of the various study designs is critical to correct interpretation of their results. Retrospective studies are an important tool to study rare diseases, manifestations and outcomes. Findings of these studies can form the basis on which prospective studies are planned. Retrospective studies however have several limitations owing to their design. Since they depend on review of charts that were originally not designed to collect data for research, some information is bound to be missing. Selection and recall biases also affect the results and reasons for differences in treatment between patients and lost follow ups can often not be ascertained and may lead to bias. Readers need to critically evaluate the methods and carefully interpret the results of retrospective studies before they put them to practice. Researchers should avoid over generalisation of results and be cautious in claiming cause-effect relationship in retrospective studies.

Keywords: Retrospective studies; bias; cause-effect relationship; chart review; interpretation; limitations.

PubMed Disclaimer

Conflict of interest statement

No conflict of interests declared.

• Retrospective studies - utility and caveats. Zealley I. Zealley I. J R Coll Physicians Edinb. 2021 Mar;51(1):106-110. doi: 10.4997/JRCPE.2021.133. J R Coll Physicians Edinb. 2021. PMID: 33877156 No abstract available.

Similar articles

• Epidemiologic studies: pitfalls in interpretation. Westhoff CL. Westhoff CL. Dialogues Contracept. 1995 Winter;4(5):5-6, 8. Dialogues Contracept. 1995. PMID: 12288680
• Secondary use of randomized controlled trials to evaluate drug safety: a review of methodological considerations. Hammad TA, Pinheiro SP, Neyarapally GA. Hammad TA, et al. Clin Trials. 2011 Oct;8(5):559-70. doi: 10.1177/1740774511419165. Epub 2011 Aug 30. Clin Trials. 2011. PMID: 21878445 Review.
• A guide to missing data for the pediatric nephrologist. Larkins NG, Craig JC, Teixeira-Pinto A. Larkins NG, et al. Pediatr Nephrol. 2019 Feb;34(2):223-231. doi: 10.1007/s00467-018-3932-4. Epub 2018 Mar 13. Pediatr Nephrol. 2019. PMID: 29536258 Review.
• Bias: a review of current understanding. Adebiyi AO. Adebiyi AO. Afr J Med Med Sci. 2010 Sep;39(3):241-8. Afr J Med Med Sci. 2010. PMID: 21416795 Review.
• Measuring change in quality of life: bias in prospective and retrospective evaluation. Blome C, Augustin M. Blome C, et al. Value Health. 2015 Jan;18(1):110-5. doi: 10.1016/j.jval.2014.10.007. Value Health. 2015. PMID: 25595241 Review.
• An evidence review and gap analysis for obstructive hypertrophic cardiomyopathy. Butzner M, Aronitz E, Cameron H, Tantakoun K, Shreay S, Drudge C. Butzner M, et al. BMC Cardiovasc Disord. 2024 Aug 10;24(1):416. doi: 10.1186/s12872-024-04084-7. BMC Cardiovasc Disord. 2024. PMID: 39127628 Free PMC article. Review.
• Performance of [ 18 F]FDG PET/CT in Diagnosing Cyst Infections in Patients with Autosomal Dominant Polycystic Kidney Disease: A Systematic Review and a Bivariate Meta-Analysis. Treglia G, Albano D, Rizzo A, Bellasi A, Glaudemans AWJM, Gheysens O. Treglia G, et al. Diagnostics (Basel). 2024 Jul 25;14(15):1603. doi: 10.3390/diagnostics14151603. Diagnostics (Basel). 2024. PMID: 39125479 Free PMC article. Review.
• Unlocking the Long-Term Effectiveness of Benralizumab in Severe Eosinophilic Asthma: A Three-Year Real-Life Study. Pini L, Bagnasco D, Beghè B, Braido F, Cameli P, Caminati M, Caruso C, Crimi C, Guarnieri G, Latorre M, Menzella F, Micheletto C, Vianello A, Visca D, Bondi B, El Masri Y, Giordani J, Mastrototaro A, Maule M, Pini A, Piras S, Zappa M, Senna G, Spanevello A, Paggiaro P, Blasi F, Canonica GW, On Behalf Of The Sani Study Group. Pini L, et al. J Clin Med. 2024 May 20;13(10):3013. doi: 10.3390/jcm13103013. J Clin Med. 2024. PMID: 38792553 Free PMC article.
• Does Timing of Radiation Therapy Impact Wound Healing in Patients Undergoing Metastatic Spine Surgery? Ahluwalia R, Chanbour H, Zeoli T, Abtahi AM, Stephens BF, Zuckerman SL. Ahluwalia R, et al. Diagnostics (Basel). 2024 May 20;14(10):1059. doi: 10.3390/diagnostics14101059. Diagnostics (Basel). 2024. PMID: 38786357 Free PMC article.
• Parenteral Nutrition in Palliative Cancer Care: Detrimental, Futile, or Beneficial? Løhre ET, Solheim TS, Jakobsen G, Vagnildhaug OM, Schmidberger Karlsen TL, Habberstad RH, Balstad TR, Thronæs M. Løhre ET, et al. Curr Oncol. 2024 May 11;31(5):2748-2757. doi: 10.3390/curroncol31050208. Curr Oncol. 2024. PMID: 38785489 Free PMC article.

Publication types

• Search in MeSH

LinkOut - more resources

Full text sources, miscellaneous.

• NCI CPTAC Assay Portal
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Overview of Analytic Studies

•   1  
• |   2  
• |   3  
• |   4  
• |   5  
• |   6  

Epi_Tools.XLSX

All Modules

Prospective and Retrospective Cohort Studies

Cohort studies can be classified as prospective or retrospective based on when outcomes occurred in relation to the enrollment of the cohort.

Prospective Cohort Studies

In a prospective study like the Nurses Health Study baseline information is collected from all subjects in the same way using exactly the same questions and data collection methods for all subjects. The investigators design the questions and data collection procedures carefully in order to obtain accurate information about exposures before disease develops in any of the subjects. After baseline information is collected, subjects in a prospective cohort study are then followed "longitudinally," i.e. over a period of time, usually for years, to determine if and when they become diseased and whether their exposure status changes. In this way, investigators can eventually use the data to answer many questions about the associations between "risk factors" and disease outcomes. For example, one could identify smokers and non-smokers at baseline and compare their subsequent incidence of developing heart disease. Alternatively, one could group subjects based on their body mass index (BMI) and compare their risk of developing heart disease or cancer.

 Examples of Prospective Cohort Studies

• The Framingham Heart Study Home Page
• The Nurses Health Study Home Page

Pitfall: Note that in these prospective cohort studies a comparison of incidence between the groups can only take place after enough time has elapsed so that some subjects developed the outcomes of interest. Since the data analysis occurs after some outcomes have occurred, some students mistakenly would call this a retrospective study, but this is incorrect. The analysis always occurs after a certain number of events have taken place. The characteristic that distinguishes a study as prospective is that the subjects were enrolled, and baseline data was collected before any subjects developed an outcome of interest.

Retrospective Cohort Studies

In contrast, retrospective studies are conceived after some people have already developed the outcomes of interest. The investigators jump back in time to identify a cohort of individuals at a point in time before they have developed the outcomes of interest, and they try to establish their exposure status at that point in time. They then determine whether the subject subsequently developed the outcome of interest.

The video below provides a brief (7:31) explanation of the distinction between retrospective and prospective cohort studies.

Link to a transcript of the video

Intervention Studies (Clinical Trials)

Intervention studies (clinical trials) are experimental research studies that compare the effectiveness of medical treatments, management strategies, prevention strategies, and other medical or public health interventions. Their design is very similar to that of a prospective cohort study. However, in cohort studies exposure status is determined by genetics, self-selection, or life circumstances, and the investigators just observe differences in outcome between those who have a given exposure and those who do not. In clinical trials  exposure status  (the treatment type)  is assigned by the investigators . Ideally, assignment of subjects to one of the comparison groups should be done randomly in order to produce equal distributions of potentially confounding factors. Sometimes a group receiving a new treatment is compared to an untreated group, or a group receiving a placebo or a sham treatment. Sometimes, a new treatment is compared to an untreated group or to a group receiving an established treatment. For more on this topic see the module on Intervention Studies .

In summary, the characteristic that distinguishes a clinical trial from a cohort study is that the investigator assigns the exposure status in a clinical trial, while subjects' genetics, behaviors, and life circumstances determine their exposures in a cohort study.

return to top | previous page | next page

Content ©2023. All Rights Reserved. Date last modified: August 15, 2023. Wayne W. LaMorte, MD, PhD, MPH

13. Study design and choosing a statistical test

Sample size.

• Mohamed Hany 1 &
• Bart Torensma 2  

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data Availability

No datasets were generated or analysed during the current study.

Arshad A, Mustafa AA, Imran L. Correspondence: Revisional one-step bariatric surgical techniques after unsuccessful laparoscopic gastric band: a retrospective cohort study with 2-year follow-up. Obes Surg. 2024;34:3133–4.

Article   PubMed   Google Scholar  

Hany M, Zidan A, Ibrahim M, Sabry A, Agayby ASS, Mourad M, et al. Revisional one-step bariatric surgical techniques after unsuccessful laparoscopic gastric band: a retrospective cohort study with 2-year follow-up. Obes Surg. 2024;34:814–29.

Article   PubMed   PubMed Central   Google Scholar  

Vandenbroucke JP. Prospective or retrospective: what’s in a name? BMJ. 1991;302:249.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Le Coq B, Frering V, Ghunaim M, Campan P, Dabrowski A, Mulliez E, et al. Impact of surgical technique on long-term complication rate after laparoscopic adjustable gastric banding (LAGB): results of a single-blinded randomized controlled trial (ANOSEAN study). Ann Surg. 2016;264:738–44.

Dang JT, Switzer NJ, Wu J, Gill RS, Shi X, Thereaux J, et al. Gastric band removal in revisional bariatric surgery, one-step versus two-step: a systematic review and meta-analysis. Obes Surg. 2016;26:866–73.

Hany M, Ibrahim M. Comparison between stable line reinforcement by barbed suture and non-reinforcement in sleeve gastrectomy: a randomized prospective controlled study. Obes Surg. 2018;28:2157–64.

Diab A-RF, Sher T, Awshah S, Noom M, Docimo S, Sujka JA, et al. Oversewing/suturing of the staple line during sleeve gastrectomy is an effective and affordable staple line reinforcement method: a meta-analysis of randomized controlled trials. Obes Surg. 2023 [cited 2023 Jul 4]; Available from: https://link.springer.com/10.1007/s11695-023-06672-6

Aiolfi A, Gagner M, Zappa MA, Lastraioli C, Lombardo F, Panizzo V, et al. Staple line reinforcement during laparoscopic sleeve gastrectomy: systematic review and network meta-analysis of randomized controlled trials. Obes Surg. 2022;32:1466–78.

Download references

Author information

Authors and affiliations.

Department of Surgery, Medical Research Institute, Alexandria University, Alexandria, Egypt

Mohamed Hany

Clinical Epidemiologist, Leiden University Medical Center, Leiden, Netherlands

Bart Torensma

You can also search for this author in PubMed   Google Scholar

Contributions

MH and BT wrote the main letter to the editor. All authors reviewed the letter to the editor.

Corresponding author

Correspondence to Mohamed Hany .

Ethics declarations

Ethical approval.

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed Consent

Informed consent does not apply.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Hany, M., Torensma, B. Reply: Correspondence: Revisional One-Step Bariatric Surgical Techniques After Unsuccessful Laparoscopic Gastric Band: A Retrospective Cohort Study with 2-Year Follow-up. OBES SURG (2024). https://doi.org/10.1007/s11695-024-07464-2

Download citation

Received : 06 August 2024

Revised : 08 August 2024

Accepted : 12 August 2024

Published : 15 August 2024

DOI : https://doi.org/10.1007/s11695-024-07464-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Pharmacol

Ten Rules for Conducting Retrospective Pharmacoepidemiological Analyses: Example COVID-19 Study

Michael powell.

1 Department of Biomedical Engineering, Institute for Computational Medicine, The Johns Hopkins University, Baltimore, MD, United States

Allison Koenecke

2 Institute for Computational & Mathematical Engineering, Stanford University, Stanford, CA, United States

James Brian Byrd

3 Department of Internal Medicine, Division of Cardiovascular Medicine, University of Michigan Medical School, Ann Arbor, MI, United States

Akihiko Nishimura

4 Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health at Johns Hopkins University, Baltimore, MD, United States

Maximilian F. Konig

5 Ludwig Center, Lustgarten Laboratory, Howard Hughes Medical Institute, The Johns Hopkins University School of Medicine, Baltimore, MD, United States

6 Division of Rheumatology, Department of Medicine, The Johns Hopkins University School of Medicine, Baltimore, MD, United States

Ruoxuan Xiong

7 Graduate School of Business, Stanford University, Stanford, CA, United States

Sadiqa Mahmood

8 Health Catalyst Inc., Salt Lake City, UT, United States

9 Datavant Inc., San Francisco, CA, United States

Chetan Bettegowda

10 Department of Neurosurgery, The Johns Hopkins University School of Medicine, Baltimore, MD, United States

11 VA Health Economics Resource Center, Palo Alto VA, Menlo Park, CA, United States

Suzanne Tamang

12 Department of Biomedical Data Science, Stanford University, Stanford, CA, United States

Adam Sacarny

13 Department of Health Policy and Management, Columbia University Mailman School of Public Health, New York, NY, United States

Brian Caffo

Susan athey, elizabeth a. stuart.

14 Department of Mental Health, Johns Hopkins Bloomberg School of Public Health at Johns Hopkins University, Baltimore, MD, United States

Joshua T. Vogelstein

Luis Laranjeira , Eli Lilly, Portugal

Since the beginning of the COVID-19 pandemic, pharmaceutical treatment hypotheses have abounded, each requiring careful evaluation. A randomized controlled trial generally provides the most credible evaluation of a treatment, but the efficiency and effectiveness of the trial depend on the existing evidence supporting the treatment. The researcher must therefore compile a body of evidence justifying the use of time and resources to further investigate a treatment hypothesis in a trial. An observational study can provide this evidence, but the lack of randomized exposure and the researcher’s inability to control treatment administration and data collection introduce significant challenges. A proper analysis of observational health care data thus requires contributions from experts in a diverse set of topics ranging from epidemiology and causal analysis to relevant medical specialties and data sources. Here we summarize these contributions as 10 rules that serve as an end-to-end introduction to retrospective pharmacoepidemiological analyses of observational health care data using a running example of a hypothetical COVID-19 study. A detailed supplement presents a practical how-to guide for following each rule. When carefully designed and properly executed, a retrospective pharmacoepidemiological analysis framed around these rules will inform the decisions of whether and how to investigate a treatment hypothesis in a randomized controlled trial. This work has important implications for any future pandemic by prescribing what we can and should do while the world waits for global vaccine distribution.

Introduction

Imagine we are only halfway through 2020; the COVID-19 pandemic is raging, and widespread vaccination is thought to be at least a year away. Treatment ideas abound for COVID-19, and around the world more than 2,000 clinical treatment trials have been initiated to begin testing a wide variety of drugs hypothesized to help infected patients. Unfortunately, constrained resources can only fund some subset of the investigator-initiated trials; hence, trials resourced to begin patient enrollment must be chosen judiciously based on the soundness of the medical hypothesis, the availability of preclinical evidence, and the trial’s feasibility, cost, and potential impact. It is in this environment that you have arrived with a novel idea for an effective pharmaceutical intervention for COVID-19 (or the next pandemic).

The gold-standard way to evaluate your hypothesis is a randomized controlled trial (RCT), but that takes time and resources you (and the world) may not have at the moment. In fact, the window to pursue your trial is limited as interest (and resources) will increasingly focus on progress in vaccine development. Assuming your trial would be ethically permissible and otherwise feasible (e.g., reasonable follow-up periods and realistic recruiting goals), is there anything you can do right now to investigate your hypothesis and determine the priority of testing it in an RCT? There are three common types of retrospective studies to consider, each of which uses observational data: cross-sectional studies, case-control studies, and cohort studies. This paper provides a framework for investigating your pharmaceutical hypothesis carefully and responsibly using a retrospective cohort study. Beyond just advocating for a clinical trial, your investigation can inform many of the decisions regarding the details of a clinical trial (e.g., which drugs and dosage levels to test), as well as who is most likely to benefit from your treatment; all of this may influence how stakeholders choose to prioritize your trial. A retrospective analysis focused on today’s disease (even after widespread vaccination) can also improve our understanding and preparedness for a novel disease we encounter in the future; completed studies targeting readily available treatment options in a related disease could help save countless lives when the next pandemic strikes and the world is again waiting for a vaccine.

Countries around the world have defended themselves against SARS-CoV-2 using travel restrictions, national lockdowns, facemask policies, and other non-pharmaceutical interventions to stop the spread of SARS-CoV-2, and evaluating these population-level actions requires different tools than what we present in this paper (i.e., there is no path to an RCT for some public health measures). Here, we use the tools of pharmacoepidemiology, a field spanning clinical pharmacology and epidemiology, to study the effects of drugs in large numbers of people in order to estimate probabilities of beneficial and/or adverse effects. We introduce this body of knowledge as 10 rules for retrospective pharmacoepidemiological analyses designed to evaluate a treatment hypothesis (see Figure 1 for the 10 rules and Table 1 for common vocabulary). These rules are the result of a community effort, including academic, health care, nonprofit, and industry contributors, to establish a set of best practices for retrospective analyses. A retrospective analysis aims to estimate the comparative effectiveness of one treatment vs. another (e.g., a new treatment vs. the standard care) using real-world evidence ( Office of the Commissioner, 2020 ) obtained from preexisting data such as electronic health records (EHR), insurance claims databases, or health care registries. We embark on a retrospective analysis knowing that it should not stand alone as the sole evidence supporting adoption of a new treatment; observational study evidence should be considered suggestive rather than conclusive . A retrospective analysis can contribute a body of real-world evidence as a supplement to the medical theory supporting the treatment and any preclinical studies conducted in vitro and/or in vivo , all of which combine to inform decisions about whether and how to pursue a randomized trial.

The first phase of the 10 rules involves building the right team to envision the perfect trial and then consider the limitations of an observational study. The study then enters a preparation phase in which the details of the study are specified: hypotheses, which population to target, essential confounders to observe, and which data sets might support the study criteria. In the analysis planning phase, the objective is to refine and validate the study definitions and selected methods without being influenced by real results. Finally, the study concludes when the study is run, carefully summarized, and reported accurately.

This table of common terms provides working definitions for vocabulary appearing in the following 10 rules.

COVID-19 Study

Here we introduce a potential COVID-19 pharmaceutical treatment to discuss the 10 rules more concretely. Prior work indicates that certain alpha-1 adrenergic receptor antagonists (alpha blockers) disrupt cytokine storm syndromes, a pathological hyperinflammatory response associated with respiratory infection and other diseases ( Staedtke et al., 2018 ; Koenecke et al., 2021 ; Thomsen et al., 2021 ). Subsequently, others determined that hyperinflammation is implicated in morbidity and mortality in COVID-19 patients ( Mehta et al., 2020 ; Li et al., 2021 ). Many COVID-19 patients were already taking alpha blockers prior to infection for unrelated, chronic medical conditions. Consistent use of doxazosin (a particular alpha blocker) prior to COVID-19 diagnosis is the exposure of interest, and the goal is to estimate its effectiveness for preventing in-hospital death.

We are now ready to dig into the 10 rules. Rules 1–3 describe three guiding principles for a retrospective pharmacoepidemiological analysis. Rules 4–7 discuss key preparations for the analysis. Rules 8–9 address how to develop and refine the analysis plan. Rule 10 concludes with executing, summarizing, and reporting the results to facilitate replicating and extending them. Each rule could have its own paper or book chapter (and in many cases they do), and we expand the discussion of each rule considerably in the supplementary material to explain the concrete, actionable steps the rules require.

Guiding Principles: Build and Focus the Team

Rule 1: form a multidisciplinary team.

Get the right people involved at the start, in the middle, and at the end. Every step of the way you are going to need to make decisions about the medical rationale for the proposed exposure, treatment practices in clinics and hospitals, the nuances of relevant data stores and common coding practices, the study design, and the statistical analyses and interpretation of results. Specifically, high-quality retrospective analyses depend on input from committed individuals with different domain expertise: medical, data sources, epidemiology, and causal analysis.

Clinicians provide insights into the differences between exposed (those prescribed doxazosin) and unexposed groups; understanding the conditions that lead to treatment is critical in designing the study. Clinical experience working with patients diagnosed with COVID-19 is also helpful for gaining insight into the dynamics of COVID-19 testing and patient care. For example, the protocols for testing and admitting patients have varied over place and time, especially early in the crisis. In an evolving pandemic, these factors motivate accounting for changing patient populations; failing to do so could result in biased estimates of treatment effects.

A COVID-19 study presents unique challenges. First, there is an urgency to rapidly (and comprehensively) assess a proposed exposure. Second, the landscape changes while the study is underway: new datasets emerge and published results change attitudes for different treatments. Third, near-constant sharing of ideas and work products is crucial, but the study team members are likely isolated. Getting feedback early and often from all parties is crucial for reducing time-to-iterate without sacrificing research quality ( London and Kimmelman, 2020 ). While still ensuring HIPAA protections are appropriately observed, tools like Slack, GitHub, and Google Docs for conversing, collaborating on code, and writing, respectively, facilitate the kind of rapid progress that is otherwise hard to achieve.

Rule 2: Emulate a Randomized Controlled Trial

Design your observational study to mimic — as closely as possible — a randomized controlled trial with similar goals, an approach known as trial emulation ( Rubin, 2004 ; Rosenbaum, 2010 ; Hernán and Robins, 2016 ; Dickerman et al., 2019 ). Carefully consider what you measure, when you measure it, and in whom you measure it. Draw a CONSORT diagram of the ideal RCT you wish you could run ( Begg et al., 1996 ). Emulating an RCT should ideally include preregistration of the study and analysis plans (described in Rule 9).

Our retrospective analysis should emulate the desired RCT investigating doxazosin as a prophylactic treatment for severe symptoms among patients with COVID-19 ( Konig et al., 2020 ). The trial would target older adults, a group who appears to have the greatest risk of adverse outcomes from COVID-19 ( D-19 Provisional Coun, 2020 ). Emulating this trial requires focusing on the same patient group in our retrospective analysis. Without random exposure assignment, the retrospective study must identify people taking doxazosin prior to a COVID-19 diagnosis. In the United States, many older adults take doxazosin for conditions including hypertension and benign prostatic hyperplasia (BPH). Thus, emulating a trial in older adults would be both meaningful (by studying the impact on a group at risk for adverse outcomes from COVID-19) and feasible (since observing doxazosin use in this group is likely). There is a cost, however, to targeting a subset of the population; the study can lose external validity for other patient groups ( Holdcroft, 2007 ).

Rule 3: Realize That Rule 2 Is Impossible and Proceed Carefully

In an observational study, our choices of what to measure and in whom to measure it are limited by what data already exists. Even more concerning, our inability to randomize exposure assignment introduces categories of variables that we worry less about in randomized controlled trials, most notably confounders. Confounders satisfy three properties: they are associated with the outcome (i.e., risk factors), they are associated with the exposure (i.e., they are unequally distributed among the exposure groups), and they are not effects of the exposure ( Jager et al., 2008 ). If not observed and sufficiently addressed, confounders lead to confounding, which is a bias in the measure of a treatment effect resulting from treatments and outcomes sharing a common cause ( Hernán and Robins, 2020 ). Review the different kinds of covariates that can exist in a causal analysis of observational data and how each can impact causal estimates (see Rule 5). Confounding by indication is likely to occur in observational data, and the primary concern in your observational study is the identification and mitigation of potential confounders. Your analysis will therefore need to address confoundedness as evidenced by observed differences in the covariate distributions of the various exposure groups, and you can conduct descriptive analysis characterizing observed differences between treatment and control groups to complement qualitative information gathering about the treatment assignment process in order to guide your thinking about what variables will be necessary to include in the data to mitigate confounding.

Expanding on our previous observation that older people are more likely to be taking doxazosin, we now consider how confounding can emerge in an observational study and the importance of addressing it. Without the deliberate recruitment and randomization of an RCT, doxazosin use will be concentrated among the older individuals eligible for our study because both hypertension and BPH prevalence increase with age ( Partin et al., 1991 ; AlGhatrif et al., 2013 ). COVID-19 outcomes appear to be worse with increased age, suggesting that age is a confounder we must address. Even if doxazosin is effective at reducing all-cause mortality, doxazosin is disproportionately prescribed to older people who disproportionately have worse outcomes. Unless we account for age, a truly beneficial treatment effect could be estimated with negative bias (possibly making the treatment appear harmful). This example from our COVID-19 observational study highlights the reasoning required to identify important covariates to consider in our analyses.

Preparation Phase: Establish the Hypotheses and Acquire Resources to Evaluate Them

Rule 4: formalize the research goal.

Specify the exposure in terms of quantity, duration, frequency, and recency. Define the comparison groups of interest (e.g., define unexposed ). Bias (e.g., selection bias ) can arise from many sources in an observational study, but it fundamentally stems from the lack of randomized exposure assignment, resulting in the construction of a control group having different concerns than the treated group with regard to censoring, missing data, self-selection, or even eligibility for treatment ( Hernán et al., 2004 ). While confounding by indication is almost guaranteed to be present in non-experimental pharmacoepidemiology research and will be addressed in other rules, we highlight the importance now of identifying comparison groups in which every individual theoretically has some probability of receiving the proposed treatment. An example of questionable comparison group construction could be comparing two groups with the same disease but where the two groups take different drugs based on significant differences in disease severity (e.g., metformin for less advanced type 2 diabetes mellitus vs. insulin for more advanced type 2 diabetes mellitus). Next, define an outcome that is specific, measurable, and sufficient to answer the research question. Finally, formalize your hypotheses (i.e., specify the null and alternative, sidedness, primary vs. secondary exposures and outcomes).

A pharmaceutical study considers a particular drug, dosage, recency, and duration by using prescription records to qualify a patient as either exposed or unexposed to the medication under investigation (e.g., doxazosin, ≥4 mg daily, prescription valid through COVID-19 diagnosis date, continuous use reflected by total days’ supply covering 80% of the previous 3 months — a quantity known as the medication possession ratio or MPR ( Andrade et al., 2006 )). When quantifying duration and recency, multiple filled prescriptions for a drug better indicate continued use than a single fill that may have gone unused. Prescriptions lasting until some key date (possibly allowing for skipped doses) provide better evidence that the drug was in use on the date of interest. Unfortunately, researchers are usually unable to confirm the medication was consumed as intended. Some patients deviate from the prescribed drug regimen, and this is often unobservable; we therefore conduct intent-to-treat analysis by grouping patients according to inferred exposures revealed in prescription records ( Gupta, 2011 ). The comparison group might include anyone who does not meet the exposure definition, only people who have not taken the proposed drug for a specified length of time, or perhaps only people who have never taken any alpha blocker. Importantly, the comparison group should not be made up of people who cannot take alpha blockers for reasons that could relate to their health outcomes.

As COVID-19 was entering its first peak, many countries’ chief concerns were ventilator resources and anticipated deaths. Outcomes related to ventilator dependence or mortality may be of particular interest. We found that using ventilator dependence as an outcome is often problematic for two reasons. First, ventilator usage depends on the standard of care with respect to administering ventilator resources at a particular time and place, and the severity of patients in the data as well as treatment protocols differed substantially by time and place during the pandemic. Second, insufficient ventilator availability and inconsistent ventilator coding practices makes ventilator dependence a complicated outcome in some places. All-cause mortality is not completely unaffected by the changing practices related to ventilators, but mortality proves to be the more clearly defined outcome of ultimate importance. Since we cannot quantify the exact role of COVID-19 in hospital deaths, the best practice is to use all-cause mortality as the primary outcome of interest.

Rule 5: Identify and Reason About Potential Confounders

Confounders will be present; make every effort to observe these confounders and adjust for them appropriately. Include standard demographic variables, relevant comorbidities, and a comorbidity index and/or other indicators of overall health. Note that identifying confounders before you have data will help you better assess the utility of candidate datasets. Organize your understanding of the key variables with a causal diagram (see Figure 2 ). A directed acyclic graph (DAG) is a powerful way to depict the causal relationships in your analysis ( Greenland et al., 1999 ; Pearl, 2009 ) and examine potential biases your analysis might permit ( VanderWeele et al., 2008 ). Bias might result from an unobserved confounder that is not measured in the data and therefore cannot be adjusted for in the analysis; a significant unobserved confounder can invalidate all results obtained from the study. Thinking through each variable and the corresponding existence and direction of arrows (representing both observed and unobserved cause-effect relationships) helps prevent unknowingly inviting bias into your analysis and mitigate potential sources of bias that you do include. Following procedures for identifying a minimally sufficient adjustment set (MSAS) of confounders in a DAG ( VanderWeele et al., 2008 ) can eliminate adjustment-induced bias. Ultimately, a DAG provides an excellent visual representation of the known or assumed relationships between variables and helps identify the necessary variables to adjust for to minimize confounding in a multivariable analysis. Know that no matter what you do, you will likely still have unobserved confounding (we describe sensitivity analyses to quantify the magnitude of this issue in the Rule 9 supplement).

This directed acyclic graph (DAG) shows the types of variable relationships described in Rule 3 using the example COVID-19 study. A DAG has no cycles, which means no variable can cause itself, either directly or through one or more other variables. In our effort to estimate the causal effect of doxazosin on mortality, this DAG helps us identify which variables will be important to adjust for in our analyses (in reality, this diagram would include many more variables of these same types). It is the set of confounders that has the ability to distort the association between exposure and outcome as revealed by the arrows leading from each confounder to both the exposure and the outcome. We highlight two observed confounders: the demographic confounder age and the comorbidity confounder hypertension (HTN). We also depict the unobserved confounder overall health, which we might attempt to measure using indicators of overall health like frequency and duration of recent inpatient stays.

Several alpha blockers (doxazosin included) have an FDA indication for hypertension, so we expect the exposed population will have higher rates of hypertension, a condition that might lead to worse outcomes. Relevant comorbidities that serve as confounders per clinicians’ expertise include sex, age, diabetes mellitus, hypertension, cardiovascular disease, and chronic obstructive pulmonary disease. For the doxazosin hypothesis, patient location has significance as prescription practices and the standard of care for relevant conditions vary around the world. Even with these considerations, unobserved confounding can still affect a study’s results. Unobserved confounding is one reason why the results of observational studies of hydroxychloroquine have differed from those of RCTs ( Hernandez et al., 2020 ).

Rule 6: Operationalize the Target Population

Select the target population for your observational study to reflect the intended RCT population. Refine the potential study population by setting the inclusion and exclusion criteria to minimize confounding. Consider the impact of refining the target population on both internal validity (focused on groups the study includes) and external validity (focused on groups to which the findings might extend).

In a COVID-19 retrospective cohort study, the defining characteristic of patients in the cohort is a COVID-19 diagnosis. In our observational study, the exposure was administered prior to the COVID-19 diagnosis. Using a post-treatment variable to define the cohort can introduce post-treatment bias, so choosing to select the sample on the basis of a post-treatment variable (COVID-19 diagnosis) implies we believe the exposure has no impact on one’s susceptibility to infection and likelihood of diagnosis. We are aware of no evidence that taking doxazosin changes one’s susceptibility to SARS-CoV-2 infection; doxazosin could, however, affect whether a person is diagnosed by mitigating symptoms to a degree that a patient self-treats rather than seeing a doctor to receive a formal diagnosis. Early in the pandemic, COVID-19 tests were only available in inpatient environments and were reserved for the sickest patients. Individuals were urged to stay home until they truly needed hospital resources. This led to many unobserved, undiagnosed patients. We cannot estimate the treatment effect in this population as we do not observe the qualifying condition: a COVID-19 diagnosis. Later in the pandemic, we face the same problem, but for a different reason; widespread community testing facilitates diagnoses, but these test results and diagnoses may not enter a patient’s health records or claims history (both common data sources for retrospective studies). We could again lose visibility of milder cases where a patient recovers at home, limiting our assessment to the severe cases warranting hospitalization. This is a notable limitation of defining the cohort by a COVID-19 diagnosis.

We focus the doxazosin study on older patients because this group is at high risk of adverse outcomes from COVID-19. Older men in the United States take doxazosin at a far higher rate than women, primarily because doxazosin is a treatment for BPH. Compared to other men of the same age, a prior BPH diagnosis is not expected to have any impact on COVID-19 outcomes. We now make the consequential restriction to focus the study on older men, allowing us to capture many exposed individuals with no above-average risk for negative outcomes. This target patient population attempts to minimize the impact of unobserved confounding. While this may be appealing, the exclusions have important implications. Pragmatically, reducing the population under consideration may reduce statistical power by limiting the sample size. Societally, focusing the study exclusively on older men limits the study’s internal validity to older men. It will take additional assumptions and/or further analyses to extend the study’s findings to women and young people.

Rule 7: Get the Best Data for the Study

Invest time in getting access to the best possible data for your study such that your desired study definitions can be realized. Know what your data source contains, where it originated, and how it was assembled. Know the biases and limitations of candidate datasets. Identify the target population using carefully selected, standardized diagnosis and/or procedure codes. Identify chronic comorbidities using standard condition code sets ( Chronic Conditions Data Warehouse, 2020 ) and sufficient patient histories.

Identifying COVID-19 patients can be difficult because of the nonexistence of COVID-19-specific International Classification of Diseases (ICD) codes early on in the pandemic. It was only on April 1, 2020 that ICD-10 U07.1 was introduced for a confirmed diagnosis of COVID-19, and adoption of this code for billing purposes remained variable and inconsistent for some time. Using an established, community-derived definition for the COVID-19 population is recommended (e.g., as provided by the National COVID Cohort Collaborative - N3C ( National COVID Cohort Collaborative, 2020 )). COVID-19 population definitions often divide into two groups: COVID-narrow includes confirmed COVID-19 diagnoses while COVID-broad adds suspected COVID-19 patients who have not been tested but exhibit multiple COVID-19 symptoms. Large hospitals that treated thousands of COVID-19 patients and performed in-house testing (e.g., Mount Sinai Hospital in New York City) are best situated to precisely construct a COVID-19 cohort ( Wang et al., 2020 ).

In the early stages of a pandemic, finding a well-curated, sufficiently sized data set to test your hypothesis on the novel disease may be impossible. Expert clinical input may identify a suitable substitute for COVID-19 that reflects the same symptoms and disease progression your treatment is theorized to target (e.g., cytokine storm syndrome resulting from acute respiratory distress or pneumonia). Identifying such a disease with established coding and extensive patient records can jumpstart your research while the data practices surrounding an emerging pandemic stabilize.

The hypothetical doxazosin study requires access to each individual’s inpatient, outpatient, and prescription drug history for at least the year leading up to COVID-19 diagnosis. Clinical data from the U.S. Veterans Health Administration (VHA) is an ideal candidate data set for this type of study for several reasons. Older adults are well represented in the VA health care system, typically with extensive patient histories. This reduces the likelihood of having the insufficient patient histories that sometimes accompany individuals in a claims database who have recently changed employers. In addition, the VA health system would have comprehensive records: diagnoses, procedures, prescription drug use, doctors’ notes, in-hospital medications received, and lab results.

Analysis Planning Phase: Develop and Refine the Analysis Plan

Rule 8: explore and model your data with surrogate outcomes.

Use permuted outcomes or synthetic data ( Koenecke and Varian, 2020 ) as you build and test your analysis code to prevent being influenced by any premature results. First, examine the univariate and pairwise distributions of the covariates that will be used in the analysis. Second, examine all covariate distributions after stratification by exposure group and/or time period, compute each individual’s propensity for treatment (i.e., estimate a propensity score), and obtain better empirical overlap using propensity trimming ( Lee et al., 2011 ). A propensity score reflects the probability that an individual would receive treatment (i.e., belong to the exposed group) on the basis of observed covariates. To counter confounding by indication, a variety of analytical techniques employ propensity scores to balance the exposed and unexposed groups by matching or weighting using propensity scores, which assign greater weight to the unexposed individuals who appear more similar to the exposed individuals in terms of the observed covariates. Third, begin modeling with an unadjusted modeling approach (e.g., simple logistic regression) to establish a baseline treatment effect estimate. Finally, use additional modeling approaches that adjust for confounders (e.g., doubly robust methods ( Bang and Robins, 2005 ) employing propensity scores and covariate adjustment in the outcome models), favoring methods that seek covariate balance.

Examining the covariate distributions of the exposed and unexposed groups will likely reveal that doxazosin users are generally older and have more comorbidities than non-users. Unadjusted models with no consideration of age would likely compare a younger, healthier unexposed group to an older, less healthy exposed group. We addressed this problem by including age as an observed confounder and by establishing inclusion/exclusion criteria that ensured anyone in the study could reasonably have been exposed to doxazosin. Now, we further exclude observations exhibiting extremely high or low propensity for treatment (on the basis of all covariates, not just age); this could include the extremely young, old, healthy, sick, etc. Extreme propensities indicate that almost all similar units share the same treatment assignment, such that there is limited information in the data about how similar individuals would have fared if their treatment assignment had been different.

Rule 9: Augment the Main Analysis With Extensive Sensitivity Analyses

Plan a thorough assessment of the robustness of your results to the many choices made along the way to estimating a treatment effect. Start by conducting supplementary analysis designed to illustrate clearly the role of observed confounders for both treatment assignment and outcome modeling, as this can build intuition about what factors are likely important in these processes ( Athey et al., 2017 ). Quantify the extent of unobserved confounding required to change your conclusions ( Rosenbaum and Rubin, 1983 ; Rosenbaum, 2010 ; VanderWeele and Ding, 2017 ) (i.e., determine how correlated an unobserved variable must be with the exposure and outcome to nullify any perceived treatment effect). Assess the robustness of your results to different modeling techniques, hyperparameters, outcome definitions, exposure definitions, inclusion/exclusion criteria, and other aspects of the study design. Explore additional sets of covariates, including different comorbidities and indicators of temporal health trends. Conduct negative outcome experiments and treatment control experiments ( Lipsitch et al., 2010 ). Refine, lock in, and preregister your formal analysis plan before examining any real model outputs using the true outcome data.

Robustness checks for a doxazosin study assess the impact of making adjustments to the treatment, outcome, and population definitions. We can test our hypothesis on both a COVID-narrow cohort and a COVID-broad cohort. Our confidence in the treatment will also be tied to how well our results hold up to changing the medication possession ratio and changing the post-diagnosis window we are monitoring for all-cause mortality. We can explore additional covariates beyond chronic comorbidities that may indicate increased health concerns closer to the COVID-19 diagnosis (e.g., other inpatient stays within 2 months of diagnosis).

Execution Phase: Execute the Analysis Plan and Report The Results

Rule 10: execute, summarize, and share (with caveats).

Execute your analysis plan with the true outcome data once you are satisfied with the quality of your data set and have sufficiently tested your code. If necessary, make the smallest possible refinements to your analysis plan and execute again, always ensuring you report deviations from your preregistered plan. Give your reader something that looks like what they are used to seeing (i.e., conventional measures of treatment effect, standard tables and figures). Explicitly describe the limitations of your study. Provide all the necessary method descriptions and code to facilitate replication.

We include a CONSORT diagram to show the split of doxazosin users and nonusers in the dataset, followed by their respective outcome counts, to help visualize the study like an RCT. We are targeting a clinical research-savvy audience including clinical trialists, so we present the treatment effect as an odds ratio (OR), which is a familiar metric for the likely readers. We define our null hypothesis as OR = 1 (i.e., the exposure does not change the odds of the outcome occurring). We then assess doxazosin to be beneficial if we find OR < 1. We present the associated confidence interval (CI) to convey the precision of our treatment effect estimate. Together, the OR and CI indicate the strength of evidence supporting further investigation of the doxazosin hypothesis.

As the pandemic is far from over, especially in lower resource countries and communities, we see the value both now and in future pandemics of responsibly investigating the efficacy of inexpensive, repurposed drugs as early treatment options while we wait for vaccine development, mass production, and global distribution. The primary benefits associated with conducting these investigations with retrospective analyses lie in reducing costs and increasing speed relative to running an RCT (assuming the RCT would be feasible and ethical). Moreover, retrospective pharmacoepidemiological analyses can be run even when no patients are available (e.g., after everyone is vaccinated) to learn more about potential treatments for future pandemics. Retrospective analyses make it easier to explore a variety of treatments with limited time and other resources, setting the stage for an RCT to test the most promising interventions. In the COVID-19 era, these are valuable benefits, but they come with a cost. The challenges facing retrospective analyses arise from the requirement to use data generated without a particular study in mind. Unlike an RCT, where researchers are able to decide exactly who will be recruited to participate, which exposure(s) will be assessed (e.g., drug, dosage, frequency, duration, etc.), and which outcome(s) will be measured, the observational study approach described here limits the researcher to only those definitions of exposure, outcome, confounders, and sample population that can be realized with available data. This places a significant burden on the researcher to determine whether the desired retrospective analysis is possible to conduct with available data. When the time and cost savings of performing a study with observational data outweigh the costs of constrained data collection and study design, using these 10 rules as a guide will support the execution of a rigorous retrospective pharmacoepidemiological analysis that speeds the time to clinical trials and, hopefully, proven effective treatments for patients.

Supplement: How to Follow These 10 Rules

This supplement serves to explain in detail the many recommendations made in the 10 rule paragraphs in the main text. Individual sentences in the rule paragraphs generally correspond to one or more paragraphs in this supplement explaining why the recommendation was made and how to satisfy its requirements.

Rule 1 Supplement: Form a Multidisciplinary Team

The main text states we require continuous input reflecting different kinds of domain expertise: medical, data sources, epidemiology, and causal analysis. Medical expertise ensures the study remains medically coherent while decisions are made throughout the design of the study. Data source expertise (including medical terminologists) can expedite the process of finding, accessing, and understanding relevant data sources and corresponding coding conventions, while also making known their potential limitations. The expertise in epidemiology that comes from working with observational health data ensures the study design and study definitions meet accepted standards in the literature (e.g., defining treatments, conditions, and other health indicators with observational data). Causal inference expertise ensures the use of appropriate analysis methods to support making a causal claim. The degree to which each expert contributes in each successive rule varies, but it is difficult to underestimate the value of assembling this group at the start.

Rule 2 Supplement: Emulate a Randomized Controlled Trial

Design your observational study to mimic — as closely as possible — a randomized controlled trial with similar goals, an approach known as trial emulation ( Rubin, 2004 ; Rosenbaum, 2010 ; Hernán and Robins, 2016 ; Dickerman et al., 2019 ). To start down this path, we must first clearly state the research objective. Most likely the clinician(s) on the team will be the source of the medical hypothesis. What is the pathophysiological mechanism this study seeks to understand? Which exposure(s) might reasonably affect this mechanism? Which subset of the population do we think the exposure(s) will benefit? Who could reasonably be eligible to receive the proposed exposure? Which measurable outcome(s) will reveal the efficacy of the proposed exposure(s)? Which analyses will be needed to do the appropriate comparisons? These details will continue to be refined as we think through the remaining rules, and we will rely on the team’s clinical expertise to ensure any refinements continue to support the primary research objective.

Carefully consider what you measure, when you measure it, and in whom you measure it. It can be helpful to lay out key aspects of the study design just as would be done in an RCT using a CONSORT flow diagram ( Begg et al., 1996 ) and other observational study reporting standards ( Benchimol et al., 2015 ; Langan et al., 2018 ). For example, a person considered for trial participation must be deemed eligible for the trial at the time of exposure group assignment, which must then occur before any follow-up periods begin or outcomes are observed. Suppose your ideal trial has an exclusion criterion barring participation of anyone with a history of heart problems. Heart problems that surface at some point after a person receives the exposure might be visible in observational data; since post-exposure health problems could not have been observed for the purposes of RCT enrollment, we ignore them when deciding the eligibility of patients for observational studies ( Dickerman et al., 2019 ).

Preregister your study and analysis plan just like an RCT. Before an RCT begins, the individuals running the trial will have already amassed a corpus of information about the relationship between the exposure and outcome (e.g., in preclinical data). They have used this information to design the trial and get approval from an institutional review board (IRB). Given this information, the study plan is fixed prior to collecting any patient information in the actual trial phase. The trial emulation proposed in this paper similarly promotes an exploratory data analysis and modeling phase that uses surrogate outcome data to refine the analysis plan before committing to a final outcome analysis to be run on actual outcome data (discussed further in Rules 8–10). Preregistering the study and documenting a final analysis plan avoids several pitfalls associated with the recent replication crisis: questionable research practices ( John et al., 2012 ), HARKing -- hypothesizing after results are known ( Kerr, 1998 ), gardens of forking paths ( Gelman and Loken, 2014 ), and p-hacking ( Schuemie et al., 2018 ). Avoiding these pitfalls is particularly important in a pandemic study since even preliminary results from individual studies can have profound policy and public health implications, as well as implications for ongoing clinical trials ( Piller and Travis, 2020 ). While the idea of preregistration in observational studies continues to grow in popularity, the effectiveness of the practice has notable limitations. For example, often the data has already been collected and been available for research prior to a study’s preregistration, making it hard to verify whether preregistration actually preceded the reported analysis.

Recall the assumptions necessary in order to make a causal claim. A key premise of an RCT is that the exposure assignment is random; in particular, exposure assignment is independent of factors that affect patient outcomes. To facilitate random exposure assignment, the study inclusion/exclusion criteria in an RCT must be designed to ensure that every trial participant can reasonably be assigned to any exposure group. Random exposure in an RCT is then accomplished by arbitrarily assigning people to either of the exposed or unexposed groups using a coin flip, or in the case of a stratified RCT, a coin flip that depends only on observed pretreatment factors. Our inability to achieve random exposure in an observational study means we must make some assumptions to estimate treatment effects when we do not observe all of the patients’ potential outcomes (e.g., both the exposed outcome and the unexposed outcome for each patient when there are two exposure groups). Here we state one of the acceptable sets of assumptions for conducting a retrospective analysis. First, theoretical overlap ensures that for any possible set of values of pretreatment traits (i.e., patient characteristics), there is a non-zero probability of being in either group. Lack of overlap might occur in practice if patients with certain characteristics are either excluded from the exposure group or always assigned to the exposure group (e.g., the exposed group only contains adults while the unexposed group contains both children and adults). Second, the property of unconfoundedness (also known as strong ignorability ) ensures that exposure assignment is independent of the potential outcomes given the observed covariates. Of these assumptions, overlap can be verified empirically, but there is no test to prove we have satisfied the unconfoundedness assumption.

Finally, we assume (both in observational studies and RCTs) that the specific exposure assigned to one individual does not interfere with the exposure or potential outcomes of any other individual in the study. For example, interference may occur when one patient in an RCT receives the exposure and is cured, which may then free up hospital resources to the benefit of an unexposed patient in an adjacent room. Furthermore, the exposure must be the same for everyone in an exposure group (e.g., identical drug regimen). Together, these two criteria comprise the Stable Unit Treatment Value Assumption (SUTVA) ( Imbens and Rubin, 2015 ).

A gold-standard randomized controlled trial satisfies all of these assumptions by construction; however, the lack of randomized exposure assignments in an observational study means there is significant work associated with emulating an RCT as closely as possible. It is almost certain that meaningful differences exist between the exposed and unexposed groups, and that the factors that differ are also related to outcomes. Confounding by indication is likely to occur in observational data, and the primary concern in your observational study is the identification and mitigation of potential confounders, which is the basis of Rule 3.

Rule 3 Supplement: Realize That Rule 2 Is Impossible and Proceed Carefully

Recall the different kinds of covariates in a causal analysis and how each can impact causal estimates. The lack of randomized exposure assignment in an observational study forces us to address the pretreatment variables that we observe in our data. Given that we are seeking to determine the causal effect of an exposure on an outcome, there are three types of observed variables that can exist in relation to this study. The first, outcome determinants, affect the outcome but do not directly affect the exposure. While you can include outcome determinants in your analysis to improve the precision of your causal effect estimate, a causal analysis can proceed without them. The second, exposure determinants, affect the exposure but do not directly affect the outcome. Exposure determinants will also not affect our analysis because there will be zero covariance between the outcome and the exposure conditional on these variables. A note beyond the scope of this paper: econometric analysis can reveal whether any of these exposure determinants is a strong instrumental variable. In this case, a separate instrumental variables analysis ( Hernán and Robins, 2006 ) is preferable for studying the effect of the exposure on the outcome by exploiting the fact that the instrumental variable’s effect on the outcome definitionally only exists via the exposure. The third type of variable affects both the exposure and the outcome; these are known as confounders and are the essential variables to identify for your study.

Think hard (and then think harder) about confounders for your study. As defined in the main text, confounders satisfy three properties: they are associated with the outcome (i.e., risk factors), they are associated with the exposure (i.e., they are unequally distributed among the exposure groups), and they are not effects of the exposure ( Jager et al., 2008 ). Identifying important confounders requires collaborating with specialists who can make appropriate clinical recommendations; for example, one might learn that there exists a comorbidity (an additional, simultaneously occurring disease or condition) for which patients would be taking the exposure drug. This comorbidity would be considered the indication or reason for prescribing the drug (as listed in the US prescribing information, though clinicians may prescribe for other reasons). Perhaps this comorbidity typically leads to worse outcomes given the worse overall health of these patients. Such a comorbidity would be a confounder; other common confounders include demographic variables such as age and sex.

Make a plan to address non-overlap and confoundedness. First, we must recognize that we only have data for observed confounders (as opposed to unobserved confounders, for which we have no data, and which in general lead to bias in estimates of causal effects). To address non-overlap, we must ensure that for any observed combination of confounder values, there are patients with very similar observed combinations of confounder values in each of the exposed and unexposed groups, even if presence in one group is more likely than another. If there are any combinations of confounder values for which the probability of exposure is either zero or one, it is impossible to estimate the treatment effect for patients with those confounder values. As a practical matter, the associated observations should be excluded to achieve overlap; the target population for which we estimate the treatment effect is correspondingly narrowed. To deal with confounders, we must mitigate the non-random exposure assignment in our data by ensuring similar distributions of confounder values between exposed and unexposed groups. There are two main approaches to doing so: outcome modeling and covariate balancing; when combined, the approaches may be doubly robust in that they are still valid if errors are made in either modeling or balancing (but not both), as discussed in more detail in Rule 8. Outcome modeling builds a model of the relationship between covariates and outcomes, allowing the analyst to adjust for the impact of differences in covariates across groups on differences in outcomes. Covariate balancing attempts to reweight or subsample from data such that the exposed and unexposed groups are comparable in terms of covariates, so that the covariates are no longer associated with exposure in the new, reweighted data; this can be accomplished, for example, through sample restriction with inclusion/exclusion criteria, reweighting by inverse propensity scores (probability of assignment), stratification, or matching ( Stuart, 2010 ) on confounders. Note that almost certainly there exists unobserved confounding in any observational study, and unobserved confounding distorts our view of the exposure-outcome relationship. If we believe there is an important unobserved confounder, it may be appropriate to abandon the study or use a different approach (e.g., instrumental variables analysis). We will address unobserved confounding in greater detail in Rule 5 and how to account for it with sensitivity analyses in Rule 9.

Rule 4 Supplement: Formalize the Research Goal

Specify the exposure in terms of quantity, duration, frequency, and recency. The study’s purpose is to evaluate the efficacy of this exposure, and this should dictate your first step in formalizing the research goal. The proposed exposure in a pharmaceutical-based hypothesis involves identifying a set of drugs for testing. At a minimum, this requires labeling each patient in the study as exposed or unexposed to one of the drugs in question; doing so requires completing two tasks. The first task is for the clinician team to specify the precise list of drugs and corresponding dosages they wish to include as the exposure drug set based on the pathophysiological mechanism they wish to target. The second task is to determine the timing of the observed drug exposure. For example, does it matter if the patient is a current, recent, or historical user of the drug at the time of the patient’s diagnosis ( Pazzagli et al., 2018 )? How long must a patient have used the drug to be part of the exposed group? These questions directly relate to the pathophysiological mechanism the proposed treatment aims to target, and the answers to these questions may have implications for the degree to which the study can truly emulate an RCT. Note that every consideration above also applies to analysis of a non-pharmaceutical exposure. Investigating the effectiveness of a non-pharmaceutical therapy requires the same attention be given to defining the precise list of qualifying therapies as well as the quantity, duration, frequency, and recency of any treatment a patient received.

Define the comparison groups of interest (e.g., define unexposed). If you could do a randomized experiment, what other exposure groups would you randomly assign people to for comparison? In a pharmaceutical study, this could include taking a placebo, taking an active comparator (an alternative treatment known to be effective), or even taking the same drug according to a different regimen. Defining a comparison condition requires the same level of detail required for the exposure definitions. Most likely the comparison condition represents the existing standard of care, and the purpose of the study is to see if the hypothesized exposure provides an improvement over the standard care. As you define the exposure and comparison conditions, it may well be the case that some individuals meet none of these group definitions and must accordingly be excluded from the study. For example, some patients may fall just short of qualifying as exposed (e.g., too few days on the proposed drug treatment, too small a dosage), but their classification as unexposed would be inappropriate as well.

Define an outcome that is specific, measurable, and sufficient to answer the research question. Defining an outcome includes clearly stating exactly what will be measured, when it will be measured, and how it will be measured for all patients in the study. The outcome must be observable in a consistent manner for all patients in your study. Thoughtful consideration should be given to the followup time required to observe the outcome in both exposed and unexposed patients. Additionally, for outcomes other than mortality, competing risks may prevent observing the outcome of interest (e.g., loss to follow-up in a lengthy study).

Formalize your hypotheses. At this point in the team’s preparation for the study we have clearly defined the exposure(s) and outcome(s) and are ready to articulate the causal effect of interest. This involves clearly stating the specific null and alternative hypotheses your analysis will test; determine if a one-sided or two-sided test is more appropriate for your medical hypothesis. Commit to the primary and secondary exposure and outcome definitions, target population, and outcome-focused results you believe will produce a credible analysis. Note that the hypothesis is based on definitions that reflect what you hope to observe, and they may not be what you can actually find in an available data set (discussed further in Rule 7).

Example Application of Rule 4 to the COVID-19 Study

This retrospective study estimates the causal effect of baseline use of doxazosin (daily dose ≥4 mg with prescriptions covering the day of COVID-19 diagnosis and at least 80% of the previous 3 months) compared to nonuse (no prescriptions for any alpha blocker in the previous year) on reducing all-cause mortality in adults over 45 years old who have been diagnosed with COVID-19. We state the following hypotheses for the odds ratio (OR) associated with the treatment effect on all-cause mortality:

Rule 5 Supplement: Identify and Reason About Potential Confounders

Confounders will be present; make every effort to observe these confounders and adjust for them appropriately. Consider a study wherein patients are prescribed a drug to treat a certain disease with varying degrees of severity. A high dosage tends to be prescribed for patients with a more severe case of the disease, whereas a low dosage tends to be prescribed for patients with a less severe case of the disease. It would be no surprise to find that patients with severe cases have worse outcomes as a group - even if the drug (and dosage) they are taking is the best option for their individual situations. In observational data, dosage level is inherently related to severity of illness. Hence, severity of illness is a confounder because it affects the exposure-outcome relationship; if left unobserved, severity of illness could irreparably confound any study results. The circumstances surrounding the administration of an exposure can also make observing confounders challenging. For example, suppose we are studying the efficacy of a drug for preventing death from an acute condition, and the drug is typically given as a last resort to patients who are nearing death from that condition. Then it may be difficult or impossible to observe the factors that affect both exposure and outcome, since not all factors that lead a physician to believe that the patient is at high risk of death will be recorded. During some time periods in the COVID-19 pandemic, different drugs (such as hydroxychloroquine) were given off-label to the sickest patients. In such circumstances, receiving the drug is an indication that the patient was very ill. In contrast, if we study exposure to a drug that was prescribed for a chronic condition long before a patient developed COVID-19, then exposure will not be determined by the patient’s severity of symptoms from COVID-19. For example, some underlying factor such as hypertension might be related to both drug exposure and risk of poor outcomes from COVID-19, so it will still be important to carefully adjust for all such factors.

Include standard demographic variables. Common demographic covariates such as sex and age (including nonlinear transformations like age-squared) are standard confounders to consider, appearing in nearly all epidemiological models. Another variable to consider is the time or location of the sample-defining diagnosis (e.g., a positive lab test or clinician diagnosis). Diseases like influenza often change from year to year in terms of which strains are more prevalent, and the geography of outbreaks may not be uniform. Depending on how fast a disease mutates or the standard of care changes, capturing the year, month, or even week of diagnosis, and/or hospital or patient location, may be important covariates when examining observed outcomes.

Include relevant comorbidities. A confounding comorbidity is one that impacts both exposure assignment and outcomes. Other comorbidities may be unrelated to the proposed exposure but could still be helpful as proxies for confounders by identifying which patients are already at higher risk for severe outcomes based on components of their health beyond basic demographics (e.g., cancer or heart failure). Still other comorbidities might serve as proxies for the proposed treatment; running an analysis that includes these comorbidities may lead to “post-treatment bias” because the comorbidities would appear as concurrent treatments, hence reducing the estimated treatment effect of the actual treatment. Post-treatment bias can also result from considering post-treatment traits. For example, controlling for emphysema when examining the causal effect of smoking on lung cancer would likely transfer some of the treatment effect from smoking to emphysema, which we might assume to have resulted from smoking. Choosing to consider a confounder that was observed post-treatment requires a deliberate assessment of the potential causal relationship between the exposure and the observed trait. For example, if an observed comorbidity is of a chronic nature, it may be unlikely that a recent exposure caused the comorbidity; most likely the unrelated condition prompting the exposure led to the healthcare encounter where the comorbidity was first diagnosed. Another class of variable to avoid is known as a collider. A collider is a variable that can be considered an effect of both the exposure and the outcome; controlling for such a variable introduces bias in the effect estimate.

Include a comorbidity index and/or other indicators of overall health. The Elixhauser comorbidity score ( Elixhauser et al., 1998 ) and Charlson comorbidity index ( D’Hoore et al., 1993 ) are two established measures combining various observed medical conditions in order to serve as more general indicators of overall health than an individual, disease-indicating covariate. The potential for unobserved, general health problems can also be addressed by looking at a patient’s recent health care encounters and prescription data. Encounter-related covariates may include the number of inpatient or outpatient visits occurring in the year preceding the relevant diagnosis, the duration of inpatient stays (i.e., the number of days the patient had been in the hospital in the previous year), and indicators for whether the comorbidities listed above were observed closer in time to the relevant diagnosis (e.g., within two months prior rather than within one year prior). Considering the recency of documented health concerns is useful for establishing whether a declining health trend exists both at the individual level and at the level of comparing different exposure groups. You may also want to consider certain procedures in addition to diagnoses (e.g., colonoscopies, flu shots ( Jackson et al., 2006 )), which can also serve as indicators of overall health and/or access to health care. As with all of our confounders, remember to ensure that any indicators of overall health only capture pretreatment health conditions.

Know that no matter what you do, you will likely still have unobserved confounding. Failing to include unobserved confounders in an analysis leads to omitted variable bias, which violates the unconfoundedness assumption. As indicated above, the missing confounders we are most concerned with relate to unobserved indications of poor or declining health; however, these may not always be available. If you determine a set of critical confounding variables and find that some are unobservable (either directly or via a proxy variable), we can investigate the potential magnitude of this unconfoundedness violation (in some cases, your proposed study may be too flawed to justify pursuing it). There is certainly a bit of tension here as we perform analysis under the assumption of unconfoundedness while simultaneously acknowledging the likelihood of unobserved confounding. We address this tension with sensitivity analyses described in Rule 9.

Example Application of Rule 5 to the COVID-19 Study

This retrospective study considers the following confounders: sex, age, diabetes mellitus, hypertension, cardiovascular disease (acute myocardial infarction, ischemic heart disease, heart failure), chronic obstructive pulmonary disease, patient location, Elixhauser comorbidity score, inpatient stays in the prior year, inpatient stays in the prior 2 months, inpatient days in the prior year, and inpatient days in the prior 2 months.

Rule 6 Supplement: Operationalize the Target Population

Select the target population for your observational study to reflect the intended RCT population. Patient selection is a key task in RCTs, and an observational study emulating an RCT should implement the same inclusion and exclusion criteria as the RCT. Given that an RCT likely excludes individuals with certain comorbidities, one benefit of an observational study is the opportunity to conduct a subanalysis of individuals that the RCT would exclude.

Refine the potential study population by expanding the inclusion and exclusion criteria to minimize confounding. In Rule 5 we described many types of potential confounders; in Rule 6 our objective is to find a subset of the population who may receive the exposure of interest for reasons that have minimal expected impact on the outcome of interest (i.e., minimal confounding); importantly, these individuals should also include candidates to remain unexposed. There is no rule of thumb for this, but rather it is through the creative efforts of your team that you can specify a target population refinement that can still potentially answer the research question while significantly reducing confounding. Note that changing the sample inherently changes the estimand, and there is often a tradeoff between studying the population that is of greatest interest and studying the population where estimates are most credible.

Consider the impact of refining the target population on internal and external validity. Minimizing confounding is desirable as it increases the internal validity of the study, but excluding certain groups from the study may limit the external validity of the results to only the refined population under study ( Imai et al., 2008 ; Rudolph et al., 2014 ). Consider again a scenario where a drug is administered in some cases for conditions with serious health risks and in other cases as more of a lifestyle drug. If we exclude from our study any patients with the more serious condition, we can likely achieve more similar exposed and unexposed groups, which is important for attributing any difference in expected outcome to the exposure under investigation. The cost is not knowing how those with the more serious condition fare with the exposure versus without the exposure. Additionally, there is an important emerging literature on demographic fairness with regard to clinical studies ( Holdcroft, 2007 ). Be careful in your efforts to minimize confounding so that you do not unintentionally or unnecessarily exclude a portion of the population that also requires study.

Example Applications of Rule 6 to the COVID-19 Study

1) This retrospective study focuses on adults over 45 years old to maintain internal validity for all older adults. 2) This retrospective study focuses on adult men over 45 years old to minimize confounding by focusing on a large group of people that use doxazosin for a condition unlikely to affect COVID-19 outcomes (BPH).

Rule 7 Supplement: Get the Best Data for the Study

Invest time in getting access to the best possible data for your study. Above all else, this means the target patient population is sufficiently represented in the dataset. Recognize that data access and sharing may be challenging; any health care data you use will often have data access restrictions due to legal and/or privacy concerns, proprietary interests, or other competitive barriers ( Byrd et al., 2020 ). Typically, IRB approval, an IRB waiver for de-identified data, or business associate agreements enable data access and permit its use for your specific research objective.

Know what your data source contains, where it originated, and how it was assembled. Having someone on the team who knows the data source well helps the team avoid the early stumbles that inevitably happen while working with new data. The best data sources will capture data on the population, exposure, outcomes, and covariates relevant for a study. Once you acquire access to potential datasets, consider the reliability of the data collection (e.g., provenance, missingness, measurement error, trends over time, and sampling or representativeness of the target population). While we recommend defining your ideal exposure(s), outcome(s), and target population first, you may have to revise some of these definitions to be compatible with the existing dataset or combination of data sources (e.g., claims data, labs, or electronic health records from multiple participating hospitals).

Know the biases and limitations of candidate datasets. It is likely the case that no single data source is sufficient to represent the broader population. The ideal data source would have extensive electronic health records with thorough patient histories documenting inpatient and outpatient encounters, diagnosed conditions, and drug prescription and fill data. Outside of national healthcare systems or other integrated systems such as the US Veterans Health Administration (VHA) and Kaiser Permanente, obtaining all relevant information about a specific patient from a single source is rare. Often, hospital data will not have extensive pre-hospitalization data (if any), and claims databases will lack the rich details of hospital records (e.g., clinicians’ notes and lab results). Further, observed outcomes in patient groups from different data sources may not always be indicative of what is expected in the broader population. Certain types of hospitals (e.g., tertiary care centers) may handle more advanced cases of a disease and have higher rates of certain outcomes in their electronic health records data. Some insurance claims databases may only represent the portion of the population that is employed, has healthcare insurance, and has demonstrated access to healthcare services. Each data source may also be idiosyncratic according to varying standards of care and coding practices for the time, location, and patient groups it represents. The information that appears in health data can also reflect payment systems and incentives; for example, minor hospital procedures may not appear in claims databases because insurers may not pay for them directly. It is important to know and understand these issues before trying to run your models across different datasets, only to be confused by the inconsistent results. The best approach is to evaluate your hypothesis using as many appropriate data sources as possible and look for consistently observed effects across data sets.

Obtain a sample of the target population using carefully selected, standardized codes. The typical way of identifying patients for a cohort study involves selecting patients with a documented record of a particular disease or medical procedure, most often by means of an International Classification of Diseases (ICD) code (e.g., ICD-10-CM Clinical Modification). Many diseases and procedures have a large number of codes delineating the various subtypes of the disease (e.g., pneumonia) or procedure (e.g., mechanical ventilation), so a careful inspection of the potential list of qualifying condition codes is necessary to properly define the intended sample. If possible, attempt to validate the cohort by also checking for confirmatory lab tests and/or prescribed medications, which may or may not be available in your data.

Identify chronic comorbidities using standard condition code sets and sufficient patient histories. The data you will need for a cohort study must contain some mechanism for observing the confounders you identified in Rule 5. Diagnoses for comorbidities, much like the diagnoses used to define our target patient population, can include a broad range of ICD codes for each disease or condition. Identify comorbidities by using a standard set of ICD codes that medical researchers generally agree encompass the common comorbid conditions, such as the Chronic Conditions Data Warehouse (CCW) ( Chronic Conditions Data Warehouse, 2020 ) produced by the Centers for Medicare & Medicaid Services (CMS). You will need reasonably long-duration patient histories (e.g., 12+ months of inpatient and outpatient records preceding the diagnosis meriting inclusion in your study's cohort) to ensure adequate opportunity to observe relevant comorbidities in patient records. As a general rule for most chronic conditions, we recommend considering a patient to be positive for a given chronic condition if any of the listed condition codes in a standard code set is referenced as a diagnosis on any inpatient or outpatient record in the 12 months preceding the qualifying diagnosis. In turn, researchers should exclude any patient that cannot be tracked in the data for that entire lookback period (e.g., in insurance claims data, if the patient was not continuously enrolled during that time). The clinicians and data source experts on the team should determine whether any alternate criteria should be considered (e.g., multiple codes, multiple occurrences, different lookback period, lab values, and procedure codes).

Make your study definitions realizable in your data. It should be expected in database-facilitated research that not all desired quantities may be available. For example, rarely can we know what medication a person actually consumed; instead, we observe what was prescribed and filled. An insurance claims database does not generally record indicators of a patient’s lifestyle such as body mass index (BMI), alcohol use, and smoking status (though they could be very useful); they may not record certain demographic and socioeconomic data (also relevant for many diseases and hypotheses). Instead, an insurance company needs to know which diagnoses were given and which procedures were administered for claims reimbursement purposes. As you look for data that allow you to operationalize your study definitions for exposure, outcome, confounders, and target population, you may be forced to adjust those definitions to reflect what is in the data. You must carefully assess whether what you do observe is close enough to what you wish you could observe to be sufficient for the research question.

Example Application of Rule 7 to the COVID-19 Study

This retrospective study uses Veterans Health Administration data with patients identified according to the National COVID Cohort Collaborative’s COVID-broad criteria. Pretreatment comorbidities are identified by searching each patient’s inpatient and outpatient records (electronic health records or insurance claims) for the presence of a qualifying ICD code for each of several comorbid conditions according to the comorbidity-specific ICD code sets provided by the Chronic Conditions Data Warehouse.

Rule 8 Supplement: Explore and Model Your Data With Surrogate Outcomes

Use permuted outcomes or synthetic data as you build and test your analysis code. In an RCT, blinding prevents patients and clinicians from knowing exposure group assignments, which might affect their respective actions. In observational studies, the concept of blinding relates to only seeing what you have to see to accomplish a certain task. Research team members can be blinded to the exposure, the outcome, and potentially even the hypothesis ( Berman and Parker, 2016 ). We start this rule by blinding ourselves to the outcome because all code goes through a debugging phase, and there is a risk that, at least subconsciously, you might be influenced by frequently seeing a range of results from different methods, confounder/covariate sets, etc. As you proceed with your analysis, you may discover that certain covariates are either sufficiently sparse or so highly correlated with other covariates that issues of numerical stability arise with certain modeling approaches. As you encounter these issues and fine-tune your list of covariates, it is best that these modifications be made without subjective bias arising from prematurely observing any effect estimates. Remember, the purpose here is to specify the details of the analysis plan and to implement working code, not to produce a final causal effect estimate just yet. If a step can be performed with surrogate outcome data for the purpose of testing, it should be.

Examine the univariate and pairwise distributions of the variables (or covariates) that will be used in the analysis. This serves to assess any issues with missingness, data entry errors, and the accuracy of any constructed variables. Also important is the opportunity to assess these distributions for their adherence to known or believed attributes of the population under study.

Examine all covariate distributions after stratification by exposure group and/or time period. A key claim in any retrospective analysis, as mentioned in Rule 3, is that the exposed and unexposed groups either have similar covariate distributions or that the authors have done something to address the fact that the distributions are meaningfully different. The difference in the exposed and unexposed groups’ covariate distributions is typically referred to as “covariate balance,” which should be calculated and visualized before and after employing certain types of models ( Austin, 2009 ).

Achieve better empirical overlap using propensity trimming. Propensity scores quantify each patient’s likelihood of receiving the exposure conditional on the observed covariates. There may exist observations in your data that possess combinations of covariate values that are only ever observed in either the exposed group or the unexposed group, but not in both (leading to uncommonly high or low propensity scores). This violates the overlap assumption we required in Rule 2 (while this statement applies as written to categorical variables, a relaxed version still applies to continuous variables where exact matches are unlikely). A standard technique to maintain overlap is to remove such observations from the data by trimming on the basis of propensity scores (i.e., restricting the sample to areas with propensity score overlap). There are many common approaches to calculating propensity scores; the R packages grf , twang , and MatchIt calculate propensity scores using honest forests, generalized boosted models, and logistic regression, respectively ( Ho et al., 2011 ; Athey et al., 2019 ; Ridgeway et al., 2020 ) (note that some machine learning models are characterized by bias or inconsistency in estimates of propensity scores, and so properties such as honesty as implemented in grf may be important if machine learning methods are used in propensity score estimation). The distributions of propensity scores in the exposed and unexposed groups are then used to identify and trim (remove) observations that are in the extremes of these distributions and have few or no counterparts in the other exposure group with a similar propensity score. This process ensures that in every region of the preserved covariate distribution, there exist observations in both the exposed and unexposed groups. Thus, overlap ensures we are estimating a causal effect over regions of the covariate distribution supported by data rather than through extrapolation. Achieving this overlap is how we most closely emulate the RCT reality in which every patient has some positive probability of assignment to each exposure group. Note that the groups as a whole could still look quite different (e.g., in terms of comorbidity prevalence).

Use an unadjusted modeling approach to establish a baseline treatment effect estimate. Assuming two exposure groups and two potential outcomes, start with any method operating on 2-by-2 contingency tables; you could use Fisher’s exact test, the chi-square test for association, or a basic logistic regression model to evaluate the exposure-outcome association with no adjustment for any confounders. Importantly, you want to obtain point estimates and confidence intervals (CI) from these methods as we are concerned with the magnitude and precision of the treatment effect estimate. Despite our repeated emphasis on identifying and accounting for confounders, having an unadjusted model result that is compatible with the adjusted model results (described next) demonstrates that you have not reached your final treatment effect estimate simply by selecting a favorable set of covariates. When unadjusted and adjusted results disagree, one explanation could be dissimilarities in the covariate distributions of the exposed and unexposed groups. For example, if certain ages or comorbidities are not approximately equally represented in all exposure groups, controlling for such covariates could potentially change the sign of the estimated treatment effect. This could be evidence that your inclusion/exclusion criteria do not by themselves go far enough to yield similar exposed and unexposed groups.

Adjust for confounders, favoring methods that both adjust for outcomes and seek covariate balance. Methods that adjust for outcomes build a model mapping covariates to expected outcomes and then adjust for these differences when estimating treatment effects. Ordinary least squares or logistic regression are common methods for outcome adjustment; machine learning methods can also be used, but caution must be exercised, as there is a danger that regularization might omit or insufficiently adjust for confounders, creating bias ( Belloni et al., 2014 ). Covariate balance goes beyond ensuring overlap: now the exposed and unexposed groups must resemble each other in their covariate distributions. More simply, observed values in the exposed group should occur with similar frequency in the unexposed group (either by weighting or excluding observations). Methods that accomplish this include inverse propensity-weighted (IPW) average of outcomes and matching ( Rubin, 2001 ; Stuart, 2010 ; Jackson et al., 2017 ).

There are many choices of regression methods that adjust for confounders; among these are a set of methods known as doubly robust methods. A doubly robust estimator is one that employs both a propensity score model and an outcome regression model in such a way that if either model is correctly specified, the resulting causal effect estimator is statistically consistent ( Bang and Robins, 2005 ). An example of a doubly robust method is inverse propensity-weighted (IPW) regression. Inverse propensity score weighting seeks covariate balance by weighting unexposed observations in the regression according to the inverse of their propensity scores ( Austin and Stuart, 2015 ). Thus, observations that do not resemble exposed observations contribute less to the treatment effect estimate, and unexposed observations resembling exposed observations count more. This type of weighting has the effect of attempting to achieve covariate balance by weighting observations rather than excluding observations. Other examples of doubly robust methods include augmented inverse propensity weighting or AIPW regression and causal forests ( Bang and Robins, 2005 ; Athey et al., 2019 ). We note that if machine learning techniques are used to estimate outcome models and propensity scores in AIPW methods, it is important to use cross-fitting, where the outcome adjustment and propensity score model for a given observation is estimated excluding that observation. When out-of-bag estimates are used with random forest methods, this will happen automatically, but with other methods, the analyst must estimate multiple versions of these models on different folds of the data.

As an alternative to the above doubly robust methods, one can employ matching methods to stratify the sample into one group per exposed observation. Groups or “matched pairs” are sized such that each exposed observation has a corresponding number of unexposed observations according to a specified match ratio. Importantly, the matching process should only retain the exposed observations for which an acceptable number of unexposed observations serve as good matches. This is the nearest you can get to seeing how a person’s potential outcomes might be different on the basis of exposure. Matching can be accomplished many ways, including on the basis of propensity score or Mahalanobis distance ( Stuart, 2010 ). To estimate the causal effect of the exposure on the outcome in the matched pairs, one might use the Cochran-Mantel-Haenszel test ( Mantel and Haenszel, 1959 ) to evaluate the collective evidence presented by a series of 2 × 2 contingency tables documenting the exposure-outcome counts in each matched pair. The process of matching could produce a potentially much smaller data set that attempts to achieve covariate balance by excluding observations.

For methods that rely on covariate balance as part of the approach to adjust for confounders, it is critical to conduct appropriate diagnostics to see if these approaches achieved acceptable covariate balance. If you are unable to achieve reasonable covariate balance between exposed and unexposed individuals, you have likely discovered fundamental differences in the two groups that no modeling approach can reliably overcome ( Glynn, 2017 ).

Example Application of Rule 8 to the COVID-19 Study

We first create a permuted copy of the outcome variable representing in-hospital death. We use the R package grf to estimate propensity scores (i.e., real exposure assignments as a function of the pretreatment traits identified in Rule 5). We then trim the sample to retain the overlapping region of the exposed and unexposed propensity score distributions by keeping scores above the maximum of the two distributions’ first percentiles and below the minimum of the two distributions’ 99th percentiles. With the remaining sample, we perform an unadjusted analysis of the exposure-outcome relationship with Fisher’s exact test (OR, CI, and p -value obtained with base R Fisher exact test). We conduct an adjusted analysis using the same pretreatment traits in an inverse propensity-weighted (IPW) logistic regression (OR, CI, and p -value obtained with the R package survey ). We use the R package MatchIt to execute 5:1 Mahalanobis distance-based matching (identify five unique, unexposed matches for each exposed patient) on the same pretreatment traits (OR, CI, and p -value obtained with base R Cochran-Mantel-Haenszel test). Finally, we assess the covariate balance achieved by IPW and matching by calculating and visualizing standardized differences of means for included covariates. Executing all of these steps with permuted outcomes helps us debug code, identify potential incompatibilities with our data and selected methods, and conduct meaningful diagnostics for covariate balancing methods — all with zero awareness of the impact on our treatment effect estimates.

Rule 9 Supplement: Augment the Main Analysis With Extensive Sensitivity Analyses

Plan a thorough assessment of the robustness of your results to the various choices you made on the way to calculating an estimated treatment effect. Maybe you left something out that could explain everything (i.e., an unobserved confounder). Do alternative design and analysis approaches yield similar results? A secondary set of analyses could include adjusting for covariates with nonlinearities or time lags; you could also try different regression or propensity estimation methods. There could be many reasonable specifications for your model; to avoid tying your results to a set of arbitrary decisions, one way to evaluate a collection of reasonable models is to observe the distribution of resulting effect estimates using specification curve analysis ( Simonsohn et al., 2019 ). Exploring different exposure or outcome definitions, covariates, designs, and analysis techniques also helps measure the sensitivity of your results to the specific choices you made along the way. Assessing robustness is by itself a comprehensive analysis.

Quantify the extent of unobserved confounding required to change your conclusions. If you are using observational health data to perform your study, you should expect that unobserved confounding exists; the difficulty lies in estimating how serious it is. There is no test for unobserved confounding (neither its existence nor its impact, given that it is unobserved), yet it likely exists in nearly all observational studies. This reality is what makes having domain experts carefully reason through confounder specification so critical. Starting with ( Rosenbaum and Rubin, 1983 ), numerous approaches have been proposed that generally aim to estimate how strongly correlated an unobserved confounder would have to be to either the exposure, the outcome, or both, to move the estimated treatment effect to the null ( Rosenbaum, 2010 ). Then you can reason about how likely it is that such a confounder might exist and is either unknown or unmeasurable. One such method for assessing unobserved confounding is the E-value ( VanderWeele and Ding, 2017 ).

Assess the robustness of your results to choices regarding specific modeling techniques, hyperparameters, etc. One way to accomplish this involves trying a range of estimation approaches. Compare the treatment effect estimates from a range of doubly robust methods, for example. Use a variety of machine learning methods to estimate propensity scores and outcome models in doubly robust methods such as AIPW, or use approaches such as residual balancing ( Athey et al., 2018 ) that do not rely on having an easy-to-estimate propensity model. The reason to augment your analysis by testing multiple approaches is to see if the obtained results were sensitive to the specific methods you chose to employ. While the methods introduced so far are designed to estimate average treatment effects for a population or some subset of the population, knowing whether the treatment effect is generally constant across the considered group can be very important. To explore this, one can construct causal trees to estimate heterogeneous treatment effects or HTE ( Athey and Imbens, 2016 ).

Assess the robustness of your results to modifications in the study definitions and study design. You can make small changes to the definitions of the exposed and unexposed groups as well as the outcomes and confounders. For example, to identify a patient as a user of a particular drug, adjust the aforementioned medication possession ratio or look-back period in the exposure definition (i.e., ensuring a medication supply of more than 50, 70, or 90% of days within a look-back period of 90, 180, or 365 days). You can consider different recency requirements such as whether the most recent prescription spanned the inpatient admission date of interest. For an outcome like all-cause mortality, you could explore all-cause mortality in the hospital or within 7, 14, 30, or 60 days of diagnosis. Comorbidity identification could employ different code sets and/or a different look-back period. You may also consider adjusting for additional (or only a subset of) potential confounders within your models, to observe the extent to which confounder choice matters. The objective here is to see whether or not any observed treatment effect is simply a chance result stemming from a very specific set of definitions. Some of these changes are sufficient to change the study design. For example, defining the unexposed group to only include users of a different, comparable drug is known as the active comparator design , which can be an effective approach for minimizing confounding as the exposed and unexposed groups will be more similar ( Yoshida et al., 2015 ). If we define the exposed group to only include new users of a drug, thus ensuring observed comorbidities existed before exposure and eliminating concerns over prevalent user bias, we are implementing a new user or incident user design. There are many study designs to choose from (e.g., prevalent user, incident user, active comparator, etc.), and each design deserves thoughtful consideration regarding the implications it has for the study in question and physiological mechanism under investigation. While investigating robustness to changes in study design can provide more evidence for the hypothesis, it can also help identify potential sources of unobserved confounding when different designs lead to different conclusions.

Explore additional sets of covariates, including different comorbidities and indicators of temporal health trends. Covariate sufficiency is the notion that no other covariate can meaningfully supplement what we have learned from the already identified covariates ( Stone, 1993 ; VanderWeele and Shpitser, 2013 ). We can explore the sufficiency of our identified confounders by observing how results are impacted by the inclusion of other comorbidities. We can also explore the impact of differing time trends in the health of the exposed and unexposed populations. If one exposure group was observed to be getting sicker faster in the months before the target inpatient admission, that could warrant different expectations for outcomes in the exposed and unexposed groups. Your confounder definitions may have difficulty addressing not only the presence of a condition, but also its recency and its severity. Many comorbidities have their own severity indices (e.g., Diabetes Complications Severity Index), but viewing all the data required to compute these scores may not always be possible in certain data sets (e.g., claims data lacks lab results). Observing health decline is thus challenging; consider examining recent inpatient stays and other medical encounters as signs of declining health that may not otherwise be captured in existing confounder definitions.

Conduct negative outcome experiments and treatment control experiments. In a negative outcome experiment ( Lipsitch et al., 2010 ), your goal is to assess whether the hypothesized exposure has an apparent benefit that extends to an outcome it could not reasonably impact (i.e., no medical theory connecting the exposure to the outcome). A negative outcome experiment is run to study the effect of the proposed treatment on an outcome not associated with that treatment. Here, we should expect to find no favorable treatment effect; otherwise, there is likely unobserved confounding contributing to better outcomes for the exposed group. A treatment control experiment is run to study a different treatment with no known connection to the outcome of interest; you should observe no protective effect of this different treatment on your original outcome. Again, if you see a benefit where there should be no benefit, the logical conclusion is the presence of unobserved confounding.

Refine, lock in, and preregister your formal analysis plan before examining any real model outputs using the true outcome data. Preregistration for observational studies involves uploading a detailed analysis plan to a study registry like the ones supported by the US National Library of Medicine ( clinicaltrials.gov ) and the Center for Open Science ( cos.io/initiatives/prereg ). While we encourage preregistration, in some cases it may not be possible to preregister an analysis plan before ever seeing the data; your understanding of the data prior to working with it may be too limited to make preregistration worthwhile. Preregistering your analysis plan is an attempt at transparency regarding what is exploratory and what is confirmatory in your final analysis. You may discover some things while exploring your data and testing your proposed statistical methods that require you to refine prior decisions. Maybe your set of confounders and outcome determinants is incompatible with a method you’ve chosen because one variable is too rarely observed or is too highly correlated with another variable. This is fine; you can make the necessary changes to your analysis plan with no fear of p-hacking because you were not using real outcomes (due to outcome permutation or synthetic data generation per Rule 8) and have not seen an effect estimate yet. Your preregistered analysis plan may include a range of exposures, outcomes, and modeling approaches you intend to evaluate, but you must clearly articulate from among these which combination you commit to reporting as your primary result. Define your primary result with a clear statement of the hypothesis, details of the modeling approach, and definitions for the cohort, treatment, outcome, and confounders.

Example Application of Rule 9 to the COVID-19 Study

We assess robustness to unobserved confounding with the E-value. We estimate the treatment effect with different exposure definitions, specifically combining 50, 70, and 90% MPR with 90-, 180-, and 365-days exposure windows. We estimate the treatment effect using AIPW and heterogeneous treatment effect with causal trees as supplementary methods. We consider mortality within 30 days of diagnosis as an alternative to in-hospital mortality. We perform a negative treatment control experiment with triptans as the exposure. We perform negative outcome control experiments using accidental injuries and non-prostate cancer as alternate outcomes.

Execution Phase: Execute the Analysis Plan and Report the Results

Rule 10 supplement: execute, summarize, and share (with caveats).

Execute your analysis plan with the true outcome data once you are satisfied with the quality of your data set and have sufficiently tested your code. A significant responsibility of your team at this point is to stick to the proposed analysis plan. Other outcomes and exposures may appear to have a stronger effect than what is observed for the primary outcome and exposure, but there was significant thought and clinical expertise applied to these decisions in the planning phase of the study. There is danger in evaluating a host of different outcomes and only reporting the most favorable outcome(s); this greatly increases the potential for a Type I error, meaning that you could be reporting a treatment effect that does not actually exist.

If necessary, make the smallest possible refinements to your analysis plan and execute again. Even with all your planning, there is a chance that your analysis plan cannot be executed as-is. For example, you may discover that a rarely observed confounder in your data is perfectly predictive of the outcome in one of your exposure groups. This perfect separation of the data could cause your preferred method to fail, leaving you no choice but to change one of your selected methods or your selected confounders or both. If this happens, all is not lost. Simply make the minimal possible change necessary to conduct your analysis, and then note in your publication how you had to amend your analysis plan and what potential impacts your change may have had on your results.

Give your reader something that looks like what they are used to seeing. If your retrospective analysis has the stated purpose of motivating a clinical trial, write your results like a clinical trial paper. Include a CONSORT flow diagram to help the reader visualize important properties of your sample. Understand how the intended audience expects to see results reported for the selected outcomes. The clinician audience you are writing for is accustomed to seeing odds ratios with corresponding confidence intervals to describe treatment effects. Presenting results in a conventional way eliminates one potential obstacle your audience may face when evaluating your work. While much attention is given to your primary result, your results in total are more than just an OR and a confidence interval; report the results of your sensitivity analyses as well to convey the robustness of your finding.

Explicitly include in your reporting the limitations of your study. You have not just completed an RCT; instead, you performed an observational study modeled after an RCT, but with many limitations and assumptions. Your biggest enemy is unobserved confounding, and it might be the case that it has seriously affected your results; however, if done well, your retrospective analysis may be just what is needed to generate the momentum and funding required to evaluate your idea in a clinical trial ( Vandenbroucke, 2004 ). Alternatively, your analysis may actually provide evidence against the hypothesized exposure. Reporting negative results is just as important; your work can help ensure limited resources are spent on more promising treatments.

Provide all the necessary details to facilitate replication. You took great care in constructing and executing a comprehensive analysis plan; as you prepare to disseminate your findings, sharing those details matters. More than just your results, some readers will want to know everything necessary to reproduce your analysis. This means you should expect to provide details about the data used, including source and provenance as well as the codes (e.g., ICD) used to define the target patient population, inclusion/exclusion criteria, the exposure(s), the outcome(s), and any confounders. It can’t be assumed that a reader will be able to guess your definitions without having them explicitly written out. Other researchers could sensibly reach many different definitions of what they believe you meant by the various outcomes, exposures, and confounders listed in your retrospective analysis. Providing text definitions, formulas, and ICD-code lookup tables ensures that any other attempts to implement your definitions are able to accurately do so. Providing all of this information in the standard organization of a clinical trial paper will help your clinical audience find the key pieces of information they need to be able to envision the trial you are emulating.

Facilitate replication by providing analysis code. You may also want to create an open-source software package (e.g., R/Python) for dynamic exploration of a data set and/or to facilitate replication of your analysis on other data sets. It is likely the case that other entities (e.g., a hospital, an insurance company, or a country) cannot legally share their data set with you; you likely have the same restrictions preventing sharing your data outside your own institution. To get around these restrictions and make replication as easy as possible, you can share instructions and code for building the data set and running your desired analysis. Whether you provide a well-documented collection of scripts in an online Git repository or a more formal software package, if you want to see replication of your results (e.g., to support an RCT you aim to start), you have an incentive to provide a reusable codebase that can facilitate rapid replication of results in other data sets as well as provide a means of quickly exploring alternate hypotheses.

Future Directions

These 10 rules are intended as introductory guidelines to one small piece of the complicated world of observational studies; there is much more to learn and consider than is offered here. Perhaps most importantly, we acknowledge this paper’s role in summarizing a framework for retrospective pharmacoepidemiological analyses, not as a template for all types of retrospective studies (e.g., investigating lockdowns and facemask policy effectiveness against the spread of COVID-19). Several other ideas came up in the course of establishing these 10 rules that fell just short of earning their own rules. Some are not yet standard practice but are growing in popularity, and others are even more aspirational. Among these are notions of sample splitting ( Fafchamps and Labonne, 2017 ) and model pooling. Sample splitting in the world of machine learning is standard practice, but typically the machine learning problem is one of prediction where there exists validation data, making it possible to know how correct a model’s predictions are and therefore tune the model. The causal inference framework differs on both those counts: prediction is not the goal, and there exists no validation data to help us see if we have missed any unobserved confounders. While sample splitting may not always be necessary, when doubly robust techniques are used and machine learning methods are used to estimate outcome models or propensity scores, cross-fitting is needed to apply existing theory ( Chernozhukov et al., 2018 ; Athey et al., 2019 ); we recommend that approach as discussed in Rule 8. There is still interest, however, in using synthetic data generation techniques such as generative adversarial networks ( Beaulieu-Jones Brett et al., 2019 ; Athey et al., 2021 ) and standard training/test splits for routine tasks like evaluating a constructed feature definition and validating code. Employing these or related techniques aims to facilitate completion of necessary tasks without being influenced by real-world results. Another growing area of interest is in the pooling of data and models from observational studies ( Bareinboim and Pearl, 2016 ). Privacy concerns often restrict the pooling of data, but these concerns do not apply to the pooling of models. Pooling different linear models is nothing new, but combining nonlinear models shows promise for providing doubly robust causal estimates with lower variance, even when the source models have different covariates as inputs. As more research on these and other areas continues, it is likely we will see the associated advances make their way into some of the key ideas we have captured here.

Acknowledgments

The contents of these 10 rules benefited from the support and feedback of a broad community, to include Evidence Accelerator, Cerner, and Observational Health Data Sciences and Informatics (OHDSI). We thank Elizabeth Ogburn, Henrik T. Sorensen, Todd Wagner, Jason LaBonte, Marc Succhard, and Sascha Dublin for many helpful discussions. We thank Julia Kuhl for producing the figures.

Author Contributions

MP, AK, AS, BC, SA, ES, and JV contributed to the initial formulation of the 10 rules framework. MP was the lead writer for this manuscript. All authors contributed key ideas related to their respective areas of expertise, reviewed multiple drafts of the manuscript, and approved the final manuscript.

Research was partially supported by funding from Microsoft Research and Fast Grants, part of the Emergent Ventures Program at The Mercatus Center at George Mason University. AK was supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE–1656518. Any opinion, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. JB was supported by NIH K23HL128909 and FastGrants. MK was supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health under Award no. T32AR048522. SA was supported by the Golub Capital Social Impact Lab, Schmidt Futures, the Sloan Foundation, Office of Naval Research Grant N00014-17-1-2131, the Mercatus Center, and Microsoft Research. ES was supported by the National Institute of Mental Health under Grant R01MH115487.

Conflict of Interest

MK received personal fees from Bristol-Myers Squibb and Celltrion, unrelated to this manuscript. SM is an employee and holds stocks in Health Catalyst, Inc. VM and JL are employees and hold equity in Datavant, Inc. CB is a consultant for Depuy-Synthes and Bionaut Labs. SA is an advisor and holds an equity stake in two private companies, Prealize (Palo Alto, California, United States) and Consulta (Brazil). Prealize is a health care analytics company, and Consulta operates a chain of low-cost medical clinics in Brazil.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

• AlGhatrif M., Strait J. B., Morrell C. H., Canepa M., Wright J., Elango P., et al. (2013). Longitudinal Trajectories of Arterial Stiffness and the Role of Blood Pressure: the Baltimore Longitudinal Study of Aging . Hypertension 62 ( 5 ), 934–941. 10.1161/HYPERTENSIONAHA.113.01445 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Andrade S. E., Kahler K. H., Frech F., Chan K. A. (2006). Methods for Evaluation of Medication Adherence and Persistence Using Automated Databases . Pharmacoepidem. Drug Safe. 15 ( 8 ), 565–574. 10.1002/pds.1230 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Athey S., Imbens G. W., Metzger J., Munro E. (2021). Using Wasserstein Generative Adversarial Networks for the Design of Monte Carlo Simulations . Mar 20 [cited 2021 Jun 14]. J Econom [Internet] Available at: https://www.sciencedirect.com/science/article/pii/S0304407621000440 . 10.1016/j.jeconom.2020.09.013 [ CrossRef ] [ Google Scholar ]
• Athey S., Imbens G. W., Wager S. (2018). Approximate Residual Balancing: De-biased Inference of Average Treatment Effects in High Dimensions . ArXiv160407125 Econ Math Stat [Internet]. Jan 31 [cited 2020 Aug 30]. Available at: http://arxiv.org/abs/1604.07125 . 10.3386/t0280 [ CrossRef ] [ Google Scholar ]
• Athey S., Tibshirani J., Wager S. (2019). Generalized Random Forests . Ann. Stat. 47 ( 2 ), 1148–1178. 10.1214/18-aos1709 [ CrossRef ] [ Google Scholar ]
• Athey S., Imbens G., Pham T., Wager S. (2017). Estimating Average Treatment Effects: Supplementary Analyses and Remaining Challenges . Am. Econ. Rev. 107 ( 5 ), 278–281. 10.1257/aer.p20171042 [ CrossRef ] [ Google Scholar ]
• Athey S., Imbens G. (2016). Recursive Partitioning for Heterogeneous Causal Effects . Proc. Natl. Acad. Sci. USA 113 ( 27 ), 7353–7360. 10.1073/pnas.1510489113 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Austin P. C. (2009). Balance Diagnostics for Comparing the Distribution of Baseline Covariates between Treatment Groups in Propensity-Score Matched Samples . Statist. Med. 28 ( 25 ), 3083–3107. 10.1002/sim.3697 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Austin P. C., Stuart E. A. (2015). Moving towards Best Practice when Using Inverse Probability of Treatment Weighting (IPTW) Using the Propensity Score to Estimate Causal Treatment Effects in Observational Studies . Statist. Med. 34 ( 28 ), 3661–3679. 10.1002/sim.6607 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bang H., Robins J. M. (2005). Doubly Robust Estimation in Missing Data and Causal Inference Models . Biometrics 61 ( 4 ), 962–973. 10.1111/j.1541-0420.2005.00377.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bareinboim E., Pearl J. (2016). Causal Inference and the Data-Fusion Problem . Proc. Natl. Acad. Sci. USA 113 ( 27 ), 7345–7352. 10.1073/pnas.1510507113 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Beaulieu-Jones Brett K., Wu Z. S., Williams C., Lee R., Bhavnani Sanjeev P., James Brian B., et al. (2019). Privacy-Preserving Generative Deep Neural Networks Support Clinical Data Sharing . Circ. Cardiovasc. Qual. Outcomes 12 ( 7 ), e005122. 10.1161/circoutcomes.118.005122 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Begg C., Cho M., Eastwood S., Horton R., Moher D., Olkin I., et al. (1996). Improving the Quality of Reporting of Randomized Controlled Trials . JAMA 276 ( 8 ), 637–639. 10.1001/jama.1996.03540080059030 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Belloni A., Chernozhukov V., Hansen C. (2014). High-Dimensional Methods and Inference on Structural and Treatment Effects . J. Econ. Perspect. 28 ( 2 ), 29–50. 10.1257/jep.28.2.29 [ CrossRef ] [ Google Scholar ]
• Benchimol E. I., Smeeth L., Guttmann A., Harron K., Moher D., Petersen I., et al. (2015). The REporting of Studies Conducted Using Observational Routinely-Collected Health Data (RECORD) Statement . PLOS Med. 12 ( 10 ), e1001885. 10.1371/journal.pmed.1001885 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Byrd J. B., Greene A. C., Prasad D. V., Jiang X., Greene C. S. (2020). Responsible, Practical Genomic Data Sharing that Accelerates Research . Nat. Rev. Genet. , 1–15. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Chernozhukov V., Chetverikov D., Demirer M., Duflo E., Hansen C., Newey W., et al. (2018). Double/debiased Machine Learning for Treatment and Structural Parameters . Econom. J. 21 ( 1 ), C1–C68. 10.1111/ectj.12097 [ CrossRef ] [ Google Scholar ]
• Chronic Conditions Data Warehouse (2020). Chronic Conditions Data Warehouse[Internet] . [cited 2020 Jul 29]. Available at: https://www2.ccwdata.org/web/guest/condition-categories . [ Google Scholar ]
• COVID-19 Provisional Counts (2020). Weekly Updates by Select Demographic and Geographic Characteristics [Internet] . [cited 2020 Aug 10]. Available at: https://www.cdc.gov/nchs/nvss/vsrr/covid_weekly/index.htm . [ Google Scholar ]
• D’Hoore W., Sicotte C., Tilquin C. (1993). Risk Adjustment in Outcome Assessment: the Charlson Comorbidity Index . Methods Inf. Med. 32 ( 5 ), 382–387. [ PubMed ] [ Google Scholar ]
• Dickerman B. A., García-Albéniz X., Logan R. W., Denaxas S., Hernán M. A. (2019). Avoidable Flaws in Observational Analyses: an Application to Statins and Cancer . Nat. Med. 25 ( 10 ), 1601–1606. 10.1038/s41591-019-0597-x [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Elixhauser A., Steiner C., Harris D. R., Coffey R. M. (1998). Comorbidity Measures for Use with Administrative Data . Med. Care 36 ( 1 ), 8–27. 10.1097/00005650-199801000-00004 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Fafchamps M., Labonne J. (2017). Using Split Samples to Improve Inference on Causal Effects . Polit. Anal. 25 ( 4 ), 465–482. 10.1017/pan.2017.22 [ CrossRef ] [ Google Scholar ]
• Gelman A., Loken E. (2014). The Statistical Crisis in Science . Amer. Scientist 102 ( 6 ), 460–466. 10.1511/2014.111.460 [ CrossRef ] [ Google Scholar ]
• Glynn R. J. (2017). Editorial: Use of Propensity Scores to Design Observational Comparative Effectiveness Studies . JNCI J. Natl. Cancer Inst. [Internet]. [cited 2020 Jul 29], 109 : 8 , djw345. Available at: https://academic.oup.com/jnci/article/109/8/djw345/3078531 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Greenland S., Pearl J., Robins J. M. (1999). Causal Diagrams for Epidemiologic Research . Epidemiology 10 ( 1 ), 37–48. 10.1097/00001648-199901000-00008 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gupta S. (2011). Intention-to-treat Concept: A Review . Perspect. Clin. Res. 2 ( 3 ), 109–112. 10.4103/2229-3485.83221 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hernán M. A., Robins J. M. (2020). Causal Inference: What if . Boca Raton: Chapman & Hall/CRC. [ Google Scholar ]
• Hernán M. A., Hernández-Díaz S., Robins J. M. (2004). A Structural Approach to Selection Bias . Epidemiology 15 ( 5 ), 615–625. 10.1097/01.ede.0000135174.63482.43 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hernán M. A., Robins J. M. (2006). Instruments for Causal Inference . Epidemiol. Camb Mass. 17 ( 4 ), 360–372. 10.1097/01.ede.0000222409.00878.37 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hernán M. A., Robins J. M. (2016). Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available . Am. J. Epidemiol. 183 ( 8 ), 758–764. 10.1093/aje/kwv254 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hernandez A. V., Roman Y. M., Pasupuleti V., Barboza J. J., White C. M. (2020). Hydroxychloroquine or Chloroquine for Treatment or Prophylaxis of COVID-19: A Living Systematic Review , [cited 2020 Aug 1]. Ann. Intern. Med. [Internet] 173 ( 4 ), 287–296. Available at: https://www.acpjournals.org/doi/full/10.7326/M20-2496 . 10.7326/M20-2496 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ho D. E., Imai K., King G., Stuart E. A. (2011). MatchIt: Nonparametric Preprocessing for Parametric Causal Inference . J. Stat. Softw. 42 ( 8 ), 1–28. 10.18637/jss.v042.i08 [ CrossRef ] [ Google Scholar ]
• Holdcroft A. (2007). Gender Bias in Research: How Does it Affect Evidence Based Medicine? J. R. Soc. Med. 100 ( 1 ), 2–3. 10.1177/014107680710000102 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Imai K., King G., Stuart E. A. (2008). Misunderstandings between Experimentalists and Observationalists about Causal Inference . J. R. Stat. Soc A 171 ( 2 ), 481–502. 10.1111/j.1467-985x.2007.00527.x [ CrossRef ] [ Google Scholar ]
• Imbens G. W., Rubin D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction . Cambridge: Cambridge University Press. 10.1017/cbo9781139025751 [ CrossRef ] [ Google Scholar ]
• Jackson J. W., Schmid I., Stuart E. A. (2017). Propensity Scores in Pharmacoepidemiology: Beyond the Horizon . Curr. Epidemiol. Rep. 4 ( 4 ), 271–280. 10.1007/s40471-017-0131-y [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jackson L. A., Jackson M. L., Nelson J. C., Neuzil K. M., Weiss N. S. (2006). Evidence of Bias in Estimates of Influenza Vaccine Effectiveness in Seniors . Int. J. Epidemiol. 35 ( 2 ), 337–344. 10.1093/ije/dyi274 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jager K. J., Zoccali C., MacLeod A., Dekker F. W. (2008). Confounding: What it Is and How to deal with it . Kidney Int. 73 ( 3 ), 256–260. 10.1038/sj.ki.5002650 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• John L. K., Loewenstein G., Prelec D. (2012). Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling . Psychol. Sci. 23 ( 5 ), 524–532. 10.1177/0956797611430953 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kerr N. L. (1998). HARKing: Hypothesizing after the Results Are Known . Pers Soc. Psychol. Rev. 2 ( 3 ), 196–217. 10.1207/s15327957pspr0203_4 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Koenecke A., Powell M., Xiong R., Shen Z., Fischer N., Huq S., et al. (2021). Alpha-1 Adrenergic Receptor Antagonists to Prevent Hyperinflammation and Death from Lower Respiratory Tract Infection . eLife 10 , e61700. 10.7554/elife.61700 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Koenecke A., Varian H. (2020). Synthetic Data Generation for Economists . ArXiv201101374 Cs Econ Q-Fin [Internet]. Cornell University; Available at: http://arxiv.org/abs/2011.01374 . [cited 2020 Nov 16]. [ Google Scholar ]
• Konig M. F., Powell M., Staedtke V., Bai R.-Y., Thomas D. L., Fischer N., et al. (2020). Preventing Cytokine Storm Syndrome in COVID-19 Using α-1 Adrenergic Receptor Antagonists . J. Clin. Invest. 130 ( 7 ), 3345–3347. 10.1172/jci139642 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Langan S. M., Schmidt S. A., Wing K., Ehrenstein V., Nicholls S. G., Filion K. B., et al. (2018). The Reporting of Studies Conducted Using Observational Routinely Collected Health Data Statement for Pharmacoepidemiology (RECORD-PE) , [cited 2020 Aug 10]. BMJ [Internet] 363 , k3532. Available at: https://www.bmj.com/content/363/bmj.k3532 . 10.1136/bmj.k3532 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lee B. K., Lessler J., Stuart E. A. (2011). Weight Trimming and Propensity Score Weighting . PLoS One 6 ( 3 ), e18174. 10.1371/journal.pone.0018174 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Li S., Jun T., Wang Z., Kao Y-H., Schadt E., Konig M. F., et al. (2021). COVID-19 Outcomes Among Hospitalized Men with or without Exposure to Alpha-1-Adrenergic Receptor Blocking Agents . medRxiv , 2021. 10.1101/2021.04.08.21255148 [ CrossRef ] [ Google Scholar ]
• Lipsitch M., Tchetgen Tchetgen E., Cohen T. (2010). Negative Controls . Epidemiol. Camb Mass. 21 ( 3 ), 383–388. 10.1097/ede.0b013e3181d61eeb [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• London A. J., Kimmelman J. (2020). Against Pandemic Research Exceptionalism . Science 368 ( 6490 ), 476–477. 10.1126/science.abc1731 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mantel N., Haenszel W. (1959). Statistical Aspects of the Analysis of Data from Retrospective Studies of Disease . J. Natl. Cancer Inst. 22 ( 4 ), 719–748. [ PubMed ] [ Google Scholar ]
• Mehta P., McAuley D. F., Brown M., Sanchez E., Tattersall R. S., Manson J. J. (2020). COVID-19: Consider Cytokine Storm Syndromes and Immunosuppression . The Lancet 395 ( 10229 ), 1033–1034. 10.1016/s0140-6736(20)30628-0 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• National COVID Cohort Collaborative (N3C) (2020). National COVID Cohort Collaborative (N3C) [Internet] . [cited 2020 Jul 29]. National Center for Advancing Translational Sciences; Available at: https://github.com/National-COVID-Cohort-Collaborative/Phenotype_Data_Acquisition/wiki/Latest-Phenotype#inclusion . [ Google Scholar ]
• Berman N. G., Parker R. A. (2016). “ Blinding in Observational Studies ,” in Planning Clinical Research [Internet] (Cambridge: Cambridge University Press; ), 334–340. [cited 2020 Aug 14]Available at: https://www.cambridge.org/core/books/planning-clinical-research/blinding-in-observational-studies/24DAB9C3061C096ED088EF538C65E4A7 . [ Google Scholar ]
• Office of the Commissioner (2020). Real-World Evidence [Internet] . [cited 2020 Jul 31]. U.S. Food & Drug Administration. FDA; Available at: https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence . [ Google Scholar ]
• Partin A. W., Oesterling J. E., Epstein J. I., Horton R., Walsh P. C. (1991). Influence of Age and Endocrine Factors on the Volume of Benign Prostatic Hyperplasia . J. Urol. 145 ( 2 ), 405–409. 10.1016/s0022-5347(17)38353-2 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Pazzagli L., Linder M., Zhang M., Vago E., Stang P., Myers D., et al. (2018). Methods for Time-Varying Exposure Related Problems in Pharmacoepidemiology: An Overview . Pharmacoepidemiol. Drug Saf. 27 ( 2 ), 148–160. 10.1002/pds.4372 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Pearl J. (2009). Causality . Cambridge: Cambridge University Press. 10.1017/cbo9780511803161 [ CrossRef ] [ Google Scholar ]
• Piller C., Travis J. (2020). Authors, Elite Journals under Fire after Major Retractions . Science 368 ( 6496 ), 1167–1168. 10.1126/science.368.6496.1167 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ridgeway G., McCaffrey D., Morral A., Griffin B. A., Burgette L., Cefalu M. (2020). Toolkit for Weighting and Analysis of Nonequivalent Groups: A Tutorial for the Twang Package . Santa Monica, CA: RAND Corporation; Available at: https://CRAN.R-project.org/package=twang . [ Google Scholar ]
• Rosenbaum P. R. (2010). Design Sensitivity and Efficiency in Observational Studies . J. Am. Stat. Assoc. 105 ( 490 ), 692–702. 10.1198/jasa.2010.tm09570 [ CrossRef ] [ Google Scholar ]
• Rosenbaum P. R., Rubin D. B. (1983). Assessing Sensitivity to an Unobserved Binary Covariate in an Observational Study with Binary Outcome . J. R. Stat. Soc. Ser. B (Methodological) 45 ( 2 ), 212–218. 10.1111/j.2517-6161.1983.tb01242.x [ CrossRef ] [ Google Scholar ]
• Rubin D. B. (2001). Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation . Health Serv. Outcomes Res. Methodol. 2 ( 3 ), 169–188. [ Google Scholar ]
• Rubin D. B. (2004). Teaching Statistical Inference for Causal Effects in Experiments and Observational Studies . J. Educ. Behav. Stat. 29 ( 3 ), 343–367. 10.3102/10769986029003343 [ CrossRef ] [ Google Scholar ]
• Rudolph K. E., Díaz I., Rosenblum M., Stuart E. A. (2014). Estimating Population Treatment Effects from a Survey Subsample . Am. J. Epidemiol. 180 ( 7 ), 737–748. 10.1093/aje/kwu197 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schuemie M. J., Ryan P. B., Hripcsak G., Madigan D., Suchard M. A. (2018). Improving Reproducibility by Using High-Throughput Observational Studies with Empirical Calibration . Phil. Trans. R. Soc. A. 376 ( 2128 ), 20170356. 10.1098/rsta.2017.0356 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Simonsohn U., Simmons J. P., Nelson L. D. (2019). Specification Curve: Descriptive and Inferential Statistics on All Reasonable Specifications . [cited 2020 Jul 31]. Report No.: ID 2694998. Rochester, NY: Social Science Research Network; Available at: https://papers.ssrn.com/abstract=2694998 .[Internet] [ Google Scholar ]
• Staedtke V., Bai R.-Y., Kim K., Darvas M., Davila M. L., Riggins G. J., et al. (2018). Disruption of a Self-Amplifying Catecholamine Loop Reduces Cytokine Release Syndrome . Nature 564 ( 7735 ), 273–277. 10.1038/s41586-018-0774-y [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Stone R. (1993). The Assumptions on Which Causal Inferences Rest . J. R. Stat. Soc. Ser. B (Methodological) 55 ( 2 ), 455–466. 10.1111/j.2517-6161.1993.tb01915.x [ CrossRef ] [ Google Scholar ]
• Stuart E. A. (2010). Matching Methods for Causal Inference: A Review and a Look Forward . Stat. Sci. Rev. J. Inst. Math. Stat. 25 ( 1 ), 1–21. 10.1214/09-sts313 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Thomsen R. W., Christiansen C. F., Heide-Jørgensen U., Vogelstein J. T., Vogelstein B., Bettegowda C., et al. (2021). Association of α1-Blocker Receipt with 30-Day Mortality and Risk of Intensive Care Unit Admission Among Adults Hospitalized with Influenza or Pneumonia in Denmark . JAMA Netw. Open 4 ( 2 ), e2037053. 10.1001/jamanetworkopen.2020.37053 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Vandenbroucke J. P. (2004). When Are Observational Studies as Credible as Randomised Trials? The Lancet 363 ( 9422 ), 1728–1731. 10.1016/s0140-6736(04)16261-2 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• VanderWeele T. J., Shpitser I. (2013). On the Definition of a Confounder . Ann. Stat. 41 ( 1 ), 196–220. 10.1214/12-aos1058 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• VanderWeele T. J., Ding P. (2017). Sensitivity Analysis in Observational Research: Introducing the E-Value . Ann. Intern. Med. 167 ( 4 ), 268–274. 10.7326/m16-2607 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• VanderWeele T. J., Hernán M. A., Robins J. M. (2008). Causal Directed Acyclic Graphs and the Direction of Unmeasured Confounding Bias . Epidemiol. Camb Mass. 19 ( 5 ), 720–728. 10.1097/ede.0b013e3181810e29 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wang Z., Zheutlin A. B., Kao Y-H., Ayers K. L., Gross S. J., Kovatch P., et al. (2020). Analysis of Hospitalized COVID-19 Patients in the Mount Sinai Health System Using Electronic Medical Records (EMR) Reveals Important Prognostic Factors for Improved Clinical Outcomes . medRxiv , 2020. 10.1101/2020.04.28.20075788 [ CrossRef ] [ Google Scholar ]
• Yoshida K., Solomon D. H., Kim S. C. (2015). Active-comparator Design and New-User Design in Observational Studies . Nat. Rev. Rheumatol. 11 ( 7 ), 437–441. 10.1038/nrrheum.2015.30 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

An analysis of ocular biometrics: a comprehensive retrospective study in a large cohort of pediatric cataract patients.

1. Introduction

2. materials and methods, 2.1. examinations, 2.2. statistical analysis, 4. discussion, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Pi, L.H.; Chen, L.; Liu, Q.; Ke, N.; Fang, J.; Zhang, S.; Xiao, J.; Ye, W.J.; Xiong, Y.; Shi, H.; et al. Prevalence of Eye Diseases and Causes of Visual Impairment in School-Aged Children in Western China. J. Epidemiol. 2012 , 22 , 37–44. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Oshika, T.; Nishina, S.; Unoki, N.; Miyagi, M.; Nomura, K.; Mori, T.; Matsuki, N.; Endo, T.; Kurosaka, D.; Negishi, K.; et al. Ten-Year Outcomes of Congenital Cataract Surgery Performed within the First 6 Months of Life. J. Cataract. Refract. Surg. 2024 , 50 , 707–712. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tarkkanen, A.; Uusitalo, R.; Mianowicz, J. Ultrasonographic Biometry in Congenital Glaucoma. Acta Ophthalmol. 1983 , 61 , 618–623. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Oh, R.; Oh, J.Y.; Choi, H.J.; Kim, M.K.; Yoon, C.H. Comparison of Ocular Biometric Measurements in Patients with Cataract Using Three Swept-Source Optical Coherence Tomography Devices. BMC Ophthalmol. 2021 , 21 , 62. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Dong, J.; Zhang, Y.; Zhang, H.; Jia, Z.; Zhang, S.; Wang, X. Comparison of Axial Length, Anterior Chamber Depth and Intraocular Lens Power between IOLMaster and Ultrasound in Normal, Long and Short Eyes. PLoS ONE 2018 , 13 , e0194273. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Orazbekov, L.; Smagulova, S.; Ruslanuly, K. Long-Term Results of Congenital Cataract Surgery with Primary Intraocular Lens Implantation: A Case-Control Study of Three Age Groups. J. Curr. Ophthalmol. 2022 , 34 , 290. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Chen, J.; Tang, Y.; Jiang, Y.; Lu, Y. Preoperative Characteristics of Ocular Biometry in Children with Unilateral Congenital Cataracts. Phenomics 2022 , 2 , 136–144. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Bhattacharjee, H.; Buragohain, S.; Javeri, H.; Deshmukh, S. Optical Biometry and Influence of Media Opacity Due to Cataract on Development of Axial Length in NorthEast Indian Paediatric Patients—A Prospective Study. BMC Ophthalmol. 2021 , 21 , 374. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• He, X.; Sankaridurg, P.; Naduvilath, T.; Wang, J.; Xiong, S.; Weng, R.; Du, L.; Chen, J.; Zou, H.; Xu, X. Normative Data and Percentile Curves for Axial Length and Axial Length/Corneal Curvature in Chinese Children and Adolescents Aged 4–18 Years. Br. J. Ophthalmol. 2023 , 107 , 167–175. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Nilagiri, V.K.; Lee, S.S.Y.; Lingham, G.; Charng, J.; Yazar, S.; Hewitt, A.; Griffiths, L.R.; Sanfilippo, P.G.; Tsai, T.H.; Mackey, D.A. Distribution of Axial Length in Australians of Different Age Groups, Ethnicities, and Refractive Errors. Transl. Vis. Sci. Technol. 2023 , 12 , 14. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Fieß, A.; Nickels, S.; Schulz, A.; Münzel, T.; Wild, P.S.; Beutel, M.E.; Urschitz, M.S.; Lackner, K.J.; Pfeiffer, N.; Schuster, A.K. The Relationship of Ocular Geometry with Refractive Error in Normal and Low Birth Weight Adults. J. Optom. 2021 , 14 , 50–57. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Lin, D.; Chen, J.; Liu, Z.; Wu, X.; Long, E.; Luo, L.; Lin, Z.; Li, X.; Zhang, L.; Chen, H.; et al. Prevalence of Corneal Astigmatism and Anterior Segmental Biometry Characteristics before Surgery in Chinese Congenital Cataract Patients. Sci. Rep. 2016 , 6 , 22092. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Sand, M.K.; Cholidis, S.; Kristianslund, O.; Drolsum, L. Primary Intraocular Lens Implantation in Infants with Unilateral Congenital Cataract Operated before 12 Weeks of Age. Acta Ophthalmol. 2022 , 100 , 526–533. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Semeraro, F.; Forbice, E.; Nascimbeni, G.; Cillino, S.; Bonfiglio, V.M.E.; Filippelli, M.E.; Bartollino, S.; Costagliola, C. Ocular Refraction at Birth and Its Development During the First Year of Life in a Large Cohort of Babies in a Single Center in Northern Italy. Front. Pediatr. 2020 , 7 , 539. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Morgan, I.G.; Rose, K.A.; Ellwein, L.B. Is Emmetropia the Natural Endpoint for Human Refractive Development? An Analysis of Population-Based Data from the Refractive Error Study in Children (RESC). Acta Ophthalmol. 2010 , 88 , 877–884. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Vasavada, A.R.; Vasavada, V.; Shah, S.K.; Praveen, M.R.; Vasavada, V.A.; Trivedi, R.H.; Rawat, F.; Koul, A. Five-Year Postoperative Outcomes of Bilateral Aphakia and Pseudophakia in Children up to 2 Years of Age: A Randomized Clinical Trial. Am. J. Ophthalmol. 2018 , 193 , 33–44. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Solebo, A.L.; Cumberland, P.; Rahi, J.S. 5-Year Outcomes after Primary Intraocular Lens Implantation in Children Aged 2 Years or Younger with Congenital or Infantile Cataract: Findings from the IoLunder2 Prospective Inception Cohort Study. Lancet Child. Adolesc. Health 2018 , 2 , 863–871. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Kaur, S.; Sukhija, J.; Ram, J. Comparison of Posterior Optic Capture of Intraocular Lens without Vitrectomy vs Endocapsular Implantation with Anterior Vitrectomy in Congenital Cataract Surgery: A Randomized Prospective Study. Indian. J. Ophthalmol. 2020 , 68 , 84–88. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Dick, H.B.; Canto, A.P.; Culbertson, W.W.; Schultz, T. Femtosecond Laser-Assisted Technique for Performing Bag-in-the-Lens Intraocular Lens Implantation. J. Cataract. Refract. Surg. 2013 , 39 , 1286–1290. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Fung, S.S.M.; Brookes, J.; Wilkins, M.R.; Adams, G.G.W. Mobile Femtosecond Laser Platform for Pediatric Cataract Surgery. Eur. J. Ophthalmol. 2018 , 28 , 246–250. [ Google Scholar ] [ CrossRef ] [ PubMed ]

Click here to enlarge figure

Share and Cite

Schwarzenbacher, L.; Wassermann, L.; Rezar-Dreindl, S.; Reiter, G.S.; Schmidt-Erfurth, U.; Stifter, E. An Analysis of Ocular Biometrics: A Comprehensive Retrospective Study in a Large Cohort of Pediatric Cataract Patients. J. Clin. Med. 2024 , 13 , 4810. https://doi.org/10.3390/jcm13164810

Schwarzenbacher L, Wassermann L, Rezar-Dreindl S, Reiter GS, Schmidt-Erfurth U, Stifter E. An Analysis of Ocular Biometrics: A Comprehensive Retrospective Study in a Large Cohort of Pediatric Cataract Patients. Journal of Clinical Medicine . 2024; 13(16):4810. https://doi.org/10.3390/jcm13164810

Schwarzenbacher, Luca, Lorenz Wassermann, Sandra Rezar-Dreindl, Gregor S. Reiter, Ursula Schmidt-Erfurth, and Eva Stifter. 2024. "An Analysis of Ocular Biometrics: A Comprehensive Retrospective Study in a Large Cohort of Pediatric Cataract Patients" Journal of Clinical Medicine 13, no. 16: 4810. https://doi.org/10.3390/jcm13164810

Article Metrics

Further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Open access
• Published: 14 August 2024

Association of Helicobacter pylori infection with complications of diabetes: a single-center retrospective study

• Zhuoya Li 1   na1 ,
• Jie Zhang 2   na1 ,
• Yizhou Jiang 1 ,
• Cheng Cui 1 &
• Xiaoyong Wang 1  

BMC Endocrine Disorders volume  24 , Article number:  152 ( 2024 ) Cite this article

Metrics details

Previous studies examined the association of Helicobacter pylori infection ( H. pylori ) with complications of diabetes, but the results have been inconsistent. The aim of this study of patients with type-2 diabetes (T2D) was to determine the association of H. pylori infection with the major complications of diabetes.

This single-center retrospective study examined patients with T2D who received H. pylori testing between January 2016 and December 2021. Logistic regression analyses were used to evaluate the association of H. pylori infection with four major complications of diabetes.

We examined 960 patients with T2D, and 481 of them (50.1%) were positive for H. pylori . H. pylori infection was significantly associated with diabetic nephropathy (odds ratio [OR] = 1.462; 95% confidence interval [CI]: 1.006,2.126; P =  0.046). In addition, the co-occurrence of H. pylori positivity with hypertension (OR = 4.451; 95% CI: 2.351,8.427; P  < 0.001), with glycated hemoglobin A1c (HbA1c) of at least 8% (OR = 2.925; 95% CI: 1.544,5.541; P =  0.001), and with diabetes duration of at least 9 years (OR = 3.305; 95% CI:1.823,5.993; P  < 0.001) further increased the risk of diabetic nephropathy. There was no evidence of an association of H. pylori infection with retinopathy, neuropathy, or peripheral vascular disease.

Conclusions

Our study of T2D patients indicated that those with H. pylori infections had an increased risk of nephropathy, and this risk was greater in patients who also had hypertension, an HbA1c level of 8% or more, and diabetes duration of 9 years or more.

Peer Review reports

Introduction

Helicobacter pylori is a Gram-negative bacterium that infects the gastric mucosa of the upper gastrointestinal tract, and is present in approximately half of all people worldwide. Although most infected individuals are asymptomatic, infection can lead to chronic gastritis, peptic ulcers, gastric adenocarcinomas, and mucosa-associated lymphoid tissue lymphoma [ 1 ]. A recent literature review concluded there were positive correlations of H. pylori infection with extra-gastroduodenal manifestations, such as diabetes, neurological diseases, hematological diseases, cardiovascular diseases, and autoimmune diseases [ 2 ].

Type-2 diabetes (T2D) is a major public health problem worldwide. In 2017, an estimated 451 million people between the ages of 18 and 99 years had diabetes, and this number is expected to increase to 693 million by 2045 [ 3 ]. The uncontrolled hyperglycemia in patients with diabetes can lead to serious microvascular and macrovascular complications, and these complications adversely affect the duration and quality of life [ 4 , 5 ] and are a significant economic burden for healthcare systems. Strict glycemic control can prevent or delay these complications, and thereby improve long-term health and reduce treatment-associated costs. A meta-analysis of 13 studies concluded that diabetes was significantly associated with H. pylori infection [ 6 ]. Another meta-analysis of 41 case-control studies identified H. pylori as a risk factor for diabetes, particularly T2D [ 7 ]. H. pylori infection is also associated with higher levels of fasting plasma glucose (FPG) and glycated hemoglobin A1c (HbA1c) in patients with diabetes [ 8 , 9 ]. The importance of these two indicators was emphasized in a longitudinal observational cohort study of Korean patients with newly diagnosed T2D, which found that early achievement of the target level of HbA1c was associated with long-term durable glycemic control and a decreased risk of complications [ 10 ]. A systematic review and meta-analysis from 2021 concluded that H. pylori eradication improved glycemic control in patients with T2D [ 11 ]. Other studies demonstrated that H. pylori eradication led to decreased levels of HbA1c and improved glycemic control [ 12 , 13 ]. Therefore, many studies support the presence of an association of H. pylori infection with diabetes and hyperglycemia.

Although several studies have examined the relationship of H. pylori infection with diabetes complications (nephropathy, retinopathy, neuropathy, and peripheral vascular disease [PVD]), their conclusions have been inconsistent [ 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. Demir et al. showed that diabetes patients with H. pylori infection had a higher incidence of neuropathy, but there was no association between retinopathy, nephropathy, and H. pylori infection [ 14 ]. A study in Turkey showed that H. pylori positivity was significantly associated with the presence of nephropathy and neuropathy [ 15 ]. Although some studies reported that the prevalences of nephropathy, neuropathy, retinopathy and PVD complications were significantly higher in diabetes patients who were H. pylori -positive [ 16 , 17 , 18 , 19 , 20 ], other studies reported contrary results [ 21 , 22 , 23 ]. These discordant results may be due to differences in study design, patient populations, sample size, or other factors. Therefore, clinical investigations with large samples are needed to investigate this topic. The aim of the present study was to investigate the association of H. pylori infection with complications of diabetes, especially diabetic nephropathy, retinopathy, neuropathy, PVD.

Patients and methods

Study population.

The electronic medical records of patients with T2D at Changzhou No. 2 People’s Hospital, Affiliated with Nanjing Medical University between January 2016 and December 2021 were retrospectively examined. Diabetes was defined according to the 1999 criteria of the World Health Organization [ 24 ] as the presence of diabetic symptoms (such as polydipsia, polyuria, polyphagia, and unexplained weight loss) and a random plasma glucose of at least 11.1 mmol/L, or a FPG of at least 7.0 mmol/L, or a plasma glucose of at least 11.1 mmol/L at 2 h after a 75 g dose of oral glucose. H. pylori infection was diagnosed by a positive result from the 13 C-urea breath test ( 13 C-UBT), or the rapid urease test (RUT), or serological testing.

The inclusion criteria were diagnosis of T2D, age of at least 18 years, receipt of an H. pylori infection test, receipt of screening for diabetes complications (diabetic nephropathy, retinopathy, neuropathy, PVD), and complete data on demographics and serum biochemical indexes.

The exclusion criteria were type 1 diabetes, a history of H. pylori eradication therapy, any malignancy, chronic renal failure requiring dialysis treatment, treatment with a PPI, bismuth, or an antibiotic in the preceding 1 month, absence of an H. pylori infection test, and missing information regarding complications of diabetes (diabetic nephropathy, retinopathy, neuropathy, PVD).

Data collection

The baseline demographic data and serum biochemical indexes included age, gender, body mass index (BMI), smoking and alcohol habits, history of hypertension, known duration of T2D, HbA1c, total cholesterol (TC), high-density lipoprotein cholesterol (HDL-C), low-density lipoprotein cholesterol (LDL-C), triglycerides (TG), calcium, endoscopic findings and comorbidities. BMI (kg/m 2 ) was calculated from body weight and height.

Assessment of diabetic complications

According to the American Diabetes Association [ 25 ], the diagnosis of diabetic nephropathy is based on the presence of a low estimated glomerular filtration rate (eGFR < 60 mL/min/1.73 m 2 ) and/or increased level of urinary albumin (≥ 30 mg/g creatinine) that persisted more than 3 months. Diabetic retinopathy is a common microvascular complication that leads to vision loss, and was determined by an ophthalmologist using a standard fundus examination [ 26 ]. Diabetic neuropathies are a heterogeneous group of disorders with diverse clinical manifestations, and diagnosis is based on the exclusion of similar disorders [ 26 ]. The diagnosis of neuropathy was based on the results of electromyography and nerve fiber conduction examinations, as previously described [ 27 ]. PVD was diagnosed from clinical findings of a history of intermittent claudication pain, absence of pulse in a physical examinations, or both of these, with confirmation from a color-Doppler ultrasound examination [ 28 ].

Statistical analysis

Descriptive data are presented as mean ± standard deviation (SD), median (interquartile range [IQR]), or number (percentage). The independent samples t -test was used to compare continuous data that had normal distributions, the Mann-Whitney U test was used to compare continuous data that had non-normal distributions, and the χ 2 test was used to compare categorical data. Receiver operating characteristic (ROC) curves were constructed, and the Youden index was used to identify the optimal cutoff values for HbA1c, age, and known duration of diabetes for prediction of diabetic nephropathy and PVD. To assess the effect of different individual risk factors on T2D complications and the joint effects of H. pylori infection and other risk factors on these complications, binary logistic regression analysis was used to estimate adjusted odds ratios (aORs) and 95% confidence intervals (CIs). A P value below 0.05 was considered to be significant, and all statistical analyses were performed using SPSS Statistics version 26.0 (IBM Corporation, Armonk, NY, USA).

We examined the records of 960 T2D patients who met all the eligibility criteria (Table  1 ). There were 481 H. pylori- positive patients and 479 H. pylori -negative patients. For the H. pylori -negative patients, the prevalence of atrophic gastritis was 13.6% and the prevalence of peptic ulcer was 5.8%. The other endoscopic findings of H. pylori -negative patients included nodular gastritis, gastric erosion, and gastric polyp. The H. pylori -positive group had a higher prevalence of diabetic nephropathy ( P  = 0.042), but the two groups had no statistically significant differences in retinopathy, neuropathy, or PVD.

We performed univariate and multivariate logistic regression to identify the risk factors for diabetic nephropathy (Table  2 ). The multivariate analysis showed that the significant and independent risk factors for diabetic nephropathy were H. pylori infection (aOR = 1.462, 95% CI: 1.006–2.126, P  = 0.046), hypertension (aOR = 2.802, 95% CI: 1.829–4.293, P  < 0.001), long duration of diabetes (aOR = 1.057, 95% CI: 1.031–1.084, P  < 0.001), high level of HbA1c (aOR = 1.161, 95% CI: 1.065–1.267, P  < 0.001), and high level of TG (aOR = 1.114, 95% CI, 1.044–1.188, P  = 0.001).

ROC analysis of the continuous variables from the multivariable analysis for the prediction of diabetic nephropathy indicated that the area under the curve (AUC) was 0.595 for HbA1c and 0.613 for diabetes duration (Fig.  1 ). Based on the Youden index, the cutoff value was 8% for HbA1c and 9 years for diabetes duration. The other 3 significant factors from the multivariable analysis were binary variables, including the TG level (< 1.7 mmol/L vs. ≥1.7 mmol/L), which was classified according to the American Association of Clinical Endocrinology Clinical Practice Guideline [ 29 ].

Receiver operating characteristic curves for prediction of diabetic nephropathy

We evaluated the joint effects of H. pylori infection with other factors on the risk of diabetic nephropathy (Table  3 ). The results demonstrated that the co-occurrence of H. pylori positivity with hypertension (aOR = 4.451, 95% CI: 2.351–8.427, P  < 0.001), with an HbA1c level of at least 8% (aOR = 2.925, 95% CI: 1.544–5.541, P =  0.001), and with diabetes duration of at least 9 years (aOR = 3.305, 95% CI:1.823–5.993, P  < 0.001) were significantly associated with diabetic nephropathy.

We then performed univariate and multivariate logistic regression analyses of the risk factors for PVD (Table  4 ). The results of the multivariate analysis showed that H. pylori infection was not a significant risk factor for PVD (aOR = 0.955, 95% CI: 0.718–1.270, P  = 0.752). However, greater age (aOR = 1.063, 95% CI: 1.046–1.080, P  < 0.001 ) , smoking (aOR = 2.278, 95% CI: 1.584–3.277, P  < 0.001), hypertension (aOR = 1.833, 95% CI: 1.372–2.449, P  < 0.001), and long duration of diabetes (OR = 1.034, 95% CI: 1.011–1.057, P  = 0.003) were significantly and independently associated with PVD.

ROC analysis of significant continuous variables from the multivariable analysis indicated that the AUC value was 0.679 for age and 0.606 for diabetes duration for prediction of PVD (Fig.  2 ). Based on the Youden index, the cutoff values were 56-years-old for age and 8 years for diabetes duration.

Receiver operating characteristic curves for prediction of peripheral vascular disease

As above, we also evaluated the joint effects of H. pylori infection with other factors on the risk of PVD (Table  5 ). The results showed that the co-occurrence of H. pylori positivity with age of at least 56 years (aOR = 2.771, 95% CI: 1.807–4.250, P  < 0.001), history of smoking (aOR = 2.344, 95% CI: 1.464–3.752, P  < 0.001) were significantly associated with PVD.

The major result of this retrospective study is that infection by H. pylori was an independent risk factor for diabetic nephropathy. We also demonstrated that the co-occurrence of H. pylori infection with traditional risk factors (hypertension, long duration of diabetes duration, high level of HbA1c) further increased the risk of diabetic nephropathy. Although our results showed that H. pylori infection was not associated with retinopathy, neuropathy, or PVD, we found that the co-occurrence of H. pylori infection with several traditional risk factors (age, smoking) increased the risk of PVD. In addition, although our control diabetic group was H. pylori -negative, these individuals had many gastric alterations, such as atrophic gastritis and peptic ulcer, suggesting that T2D patients are more vulnerable to gastric mucosal injuries.

It is uncertain how H. pylori infection in the gut affects the pathogenic processes that are responsible for diabetic nephropathy, although there are several possible mechanisms. First, several studies suggested that inflammatory responses that are secondary to infection could lead to systemic inflammation, and systemic inflammation is an established risk factor for diabetic nephropathy [ 30 ]. In support of this interpretation, patients with diabetes are more vulnerable to H. pylori infections, and several studies reported that T2D patients with H. pylori infections were more likely to have elevated levels of multiple inflammatory cytokines, including C-reactive protein, tumor necrosis factor-α (TNF-α), interleukin-1 (IL-1), interleukin-6 (IL-6), and interleukin-8 (IL-8) [ 15 , 16 , 21 ]. There is also a close relationship of T2D with dysfunctional endothelial cells, increased insulin resistance, disrupted lipid metabolism, and proteinuria [ 31 , 32 , 33 , 34 ].

Second, chronic H. pylori infection can lead to atrophic gastritis, which reduces the absorption of folate and vitamin-B 12 . The conversion of homocysteine (HCY) into methionine requires these two co-factors, and their depletion leads to an increased level of HCY. An elevated level of HCY can contribute to vascular endothelial damage, in that it increases atherosclerosis and thrombogenesis and it inhibits the secretion of nitric oxide (NO) by endothelial cells. These responses can cause platelet aggregation and vasoconstriction, and increase the risk for arteriosclerosis and hypertension [ 35 , 36 ], which can lead to diabetic nephropathy.

Our results also demonstrated that H. pylori infection, hypertension, diabetes duration, HbA1c level, and TG level were independent risk factors for diabetic nephropathy. Moreover, we showed that the co-occurrence of H. pylori positivity with hypertension, with an HbA1c level of 8% or more, and with diabetes duration of 9 or more years further increased the risk of diabetic nephropathy. These additive or synergistic effects are biologically plausible. In particular, nuclear factor-kappa B (NF-κB) is a transcription factor that regulates the level of many chemokines, cell adhesion proteins, inflammatory cytokines, and other molecules that function in the pathogenesis of diabetic nephropathy [ 30 ]. In addition, the NF-κB-mediated stimulation of the expression of pro-inflammatory genes and their signaling pathways has a major effect in promoting the progression of hypertension to diabetic nephropathy [ 37 ], and H. pylori infection can activate NF-κB signaling [ 38 ]. Taken together with our results, we suggest that the NF-κB-mediated secretion of inflammatory cytokines may be responsible for kidney damage in patients who have concomitant H. pylori infection and hypertension. This alternative interpretation can also explain why T2D patients with hypertension and H. pylori infection have a greater risk of diabetic nephropathy.

HbA1c is the best single biomarker of long-term glycemic control, and higher levels reflect long-term hyperglycemia [ 39 ]. Hyperglycemia is a well-established risk factor for diabetic nephropathy [ 40 ], and several studies reported that H. pylori infection was associated with increased levels of plasma glucose and HbA1c in patients with T2D [ 8 , 9 , 41 ]. These previous studies suggest that the combination of an elevated level of HbA1c and H. pylori infection is a biologically plausible explanation for the increased risk of diabetic nephropathy.

A prolonged duration of diabetes is another significant risk factor for diabetic nephropathy [ 40 ], and there is evidence that diabetes duration is positively associated with H. pylori infection [ 42 ]. This is consistent with our finding that the co-occurrence of a longer duration of diabetes and H. pylori infection further increased the risk of diabetic nephropathy. However, we found no elevated risk of diabetic nephropathy in patients who had H. pylori infection with an elevated TG level (≥ 1.7 mmol/L). This may be because of our small sample size, in that only 41 of our patients (4.3%) had both H. pylori infection and an elevated TG level. We therefore suggest that future studies with larger samples examine this relationship.

A recent systematic review and meta-analysis concluded that infection by H. pylori was associated with atherosclerosis [ 43 ]. An earlier study by Hamed et al. demonstrated that the prevalence of PVD was significantly greater in H. pylori -positive patients who had diabetes, and that this effect may be mediated by the increased levels of inflammatory cytokines [ 16 ]. Even though we found no evidence that H. pylori infection affected the risk for PVD, we did find that T2D patients who had H. pylori infection combined with an age of 56 years or more or with a history of smoking had a much higher risk of PVD. Therefore, we suggest that physicians should consider the use of H. pylori eradication therapy to reduce the risk of PVD in diabetes patients who are elderly or have a history of smoking. If these patients are positive for H. pylori , then eradication therapy should be implemented.

This study had two major strengths. First, to our best knowledge, this is the first study to demonstrate that the combination of H. pylori infection with traditional risk factors increased the risk for complications of T2D. This finding may be helpful for improving the treatments and outcomes of patients with T2D complications in clinical settings. Second, we had a relatively large sample of T2D patients, all of whom were tested for H. pylori infection, and we adjusted for the major traditional risk factors in our statistical analyses, and this increased the reliability of the results.

Nevertheless, there were also some limitations in our study. First, because this was a retrospective observational study, we could only evaluate the significance of associations, and could not identify causal relationships. Second, all patients were from a single medical center, and therefore might not be representative of the general population of China. Therefore, large, multicenter, prospective studies of this topic are warranted. Third, there were differences in the sensitivity and specificity of the three different tests used to diagnose H. pylori infection, and these differences could have affected the reported associations. Fourth, vacuolating cytotoxin-A (VacA) and cytotoxin-associated gene A protein (CagA) play a role in the pathogenesis of H. pylori -related diseases. Due to the lack of VacA/CagA data, we are unable to determine the association of these virulence factors with the major complications of diabetes. Fifth, although our control diabetic group was H. pylori -negative, these individuals had many gastric alterations, such as atrophic gastritis and peptic ulcer. Comparison with a diabetic control group that had no gastric alterations would likely lead to more significant findings. Finally, we did not assess the effect of H. pylori eradication and medication on diabetic nephropathy because we did not have access to follow-up data.

In conclusion, our results suggest that H. pylori infection of patients with T2D is an independent risk factor for diabetic nephropathy. We also found that the co-occurrence of H. pylori infection with hypertension, with diabetes duration of 9 years or more, and with an HbA1c level of 8% or more further increased the risk for diabetic nephropathy. We suggest that clinicians should pay more attention to T2D patients with H. pylori infections to better prevent the progression to diabetic nephropathy, and should also consider H. pylori eradication therapy to prevent or slow the development of diabetic nephropathy.

Data availability

It is possible to access the data after coordination with the corresponding author by email.

Abbreviations

• Helicobacter pylori
• Type-2 diabetes

Confidence interval

Glycated hemoglobin A1c

Peripheral vascular disease

13 C-urea breath test

Rapid urease test

Body mass index

Total cholesterol

High-density lipoprotein cholesterol

Low-density lipoprotein cholesterol,

Triglycerides

Standard deviation

Interquartile range

Receiver operating characteristic curves

Tumor necrosis factor-α

Interleukin-1

Interleukin-6

Interleukin-8

Homocysteine

Nitric oxide

Nuclear factor-kappa B

Vacuolating cytotoxin A

Cytotoxin-associated gene-A

Martin-Nunez GM, Cornejo-Pareja I, Clemente-Postigo M, Tinahones FJ. Gut microbiota: the Missing Link between Helicobacter pylori infection and metabolic disorders? Front Endocrinol (Lausanne). 2021;12:639856. https://doi.org/10.3389/fendo.2021.639856 .

Article   PubMed   Google Scholar  

Pellicano R, Ianiro G, Fagoonee S, Settanni CR, Gasbarrini A. Review: extragastric diseases and Helicobacter pylori. Helicobacter. 2020;25(Suppl 1):e12741. https://doi.org/10.1111/hel.12741 .

Cho NH, Shaw JE, Karuranga S, Huang Y, da Rocha Fernandes JD, Ohlrogge AW, et al. IDF Diabetes Atlas: global estimates of diabetes prevalence for 2017 and projections for 2045. Diabetes Res Clin Pract. 2018;138:271–81. https://doi.org/10.1016/j.diabres.2018.02.023 .

Article   CAS   PubMed   Google Scholar  

Faselis C, Katsimardou A, Imprialos K, Deligkaris P, Kallistratos M, Dimitriadis K. Microvascular complications of type 2 diabetes Mellitus. Curr Vasc Pharmacol. 2020;18(2):117–24. https://doi.org/10.2174/1570161117666190502103733 .

Viigimaa M, Sachinidis A, Toumpourleka M, Koutsampasopoulos K, Alliksoo S, Titma T. Macrovascular complications of type 2 diabetes Mellitus. Curr Vasc Pharmacol. 2020;18(2):110–6. https://doi.org/10.2174/1570161117666190405165151 .

Mansori K, Dehghanbanadaki H, Naderpour S, Rashti R, Moghaddam AB, Moradi Y. A systematic review and meta-analysis of the prevalence of Helicobacter pylori in patients with diabetes. Diabetes Metab Syndr. 2020;14(4):601–7. https://doi.org/10.1016/j.dsx.2020.05.009 .

Mansori K, Moradi Y, Naderpour S, Rashti R, Moghaddam AB, Saed L, et al. Helicobacter pylori infection as a risk factor for diabetes: a meta-analysis of case-control studies. BMC Gastroenterol. 2020;20(1):77. https://doi.org/10.1186/s12876-020-01223-0 .

Article   PubMed   PubMed Central   Google Scholar  

Hsieh MC, Wang SS, Hsieh YT, Kuo FC, Soon MS, Wu DC. Helicobacter pylori infection associated with high HbA1c and type 2 diabetes. Eur J Clin Invest. 2013;43(9):949–56. https://doi.org/10.1111/eci.12124 .

Chen J, Xing Y, Zhao L, Ma H. The Association between Helicobacter pylori Infection and glycated hemoglobin A in diabetes: a Meta-analysis. J Diabetes Res. 2019;2019:3705264. https://doi.org/10.1155/2019/3705264 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Kim KJ, Choi J, Bae JH, Kim KJ, Yoo HJ, Seo JA, et al. Time to Reach Target Glycosylated Hemoglobin is Associated with long-term durable Glycemic Control and Risk of Diabetic complications in patients with newly diagnosed type 2 diabetes Mellitus: a 6-Year observational study. Diabetes Metab J. 2021;45(3):368–78. https://doi.org/10.4093/dmj.2020.0046 .

Song X, Cai C, Jin Q, Chen X, Yu C. The efficacy of Helicobacter pylori eradication in diabetics and its effect on glycemic control: a systematic review and meta-analysis. Helicobacter. 2021;26(2):e12781. https://doi.org/10.1111/hel.12781 .

Cheng KP, Yang YJ, Hung HC, Lin CH, Wu CT, Hung MH et al. Helicobacter pylori eradication improves glycemic control in type 2 diabetes patients with asymptomatic active Helicobacter pylori infection. J Diabetes Investig. 2019;10(4):1092 – 101.https://doi.org/0.1111/jdi.12991.

Kim WS, Choi Y, Kim N, Lim SH, Noh G, Kim KW, et al. Long-term effect of the eradication of Helicobacter pylori on the hemoglobin A1c in type 2 diabetes or prediabetes patients. Korean J Intern Med. 2022;37(3):579–90. https://doi.org/10.3904/kjim.2021.194 .

Demir M, Gokturk HS, Ozturk NA, Kulaksizoglu M, Serin E, Yilmaz U. Helicobacter pylori prevalence in diabetes mellitus patients with dyspeptic symptoms and its relationship to glycemic control and late complications. Dig Dis Sci. 2008;53(10):2646–9. https://doi.org/10.1007/s10620-007-0185-7 .

Kayar Y, Pamukcu O, Eroglu H, Kalkan Erol K, Ilhan A, Kocaman O. Relationship between Helicobacter pylori infections in Diabetic patients and inflammations, metabolic syndrome, and complications. Int J Chronic Dis. 2015;2015:290128. https://doi.org/10.1155/2015/290128 .

Hamed SA, Amine NF, Galal GM, Helal SR, Tag El-Din LM, Shawky OA et al. Vascular risks and complications in diabetes mellitus: the role of helicobacter pylori infection. J Stroke Cerebrovasc Dis. 2008;17(2):86–94. https://doi.org/0.1016/j.jstrokecerebrovasdis.2007.10.006.

Agrawal RP, Sharma R, Garg D, Pokharna R, Kochar DK, Kothari RP. Role of Helicobacter pylori in causation of diabetic gastropathies and non-gastrointestinal complications in type 2 diabetes. J Indian Med Assoc. 2010;108(3):140–3. PMID: 21043350.

CAS   PubMed   Google Scholar  

Wang F, Fu Y, Lv Z. Association of Helicobacter pylori infection with diabetic complications: a meta-analysis. Endocr Res. 2014;39(1):7–12. https://doi.org/10.3109/07435800.2013.794426 .

Wang F, Liu J, Lv Z. Association of Helicobacter pylori infection with diabetes mellitus and diabetic nephropathy: a meta-analysis of 39 studies involving more than 20,000 participants. Scand J Infect Dis. 2013;45(12):930–8. https://doi.org/10.3109/00365548.2013.844351 .

Zhou F, Zhong X, Chen J, Li C, Shang M, Jiang C, et al. Helicobacter pylori infection associated with type 2 diabetic nephropathy in patients with dyspeptic symptoms. Diabetes Res Clin Pract. 2015;110(3):328–34. https://doi.org/10.1016/j.diabres.2015.09.008 .

de Luis DA, de la Calle H, Roy G, de Argila CM, Valdezate S, Canton R, et al. Helicobacter pylori infection and insulin-dependent diabetes mellitus. Diabetes Res Clin Pract. 1998;39(2):143–6. https://doi.org/10.1016/s0168-8227(97)00127-7 .

Schimke K, Chubb SA, Davis WA, Davis TM. Helicobacter pylori cytotoxin-associated gene-A antibodies do not predict complications or death in type 2 diabetes: the Fremantle Diabetes Study. Atherosclerosis. 2010;212(1):321–6. https://doi.org/10.1016/j.atherosclerosis.2010.05.021 .

Pyo JH, Lee H, Choi SC, Cho SJ, Choi YH, Min YW, et al. Lack of Association between Past Helicobacter pylori infection and diabetes: a two-cohort study. Nutrients. 2019;11(8). https://doi.org/10.3390/nu11081874 .

Alberti KG, Zimmet PZ. Definition, diagnosis and classification of diabetes mellitus and its complications. Part 1: diagnosis and classification of diabetes mellitus provisional report of a WHO consultation. Diabet Med. 1998;15(7):539–53. https://doi.org/10.1002/(SICI)1096-9136(199807)15:7%3C;539::AID-DIA668%3E;3.0.CO;2-S .

Tuttle KR, Bakris GL, Bilous RW, Chiang JL, de Boer IH, Goldstein-Fuchs J, et al. Diabetic kidney disease: a report from an ADA Consensus Conference. Diabetes Care. 2014;37(10):2864–83. https://doi.org/10.2337/dc14-1296 .

American Diabetes Association Professional Practice C. 12. Retinopathy, Neuropathy, and Foot Care: Standards of Medical Care in Diabetes-2022. Diabetes Care. 2022;45(Suppl 1):S185-S94. https://doi.org/10.2337/dc22-S012

Carmichael J, Fadavi H, Ishibashi F, Shore AC, Tavakoli M. Advances in screening, early diagnosis and Accurate Staging of Diabetic Neuropathy. Front Endocrinol (Lausanne). 2021;12:671257. https://doi.org/10.3389/fendo.2021.671257 .

Chin JA, Sumpio BE. Diabetes mellitus and peripheral vascular disease: diagnosis and management. Clin Podiatr Med Surg. 2014;31(1):11–26. https://doi.org/10.1016/j.cpm.2013.09.001 .

Blonde L, Umpierrez GE, Reddy SS, McGill JB, Berga SL, Bush M, et al. American Association of Clinical Endocrinology Clinical Practice Guideline: developing a diabetes Mellitus Comprehensive Care Plan-2022 Update. Endocr Pract. 2022;28(10):923–1049. https://doi.org/10.1016/j.eprac.2022.08.002 .

Samsu N. Diabetic Nephropathy: challenges in Pathogenesis, diagnosis, and treatment. Biomed Res Int. 2021;2021:1497449. https://doi.org/10.1155/2021/1497449 .

Navarro-Gonzalez JF, Mora-Fernandez C, Muros de Fuentes M, Garcia-Perez J. Inflammatory molecules and pathways in the pathogenesis of diabetic nephropathy. Nat Rev Nephrol. 2011;7(6):327–40. https://doi.org/10.4103/0366-6999.245269 .

Shi Y, Duan JY, Liu DW, Qiao YJ, Han QX, Pan SK, Helicobacter pylori Infection is Associated with Occurrence of Proteinuria in Type 2 Diabetes Patients: A Systemic Review and Meta-Analysis. Chin Med, Engl J et al.). https://doi.org/2018;131(22):2734-40. https://doi.org/10.4103/0366-6999.245269

Lim SH, Kim N, Kwon JW, Kim SE, Baik GH, Lee JY, et al. Positive Association between Helicobacter pylori Infection and metabolic syndrome in a Korean Population: a Multicenter Nationwide Study. Dig Dis Sci. 2019;64(8):2219–30. https://doi.org/10.1007/s10620-019-05544-3 .

King GL. The role of inflammatory cytokines in diabetes and its complications. J Periodontol. 2008;79(8 Suppl):1527–34. https://doi.org/10.1902/jop.2008.080246 .

Esse R, Barroso M, Tavares de Almeida I, Castro R. The contribution of Homocysteine Metabolism disruption to endothelial dysfunction: state-of-the-art. Int J Mol Sci. 2019;20(4). https://doi.org/10.3390/ijms20040867 .

Cohen E, Levi A, Vecht-Lifshitz SE, Goldberg E, Garty M, Krause I. Assessment of a possible link between hyperhomocysteinemia and hyperuricemia. J Investig Med. 2015;63(3):534–8. https://doi.org/10.1097/JIM.0000000000000152 .

Lopes de Faria JB, Silva KC, Lopes de Faria JM. The contribution of hypertension to diabetic nephropathy and retinopathy: the role of inflammation and oxidative stress. Hypertens Res. 2011;34(4):413 – 22. https://doi.org/413-422.10.1038/hr.2010.263.

Chen M, Huang X, Gao M, Yang Z, Fang Z, Wei J, et al. Helicobacter pylori promotes inflammatory factor secretion and lung injury through VacA exotoxin-mediated activation of NF-kappaB signaling. Bioengineered. 2022;13(5):12760–71. https://doi.org/10.1080/21655979.2022.2071011 .

Kostov K, Blazhev A. Use of Glycated Hemoglobin (A1c) as a biomarker for vascular risk in type 2 diabetes: its relationship with Matrix Metalloproteinases-2, -9 and the metabolism of collagen IV and Elastin. Med (Kaunas). 2020;56(5). https://doi.org/10.3390/medicina56050231 .

Tommerdahl KL, Shapiro ALB, Nehus EJ, Bjornstad P. Early microvascular complications in type 1 and type 2 diabetes: recent developments and updates. Pediatr Nephrol. 2022;37(1):79–93. https://doi.org/10.1007/s00467-021-05050-7 .

Wan Z, Song L, Hu L, Hu M, Lei X, Huang Y, et al. Helicobacter pylori infection is associated with diabetes among Chinese adults. J Diabetes Investig. 2020;11(1):199–205. https://doi.org/10.1111/jdi.13102 .

Jeon CY, Haan MN, Cheng C, Clayton ER, Mayeda ER, Miller JW, et al. Helicobacter pylori infection is associated with an increased rate of diabetes. Diabetes Care. 2012;35(3):520–5. https://doi.org/10.2337/dc11-1043 .

Shi H, Li Y, Dong C, Si G, Xu Y, Peng M, et al. Helicobacter pylori infection and the progression of atherosclerosis: a systematic review and meta-analysis. Helicobacter. 2022;27(1):e12865. https://doi.org/10.1111/hel.12865 .

Download references

Acknowledgements

Not applicable.

The Top Talent of Changzhou “The 14th Five-Year Plan” High-Level Health Talents Training Project, NO. 2022CZBJ051; Clinical Research Project of Changzhou Medical Center of Nanjing Medical University (CMCC202309); Changzhou commission of health Science and Technology Project (ZD202336).

Author information

Zhuoya Li and Jie Zhang should be considered joint first author.

Authors and Affiliations

Department of Gastroenterology, The Affiliated Changzhou No. 2 People’s Hospital of Nanjing Medical University, Changzhou Medical Center, Nanjing Medical University, 29 Xinglong Lane, Tianning District, Changzhou, China

Zhuoya Li, Yizhou Jiang, Kai Ma, Cheng Cui & Xiaoyong Wang

Department of Endocrinology, The Affiliated Changzhou No. 2 People’s Hospital of Nanjing Medical University, Changzhou Medical Center, Nanjing Medical University, Changzhou, Jiangsu Province, 213000, China

You can also search for this author in PubMed   Google Scholar

Contributions

Z. L. and J. Z. : acquisition of data, analysis and interpretation of data, drafting the article. Y.J. and K. M. : acquisition of data, analysis and interpretation of data. C. C. : interpretation of data, revising the article. X. W. : conception and design of the study, critical revision, analysis and interpretation of data, final approval.All authors read and approved the final manuscript.

Corresponding author

Correspondence to Xiaoyong Wang .

Ethics declarations

Ethics approval and consent to participate.

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000. This study was approved by the Institutional Review Board of Changzhou No. 2 People’s Hospital, Affiliated with Nanjing Medical University, Changzhou, China ([2022] KY010-01). Since the present study is a retrospective analysis of patient data, the need for informed consent was waived by the Institutional Review Board of Changzhou No. 2 People’s Hospital, Affiliated with Nanjing Medical University.

Consent for publication

Not Applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Li, Z., Zhang, J., Jiang, Y. et al. Association of Helicobacter pylori infection with complications of diabetes: a single-center retrospective study. BMC Endocr Disord 24 , 152 (2024). https://doi.org/10.1186/s12902-024-01678-2

Download citation

Received : 03 January 2024

Accepted : 02 August 2024

Published : 14 August 2024

DOI : https://doi.org/10.1186/s12902-024-01678-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Complication
• Nephropathy

BMC Endocrine Disorders

ISSN: 1472-6823

• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 13 August 2024

Impact of neoadjuvant chemotherapy on perioperative immune function in breast cancer patients: a propensity score-matched retrospective study

• Qi-Hua Jiang 1   na1 ,
• Hai Hu 1 , 2   na1 ,
• Zhi-Hong Xu 1 ,
• Peng Duan 3 , 4 ,
• Zhi-Hua Li 1 , 3 &
• Jun-Tao Tan 1 , 3  

Scientific Reports volume  14 , Article number:  18738 ( 2024 ) Cite this article

204 Accesses

Metrics details

• Medical research

To evaluate the impact of neoadjuvant chemotherapy on perioperative immune function in breast cancer patients, focusing on CD3 + , CD4 + , CD8 + , and natural killer (NK) cells, as well as the CD4 + /CD8 + ratio. We retrospectively reviewed medical records of breast cancer patients who underwent surgery with or without neoadjuvant chemotherapy at our medical center from January 2020 to December 2022. Patients were matched 1:1 based on propensity scores. Immune cell proportions and the CD4 + /CD8 + ratio were compared on preoperative day one and postoperative days one and seven. Among matched patients, immune cell proportions and the CD4 + /CD8 + ratio did not significantly differ between those who received neoadjuvant chemotherapy and those who did not at any of the three time points. Similar results were observed in chemotherapy-sensitive patients compared to the entire group of patients who did not receive neoadjuvant chemotherapy. However, chemotherapy-insensitive patients had significantly lower proportions of CD4 + and NK cells, as well as a lower CD4 + /CD8 + ratio, at all three time points compared to patients who did not receive neoadjuvant chemotherapy. Neoadjuvant chemotherapy may impair immune function in chemotherapy-insensitive patients, but not in those who are sensitive to the treatment.

Similar content being viewed by others

Pan-immune-inflammation value and survival in patients with breast cancer from a Peruvian reference hospital

Low pan-immune-inflammation-value predicts better chemotherapy response and survival in breast cancer patients treated with neoadjuvant chemotherapy

Enhanced immune response outperform aggressive cancer biology and is associated with better survival in triple-negative breast cancer

Introduction.

Breast cancer is one of the most common malignancies in women, and its incidence is increasing 1 . Although surgery is effective for many patients 2 , the prognosis may be poor for those with specific molecular signatures, larger tumors, or lymph node involvement 3 , 4 . Neoadjuvant chemotherapy, also known as preoperative or induction chemotherapy 5 , 6 , 7 , 8 , is often given to breast cancer patients to reduce tumor size, prevent spread, and improve the chances of a favorable prognosis after surgery 6 , 9 .

Neoadjuvant chemotherapy has several roles in breast cancer management 10 , 11 , 12 , 13 . Firstly, it aims to shrink the tumor, facilitating surgical resection and possibly enabling breast-conserving surgery in patients with initially unresectable tumors 14 . Secondly, it targets micrometastases, reducing the risk of distant metastasis and improving long-term survival 15 . Additionally, neoadjuvant chemotherapy can help assess tumor response to treatment, guiding further therapeutic decisions and prognosis 16 . The principles and mechanisms of neoadjuvant chemotherapy include tumor shrinkage, minimization of dissemination, and potential enhancement of surgical outcomes. Furthermore, neoadjuvant chemotherapy may affect immune cells, specifically CD4 + and CD8 + cells, potentially altering immune function and anti-tumor responses 17 . Understanding these effects is crucial for our study’s rationale and elucidating the implications of neoadjuvant chemotherapy on immune parameters.

However, not all breast cancer patients exhibit a favorable response to initial chemotherapy, underscoring the challenge of chemotherapy resistance 18 . Tumor heterogeneity, genetic variations, and immune responses are contributing factors to this resistance 19 , 20 . Understanding these mechanisms is vital for optimizing treatment strategies. Despite its benefits, neoadjuvant chemotherapy may harm normal tissue and immune cells in the body 21 . Specifically, it may decrease the counts of CD4 + and CD8 + T cells in peripheral blood, thereby weakening the patient's anti-tumor responses 22 , 23 and their basic immune defenses against postoperative infection 24 .

Here, our aim was to investigate and validate the potential adverse effects of neoadjuvant chemotherapy on perioperative immune function in breast cancer patients treated at our medical center.

Materials and methods

Study population.

A retrospective analysis was performed on the medical records of 622 breast cancer patients who underwent surgery at the Third Hospital of Nanchang, China, from January 2020 to December 2022. Patients were divided into two groups: those who received surgery following neoadjuvant chemotherapy (Group A) and those who underwent surgery without prior neoadjuvant chemotherapy (Group B). Due to the retrospective nature of the study, the Ethics Committee of The Third Hospital of Nanchang waived the need for obtaining informed consent. Nonetheless, this study, identified by the ethics review number K-lw2023002, received approval from the Ethics Committee of The Third Hospital of Nanchang, thus ensuring compliance with ethical guidelines and patient privacy.

Stringent eligibility and exclusion criteria were established to ensure the homogeneity and safety of the study population. Eligibility criteria included patients under 70 years old, confirmed with histopathological diagnosis of breast cancer from surgical samples. Disease staging (I-III) was determined through comprehensive evaluation, including breast and axillary ultrasonography with biopsy, mammography, or CT scans of cranial, thoracic, and abdominal regions. Positron emission tomography (PET) imaging was used in certain cases to aid staging. Moreover, patients needed to demonstrate sufficient hematological, renal, hepatic, and pulmonary functions to tolerate interventions effectively. Conversely, exclusion criteria included patients intolerant to general anesthesia, those with distant metastases, or concurrent autoimmune diseases or other malignancies. Patients with a history of chemotherapy, radiotherapy, or other antitumor therapies were also excluded to isolate the effects of neoadjuvant chemotherapy. Furthermore, pregnant or lactating individuals were excluded due to potential risks associated with chemotherapy exposure. These comprehensive criteria aimed to ensure the validity, safety, and homogeneity of the study cohort.

Staging definitions

In this study, tumor staging was assessed utilizing the TNM classification system, with "T" representing primary tumor size and extent, "N" indicating regional lymph node involvement, and "M" indicating distant metastases. Specifically, the "T" category includes T0 (no evidence of primary tumor), T1 (tumor size ≤ 2 cm), T2 (tumor size > 2 cm but ≤ 5 cm), T3 (tumor size > 5 cm), and T4 (tumor of any size with direct extension to the chest wall or skin). The "N" category is further divided into N0 (no regional lymph node metastasis), N1 (metastasis to movable ipsilateral level I, II axillary lymph nodes), N2 (metastasis to ipsilateral level III axillary lymph nodes or infraclavicular lymph nodes), and N3 (metastasis to ipsilateral internal mammary lymph nodes) 25 .

Patients in Group A received surgery in conjunction with neoadjuvant chemotherapy. The primary chemotherapy regimens were as follows, with detailed dosages and cycles: Cyclophosphamide + Epirubicin + Fluorouracil (CEF): Cyclophosphamide was administered at a dose of 600 mg/m 2 , Epirubicin at 90 mg/m 2 , and Fluorouracil at 600 mg/m 2 . These were administered on day 1 of each 28-day cycle, with most patients receiving 2–4 cycles before surgery, which was performed 2-4 weeks after the last chemotherapy course. CEF-docetaxel (CEF-T): This regimen consisted of the CEF components as described above, with the addition of Docetaxel administered at a dose of 75 mg/m 2 on day 1 of each cycle. The regimen was administered over 28-day cycles, with most patients receiving 2-4 cycles. Cyclophosphamide + Methotrexate + Fluorouracil + Docetaxel (CMF-T): Cyclophosphamide was administered at 600 mg/m 2 , Methotrexate at 40 mg/m 2 , Fluorouracil at 600 mg/m 2 , and Docetaxel at 75 mg/m 2 . These were administered on day 1 of each 28-day cycle, similar to the CEF-T regimen, with 2-4 cycles being most common.

Adjustment Protocols for Chemotherapy Regimens: In cases of significant toxicity (e.g., neutropenia, anemia, or liver function abnormalities), dosages were adjusted according to standardized guidelines, which included dose reduction or cycle extension as warranted 26 . Treatment could be paused for severe adverse reactions, with continuation or further adjustment based on a comprehensive evaluation by the medical team. Close monitoring, including hematologic, liver, and kidney function tests, as well as cardiac function assessments, was conducted throughout the chemotherapy period to ensure patient safety and treatment efficacy. Patients in Group B underwent surgery without any prior chemotherapy.

To adjust for preoperative clinicodemographic differences between patients, who received neoadjuvant chemotherapy or not, we took the two groups of patients and matched them 1:1 to each other using propensity scoring based on age, body mass index, clinical T and N stages, tumor pathology subtype, hormone status, Her2 status, and Ki-67 status.

The score-matched patients with or without neoadjuvant chemotherapy were compared based on percentages of CD3 + , CD4 + , CD8 + , and the CD4 + /CD8 + ratio on preoperative day 1 and postoperative days 1 and 7. Comparisons were also made between the score-matched groups regarding clinicodemographic profiles and postoperative complications.

Finally, we analyzed the diverse outcomes in subgroups of score-matched patients, who received neoadjuvant chemotherapy and, based on the tumor response, were classified as sensitive or insensitive. Patients were classified as sensitive if they showed partial or complete tumor response based on a previously described tumor response grading system 27 ; otherwise, they were classified as insensitive.

Flow cytometry

Flow cytometry was utilized to quantify the proportions of CD4 + and CD8 + T lymphocytes in peripheral blood samples collected from patients on preoperative day 1 and postoperative days 1 and 7. Peripheral venous blood samples (10 mL) were drawn into EDTA-coated tubes and processed for peripheral blood mononuclear cell (PBMC) isolation using Ficoll-Paque PLUS (GE Healthcare). Blood samples were diluted 1:1 with phosphate-buffered saline (PBS) and layered over Ficoll-Paque, followed by centrifugation at 400 × g for 30 minutes at room temperature. The PBMC layer was carefully harvested, washed twice with PBS, and resuspended in RPMI-1640 medium supplemented with 10% fetal bovine serum (FBS). For flow cytometry staining, 1 × 10 6 PBMCs were incubated with Fc receptor blocking solution (Human TruStain FcX, BioLegend) for 10 minutes at 4°C to minimize nonspecific binding, and subsequently stained with APC-conjugated anti-CD3, FITC-conjugated anti-CD4, and PE-conjugated anti-CD8 monoclonal antibodies (BD Biosciences) for 30 minutes at 4 °C in the dark. Post-staining, cells were washed with PBS containing 2% FBS and resuspended in 500 μL of PBS for analysis. Data acquisition was performed using a BD FACSCanto II flow cytometer, with instrument settings optimized using unstained and single-stained controls for compensation, and fluorescence minus one (FMO) controls to set gating thresholds. A minimum of 10,000 events were recorded per sample. Data analysis was conducted using FlowJo software (version 10.7.1), where lymphocytes were gated based on forward and side scatter properties, followed by identification of CD3 + T cells and subsequent gating of CD4 + and CD8 + subsets within the CD3 + population. To ensure reproducibility and accuracy, all samples were processed and analyzed in triplicate, with inter-assay and intra-assay variations monitored by including a standard control sample in each run, maintaining a coefficient of variation (CV) below 10%. Detailed protocols for flow cytometry procedures can be referenced from previously reported studies 28 .

Statistical analysis

Continuous data were presented as mean ± standard deviation (SD) if normally distributed, or median (range) if skewed. Categorical data were presented as counts (percentages). Differences between groups were assessed using appropriate statistical tests. Propensity score matching (PSM) was performed to minimize potential confounding factors, based on age, BMI, clinical T stage, and N stage. Post-PSM, these characteristics were comparable between the groups. One-way ANOVA was used to compare changes in T lymphocyte and natural killer cell subsets across different time points within the same group. Independent samples t-test was used to compare differences between groups at the same time point. Paired samples t-test was used to compare changes within the same group at different time points. Bonferroni correction was applied to control for type I errors in multiple comparisons. Statistical significance was defined as P ≤ 0.05. All analyses were performed using SPSS 17.0 (IBM, Chicago, IL, USA).

Ethics statement

The subjects included in this study were human participants, and the research was approved by the Ethics Committee of the Third Hospital of Nanchang City (Ethics review number: K-lw2023002).

Patient demographics and clinical characteristics

A total of 439 patients were ultimately included in the analysis (Fig. 1 ). Among them, 119 (27.1%) received neoadjuvant chemotherapy, while the majority, 320 (72.9%), did not. The clinicodemographic characteristics of study participants before and after matching are presented in Table 1 . Prior to propensity score matching, patients who received neoadjuvant chemotherapy were notably younger, had lower body mass index, and exhibited more advanced clinical T and N stages compared to those who did not. Following propensity score matching, there were no significant differences observed across any of the examined clinicopathologic characteristics between the two patient groups: those who underwent neoadjuvant chemotherapy (Group A, n=106) and those who did not (Group B, n=106).

Study flowchart. NCT, neoadjuvant chemotherapy.

Immune cell analysis

No significant differences in immune cell percentages (Table 2 , Fig. 2 ) were observed between Group A and Group B at any time point examined. In both groups, the percentages of CD3 + , CD4 + , and the CD4 + /CD8 + ratio decreased from preoperative day 1 to postoperative day 1, but reverted to preoperative levels by postoperative day 7. We hypothesize that the initial decrease in CD3 + , CD4 + , CD8 + levels, and the CD4 + /CD8 + ratio following neoadjuvant chemotherapy may be attributed to the cytotoxic effects of the treatment on immune cells, particularly T lymphocytes. Subsequently, the observed increase could potentially be due to the recovery of immune function or compensatory mechanisms triggered by the body in response to the initial depletion.

The proportions of ( A ) CD3+ cells, ( B ) CD4+ cells, ( C ) CD8+ cells, and the ratio of ( D ) CD4+ to CD8+ cells were assessed at various time points in patients who received neoadjuvant chemotherapy (Group A, n = 106) and those who did not (Group B, n = 106). The timepoints "-1", "1", and "7" correspond, respectively, to preoperative day 1, postoperative day 1, and postoperative day 7. Values are mean ± SD. NCT, neoadjuvant chemotherapy; ns, not significant.

Subgroup analysis within group A

Patients in Group A were further divided into two categories based on their response to chemotherapy: Group A sensitive (n = 70) and Group A insensitive (n = 36). The sensitive subgroup in Group A showed no significant differences in immune cell percentages compared to the entirety of Group B across all three time points (Table 4 , Fig. 3 ). However, they exhibited significantly lower T and N stages (Table 3 ).

The proportions of ( A ) CD3 + cells, ( B ) CD4 + cells, ( C ) CD8 + cells, ratio of ( D ) CD4 + to CD8 + cells were measured at different time points in all patients who did not receive NCT (Group B, n =106) and in patients who received NCT. Patients treated by NCT were divided into those sensitive (Group A sensitive, n = 70) or insensitive (Group A insensitive, n = 36) to NCT. The timepoints "-1", "1", and "7" correspond, respectively, to preoperative day 1, postoperative day 1, and postoperative day 7. Values are mean ± SD. NCT, neoadjuvant chemotherapy; ns, not significant. * P < 0.05, ** P < 0.01, *** P < 0.001.

Conversely, the insensitive subgroup in Group A exhibited significantly lower percentages of CD4 + cells and a reduced CD4 + /CD8 + ratio compared to Group B across all three time points (Table 4 , Fig. 3 B, D ). However, the percentages of CD3 + and CD8 + cells were similar between these two groups (Fig. 3 A, C), and there were no significant differences observed in clinical T or N stage (Table 3 ).

Postoperative complications

When comparing Group A with Group B, patients in Group A who were sensitive to chemotherapy exhibited a similar rate of postoperative complications. In contrast, patients in Group A who were insensitive to chemotherapy showed significantly higher rates of infection and fat necrosis compared to their sensitive counterparts (Table 3 ).

Our single-center retrospective study suggests that neoadjuvant chemotherapy may induce immunosuppression in breast cancer patients who are insensitive to the treatment, while having minimal impact on those who are sensitive to it. Our study revealed that neoadjuvant chemotherapy decreased the levels of CD4 + and CD8 + cells, as well as the CD4 + /CD8 + ratio in insensitive patients, suggesting an elevated risk of postoperative infection and a weakened anti-tumor response. Notably, insensitive patients in our study showed significantly higher rates of infection and fat necrosis than the entire group of patients who did not receive neoadjuvant chemotherapy.

The reliability of our study might exceed that of previous research because we matched patients who underwent neoadjuvant chemotherapy with those who did not based on propensity scoring. This scoring method took into account three key factors influencing the postoperative prognosis of breast cancer: age, body mass index, and clinical stage 29 , 30 , 31 . Before matching, these three variables differed significantly between patients in our study who received neoadjuvant chemotherapy or not, indicating that patients who receive such therapy tend to be younger and to have larger tumors and more aggressive disease. Our propensity score-matched patients who received neoadjuvant chemotherapy and those who did not did not differ significantly in postoperative complications, consistent with a previous report involving breast cancer patients 32 . However, our subgroup analysis suggests that such a conclusion may depend on whether patients are sensitive or insensitive to neoadjuvant chemotherapy. Indeed, our findings differ from those of a previous study that indicated that neoadjuvant chemotherapy could significantly reduce the percentages of CD3 + and CD4 + T cells while having no effect on the CD4 + /CD8 + ratio 33 , 34 . The difference between these findings and ours may be partly attributed to the sensitivity or insensitivity of patients to neoadjuvant chemotherapy 35 , 36 , 37 , 38 . One possibility is that in patients sensitive to neoadjuvant chemotherapy, the treatment weakens the tumor, thus preserving immune cells, while in patients insensitive to chemotherapy, the tumor burden remains constant or may even increase, resulting in immune function suppression. Further complicating this hypothesis is the potential toxicity of neoadjuvant chemotherapy to immune cells, leading to perioperative immunosuppression 39 , 40 , 41 . The overall impact of chemotherapy on breast cancer patients may involve a multifaceted interplay of various factors affecting immune function 42 , 43 , 44 , including disease characteristics and preoperative interventions like immunotherapy, nutritional support, and radiotherapy 45 , 46 , 47 .

This study highlights the complex interplay between neoadjuvant chemotherapy and immune function, particularly in chemotherapy-insensitive patients. The potential of this research lies in its ability to inform more personalized treatment strategies. By identifying patients who are at higher risk of immune suppression, we can better tailor supportive interventions to improve outcomes. However, significant knowledge gaps remain, particularly regarding the mechanisms underlying the differential impact of chemotherapy on immune function. Addressing these gaps will require multi-center studies with standardized treatment protocols and comprehensive documentation of patient characteristics. Future research should also explore the role of genetic factors and adjunctive therapies, such as immunotherapy, to enhance the understanding of how neoadjuvant chemotherapy affects immune function. Over the next five years, we anticipate a more nuanced approach to treatment that integrates these findings, ultimately leading to improved patient care and outcomes in breast cancer treatment.

The interpretation of our study must consider several important limitations. Firstly, we acknowledge that we did not fully account for variations in patients’ baseline health or lifestyle factors, such as diet and exercise, which could introduce bias into our outcomes. Healthier individuals may inherently have better surgical outcomes, independent of the treatment type they receive. Additionally, the lack of detailed records regarding other concurrent treatments or interventions limits our ability to rule out their potential impact on our findings. This includes any additional medications or supportive therapies that were not documented in our analysis, which could confound the results. Moreover, the variation in chemotherapy regimens among participants presents another significant challenge. This diversity complicates direct comparisons between groups and may influence the observed outcomes, making it difficult to attribute differences solely to neoadjuvant chemotherapy. Given these limitations, our results should be interpreted with caution. Future research should aim for larger, more controlled studies that thoroughly document baseline health status, lifestyle factors, and all treatments received. Such studies would provide clearer insights into the effects of neoadjuvant chemotherapy on surgical outcomes and help to better understand the complex interactions between these variables.

Despite these limitations, our study provides evidence that neoadjuvant chemotherapy can significantly immunosuppress breast cancer patients who do not respond to such chemotherapy. Future studies should verify and extend our findings by analyzing a broader range of immune and inflammatory indicators as well as recurrence and other outcomes during follow-up. Such work should also take into account the potential influence of genetic polymorphism related to immune function. It may be possible to identify prognostic indicators that can differentiate patients more likely to benefit from such chemotherapy or from immediate surgery.

Neoadjuvant chemotherapy potentially impairs immune function in chemotherapy-insensitive patients, but not in those sensitive to it.

Data availability

The primary data can be acquired from the corresponding authors in compliance with privacy and ethical constraints.

Toohey, K. et al. systematic review of multimodal prehabilitation in breast cancer. Breast Cancer Res. Treat. https://doi.org/10.1007/s10549-022-06759-1 (2022).

Article   PubMed Central   Google Scholar  

Al-Masri, M., Aljalabneh, B., Al-Najjar, H. & Al-Shamaileh, T. Effect of time to breast cancer surgery after neoadjuvant chemotherapy on survival outcomes. Breast Cancer Res. Treat. 186 , 7–13. https://doi.org/10.1007/s10549-020-06090-7 (2021).

Article   Google Scholar  

Morgan, J. L. et al. Breast cancer surgery in older women: Outcomes of the Bridging Age Gap in Breast Cancer study. Br J Surg 107 , 1468–1479. https://doi.org/10.1002/bjs.11617 (2020).

Rizzo, A., Cusmai, A., Acquafredda, S., Rinaldi, L. & Palmiotti, G. Ladiratuzumab vedotin for metastatic triple negative cancer: Preliminary results, key challenges, and clinical potential. Expert Opin. Investig. Drugs 31 , 495–498. https://doi.org/10.1080/13543784.2022.2042252 (2022).

Krug, D. & Loibl, S. Neoadjuvant chemotherapy for early breast cancer. Lancet Oncol. 19 , e129 (2018).

Tamirisa, N. & Hunt, K. K. Neoadjuvant chemotherapy, endocrine therapy, and targeted therapy for breast cancer: ASCO guideline. Ann. Surg. Oncol. 29 , 1489–1492 (2022).

Tian, H. et al. Platinum and taxane based adjuvant and neoadjuvant chemotherapy in early triple-negative breast cancer: A narrative review. Front. Pharmacol. 12 , 770663 (2021).

Xu, H. D. & Zhang, Y. Q. Evaluation of the efficacy of neoadjuvant chemotherapy for breast cancer using diffusion-weighted imaging and dynamic contrast-enhanced magnetic resonance imaging. Neoplasma J. Exp. Clin. Oncol. 64 , 430–436 (2017).

Google Scholar  

Inari, H., Teruya, N., Kishi, M., Horii, R. & Ohno, S. Survival in cytologically proven node-positive breast cancer patients with nodal pathological complete response after neoadjuvant chemotherapy. Cancers 12 , 2633 (2020).

Caputo, R. et al. Sacituzumab govitecan for the treatment of advanced triple negative breast cancer patients: A multi-center real-world analysis. Front. Oncol. 14 , 1362641. https://doi.org/10.3389/fonc.2024.1362641 (2024).

Rizzo, A. et al. KEYNOTE-522, IMpassion031 and GeparNUEVO: Changing the paradigm of neoadjuvant immune checkpoint inhibitors in early triple-negative breast cancer. Future Oncol. 18 , 2301–2309. https://doi.org/10.2217/fon-2021-1647 (2022).

Rizzo, A. et al. Discontinuation rate and serious adverse events of chemoimmunotherapy as neoadjuvant treatment for triple-negative breast cancer: A systematic review and meta-analysis. ESMO Open 8 , 102198. https://doi.org/10.1016/j.esmoop.2023.102198 (2023).

Sahin, T. K., Rizzo, A., Aksoy, S. & Guven, D. C. Prognostic significance of the royal marsden hospital (RMH) score in patients with cancer: A systematic review and meta-analysis. Cancers (Basel) https://doi.org/10.3390/cancers16101835 (2024).

Iwamoto, T., Kajiwara, Y., Zhu, Y. & Iha, S. Biomarkers of neoadjuvant/adjuvant chemotherapy for breast cancer. Chin. Clin. Oncol. 9 , 27. https://doi.org/10.21037/cco.2020.01.06 (2020).

Long-term outcomes for neoadjuvant versus adjuvant chemotherapy in early breast cancer: meta-analysis of individual patient data from ten randomised trials. Lancet Oncol 19 , 27-39, https://doi.org/10.1016/s1470-2045(17)30777-5 (2018).

Kaufmann, M. et al. Recommendations from an international expert panel on the use of neoadjuvant (primary) systemic treatment of operable breast cancer: New perspectives 2006. Ann. Oncol. 18 , 1927–1934. https://doi.org/10.1093/annonc/mdm201 (2007).

Bruni, D., Angell, H. K. & Galon, J. The immune contexture and Immunoscore in cancer prognosis and therapeutic efficacy. Nat. Rev. Cancer 20 , 662–680. https://doi.org/10.1038/s41568-020-0285-7 (2020).

Ding, S., Chen, X. & Shen, K. Single-cell RNA sequencing in breast cancer: Understanding tumor heterogeneity and paving roads to individualized therapy. Cancer Commun. (Lond.) 40 , 329–344. https://doi.org/10.1002/cac2.12078 (2020).

DeMichele, A., Yee, D. & Esserman, L. Mechanisms of resistance to neoadjuvant chemotherapy in breast cancer. N. Engl. J. Med. 377 , 2287–2289. https://doi.org/10.1056/NEJMcibr1711545 (2017).

Chang, H. L., Schwettmann, B., McArthur, H. L. & Chan, I. S. Antibody-drug conjugates in breast cancer: overcoming resistance and boosting immune response. J. Clin. Invest. https://doi.org/10.1172/jci172156 (2023).

Ding, L., Wang, L., Yin, J., Fan, Z. & He, Z. Effects of neoadjuvant chemotherapy on respiratory function in patients with breast cancer. Chin. J. Cancer Res. 32 , 36–42. https://doi.org/10.21147/j.issn.1000-9604.2020.01.05 (2020).

Garciamartinez, E. et al. Baseline CD4/CD8 tumor infiltrating lymphocytes (TIL) ratio predicts pathologic response to neoadjuvant chemotherapy (NC) in breast cancer. Cancer Res. 72 (1024), P3-06–15 (2012).

Handgraaf, S. M. et al. Regulatory T cells play a crucial part in radio-immunotherapy resistance of brain tumors. Nat. Cancer 4 , 590–591 (2023).

Itoi, N. et al. Infiltration of CD4, CD8, CD56, and Fox-P3-positive lymphocytes in breast carcinoma tissue after neoadjuvant chemotherapy with or without trastuzumab. Breast Dis. 38 , 57–65 (2019).

Amin, M. B. et al. The eighth edition AJCC cancer staging manual: Continuing to build a bridge from a population-based to a more “personalized” approach to cancer staging. CA Cancer J. Clin. 67 , 93–99. https://doi.org/10.3322/caac.21388 (2017).

Schipilliti, F. M. et al. Datopotamab deruxtecan: A novel antibody drug conjugate for triple-negative breast cancer. Heliyon 10 , e28385. https://doi.org/10.1016/j.heliyon.2024.e28385 (2024).

Meredith, K. L. et al. Pathologic response after neoadjuvant therapy is the major determinant of survival in patients with esophageal cancer. Ann. Surg. Oncol. 17 , 1159–1167 (2010).

Tan, J.-T. et al. Comparison of postoperative immune function in patients with thoracic esophageal cancer after video-assisted thoracoscopic surgery or conventional open esophagectomy. Int. J. Surg. 30 , 155–160. https://doi.org/10.1016/j.ijsu.2016.04.052 (2016).

Gennari, A. et al. Body mass index and prognosis of metastatic breast cancer patients receiving first-line chemotherapy. Cancer Epidemiol. Biomark. Prevent. 22 , 1862–1867 (2013).

Azim, H. A. & Partridge, A. H. Biology of breast cancer in young women. Breast Cancer Res. 16 , 427 (2014).

Kimbung, S., Loman, N. & Hedenfalk, I. Clinical and molecular complexity of breast cancer metastases. Semin. Cancer Biol. 35 , 85–95 (2015).

Lorentzen, T., Heidemann, L. N., Moller, S. & Bille, C. Impact of neoadjuvant chemotherapy on surgical complications in breast cancer: A systematic review and meta-analysis. Eur. J. Surg. Oncol. 48 , 44–52. https://doi.org/10.1016/j.ejso.2021.09.007 (2022).

Wang, W. H., Xu, H. Y., Zhao, Z. M., Zhang, G. M. & Lin, F. W. Dynamic and significant changes of T-cell subgroups in breast cancer patients during surgery and chemotherapy. Int. Immunopharmacol. 65 , 279–283. https://doi.org/10.1016/j.intimp.2018.09.039 (2018).

Costa, A. D., Vyrynen, S., Chawla, A., Zhang, J. & Nowak, J. A. Proc. Abstracts: AACR Virtual Special Conference on Pancreatic Cancer; September 29-30 , (2020).

Rizzo, A. et al. Peripheral neuropathy and headache in cancer patients treated with immunotherapy and immuno-oncology combinations: The MOUSEION-02 study. Expert Opin. Drug Metab. Toxicol. 17 , 1455–1466. https://doi.org/10.1080/17425255.2021.2029405 (2021).

Viscardi, G. et al. Comparative assessment of early mortality risk upon immune checkpoint inhibitors alone or in combination with other agents across solid malignancies: A systematic review and meta-analysis. Eur. J. Cancer 177 , 175–185. https://doi.org/10.1016/j.ejca.2022.09.031 (2022).

Comes, M. C. et al. Explainable 3D CNN based on baseline breast DCE-MRI to give an early prediction of pathological complete response to neoadjuvant chemotherapy. Comput. Biol. Med. 172 , 108132. https://doi.org/10.1016/j.compbiomed.2024.108132 (2024).

Fanizzi, A. et al. Prognostic power assessment of clinical parameters to predict neoadjuvant response therapy in HER2-positive breast cancer patients: A machine learning approach. Cancer Med. 12 , 20663–20669. https://doi.org/10.1002/cam4.6512 (2023).

Lundy, J., Iii, E. J. L., Hamilton, S. & Dvm, P. C. Halothane, surgery, immunosuppression and artificial pulmonary metastases. Cancer 41 , 827–830 (2015).

Napoletano, C. et al. Ovarian cancer cytoreduction induces changes in T cell population subsets reducing immunosuppression. J. Cell. Mol. Med. 14 , 2748–2759 (2010).

Benisovich, V. I., Silverman, L., Slifkin, R., Stone, N. & Cohen, E. Cisplatin-based chemotherapy in renal transplant recipients. A case report and a review of the literature. Cancer 77 , 160–163 (2015).

Strasser, A., Jost, P. J. & Nagata, S. The many roles of FAS receptor signaling in the immune system. Immunity 30 , 180–192 (2009).

Asano, Y. et al. Prediction of treatment responses to neoadjuvant chemotherapy in triple-negative breast cancer by analysis of immune checkpoint protein expression. J. Transl. Med. 16 , 87 (2018).

Zheng, Y., Chen, Z., Han, Y., Han, L. & Shen, L. Immune suppressive landscape in the human esophageal squamous cell carcinoma microenvironment. Nat. Commun. 11 , 6268 (2020).

Article   ADS   PubMed Central   Google Scholar  

Parkin, J. & Cohen, B. An overview of the immune system: The lancet. Lancet 357 , 629–653 (1997).

Venter, C. et al. Nutrition and the immune system: A complicated tango. Int. J. Appl. Mech. 12 , 818 (2020).

Lundholm, K., Daneryd, P., Bosaeus, I., Krner, U. & Lindholm, E. Palliative nutritional intervention in addition to cyclooxygenase and erythropoietin treatment for patients with malignant disease: Effects on survival, metabolism, and function. Cancer 100 , 1967–1977 (2010).

Download references

This study was supported by the Major Project of Nanchang Science and Technology (Contract Grant Number: HKZ2020-133-1 to Z Li).

Author information

These authors contributed equally: Qi-Hua Jiang and Hai Hu.

Authors and Affiliations

Department of Breast Surgery, The Third Hospital of Nanchang, No. 2, Xiangshan South Road, Xi hu District, Nanchang City, 330008, Jiangxi Province, China

Qi-Hua Jiang, Hai Hu, Zhi-Hong Xu, Zhi-Hua Li & Jun-Tao Tan

Department of General Surgery, The Third Hospital of Nanchang, Nanchang City, 330008, China

Jiangxi Province Key Laboratory of Breast Diseases, The Third Hospital of Nanchang, Nanchang City, 330008, China

Peng Duan, Zhi-Hua Li & Jun-Tao Tan

Department of Endocrinology, The Third Hospital of Nanchang, No. 2, Xiangshan South Road, Xihu District, Nanchang City, 330008, Jiangxi Province, China

You can also search for this author in PubMed   Google Scholar

Contributions

J.T., Z.L. and P.D. conceived the study. Q.J. and H.H. collected the data. Q.J. and Z.X. analyzed the data. Q.J. wrote the manuscript. J.T., Z.L. and P.D. revised the manuscript. H.H. conducted the statistical analysis. All authors contributed to the article and approved the submitted version.

Corresponding authors

Correspondence to Peng Duan , Zhi-Hua Li or Jun-Tao Tan .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Jiang, QH., Hu, H., Xu, ZH. et al. Impact of neoadjuvant chemotherapy on perioperative immune function in breast cancer patients: a propensity score-matched retrospective study. Sci Rep 14 , 18738 (2024). https://doi.org/10.1038/s41598-024-69546-6

Download citation

Received : 05 June 2024

Accepted : 06 August 2024

Published : 13 August 2024

DOI : https://doi.org/10.1038/s41598-024-69546-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Breast cancer
• Neoadjuvant chemotherapy
• Perioperative immune function
• CD4 + /CD8 + ratio
• Chemotherapy-sensitive patients

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.